Project SummaryTo wrap up this class you will use your Python knowledge and skills to design, execute, and present the results of a Data ETL pipeline project. You will create original code and a written report summarizing the goals, methods, and results of your project.Step 1: Describe your projectThe project begins with an exploration of the New York Times API endpoints. You might have an idea about what kind of data you want to analyze or what kind of question you want to answer. But if the data you need is not available, then you are only setting yourself up for a very challenging – perhaps impossible – final project. Therefore, your first step is to identify an actionable goal, e.g., a question to answer, for your project by exploring the documentation for the New York Times API.Data I want to analyze:Books API-Provide a list of books by year-Provide best seller books based on the book category (category: sci-fi, romance, adventure etc.)along with the priceWith that question in mind, it’s time to sketch out each of the stages in your ETL pipeline. The main questions for which you need to provide answers are as follows:1. Where is the data I need? Be specific! Which API endpoint? Are the important pieces of data I need to extract from the API actually available from the endpoint(s) I have identified?2. What do I need to do to transform the raw data extracted from the API into something usable? You do not need to be writing code yet! In fact, thinking in Python terms at this stage could be counter-productive. Just think in human terms for now: What about your data needs to be transformed? What parts do you need to keep and what can you discard?* ETL projects often require multiple transform stages. For example, one transform stage that cleans and prepares the extracted data, and another that uses built-in Python and/or third-party libraries to transform and even generate new data.3. What will I load my data into, and how? Your raw data, once processed in your transform stages, is going to be loaded into a file or database depending on what software you plan to use to present your results. How should you store it? What format and structure should the data be stored in?For the first step in your project, describe the following:1. What is the goal or question of your project?2. Where is the data?3. What will you do to transform the data?4. How will you store the processed data?Step 2: Write and run your code exampleWhen you are considering how much code you need to write, keep in mind the following: “How much do I need to write?” is never a good question. Moreover, when it comes to code, less code is often better than more code. If you can achieve the same results with half of the code, that’s a good thing! In fact, the same is true when you’re writing prose but that’s outside the scope of this course.What matters is not how much code you write, but how much your code does.What does you code need to do? Your code needs to perform a Data ETL process. The rest is detail. For an idea of what I’ll be looking for, check out the project checklist that is (or will soon be) posted to Canvas.Checklist for your codeRequired. You will not pass this assignment without meeting these requirements:* Your code executes successfully* Your code extracts data from an external source* Your code transforms data from its raw form to something more directly useful* Your code loads data into some format, file, or system that a human can use to interpret your output.Important. Missing these elements or doing them poorly will substantially impact your grade to the negative; doing them well will substantially improve your grade.* Using file I/O or database operations to store data* Use of one or more third-party module(s)* Functionality (doing things in your code that demonstrate the ability to use Python constructs like variables, flow control, etc.)* Code structure, including the use of functions* Code comments and stylePlease make sure to create the code as simple as possible. Thank you!
