Python for Data Engineering
This repo contains all the code used in the Python for Data Engineering Course
Code is free to use, if you want in-depth understanding of concepts then check out my python course designed to teach you how to think programmatically and perform data engineering tasks using python
Module 1: iPhone Data Analysis Project
Get right into it, don't waste time learning basics, and execute the complete project to boost your confidence.
Work on Python Data Analysis project on iPhone Apple Dataset, do some basic transformation and analysis.
Module 2: Python Basics
Build strong programming fundamentals by learning the basics of python
- Working with Strings
- Working with Numbers
- Understanding DataTypes
- Python Data Structures (List, Dictionary, Sets, Tuples)
- Working with Loops (For & While)
- Conditional Statements (IF-ELSE)
Module 3: Python Intermediate
Let's learn some intermediate concepts to build a strong foundation.
- List Compherention
- Understand Python Packages
- Exception Handling
- Working with a text file
- Working with Functions
- Lambda functions basics
Module 4: Python Advance
Let's learn some advanced topics and packages used by data engineer on a daily basis
- Py Package - Numpy
- Py Package - Pandas (Basics and Understanding Transformation with Project)
Module 5: Python for Data Engineering
Python for data engineering, learn how to work with different file formats and handle DateTime columns
- Working with DateTime Formats
- Working with different file formats (JSON, CSV, Excel, AVRO)
- Scrape Amazon Data Using Python
Module 6: Spotfiy Data Pipeline End-To-End Python Data Engineering Project
Implement Complete Data Pipeline Data Engineering Project using Spotify
- Integrating with Spotify API and extracting Data
- Deploying code on AWS Lambda for Data Extraction
- Adding trigger to run the extraction automatically
- Writing transformation function
- Building automated trigger on transformation function
- Store files on S3 properly
- Building Analytics Tables on data files using Glue and Athena