josephmachado/e2e_datapipeline_test

Stars
12
Rank 1,597,372 (Top 32 %)
Language
Python
Created almost 3 years ago
Updated over 1 year ago

josephmachado/e2e_datapipeline_test

josephmachado

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Example repo to create end to end tests for data pipeline.

beginner_de_project

Beginner data engineering project - batch edition

data_engineering_project_template

A template repository to create a data project with IAC, CI/CD, Data migrations, & testing

simple_dbt_project

Code for dbt tutorial

bitcoinMonitor

Near real time ETL to populate a dashboard.

online_store

End to end data engineering project

beginner_de_project_stream

Simple stream processing pipeline

spark_submit_airflow

Simple repo to demonstrate how to submit a spark job to EMR from Airflow

analytical_dp_with_sql

Code for my "Analytical Data Processing in SQL" book.

socialetl

Project for "Data pipeline design patterns" blog.

local_dev

Local development environment for python data projects, with Docker

data_test_ci

Repository showing how to automate data testing as part of CI

change_data_capture

Repo for CDC with debezium blog post

idempotent-data-pipeline

Making data pipelines idempotent

dbt_development

Repo to explain development, CI/CD cycle in dbt

data-engineering-interview-series

WIP repository for Data Engineering Interview Series

Jupyter Notebook

trigger_spark_with_lambda

Simple example showing how to trigger a spark job with AWS Lambda

adv_data_transformation_in_sql

Code for "Advanced data transformations in SQL" free live workshop

data_engineering_best_practices

spark_submit_airflow-

Simple repo to demonstrate how to submit a spark job

python_essentials_for_data_engineers

recipes

personal how-tos for common DE tasks

josephmachado

unit_test_dbt

unit test example in DBT

sde_superset_demo

Apache Superset Demp

docker_for_data_engineers

data_engineering_best_practices_log

Code to demonstrate data engineering metadata & logging best practices

how-to-slash-dbt-cost-w-duckdb-