Discover @josephmachado Open Source projects

Start Data Engineering (@josephmachado)

josephmachado

Stars
599
Global Rank 49,366 (Top 2 %)
Followers 389
Registered over 4 years ago
Most used languages

Python
57.1 %

HCL 9.5 %

Shell
9.5 %

Makefile 4.8 %

JavaScript
4.8 %

C
4.8 %

Scala
4.8 %

Jupyter Notebook
4.8 %
Location 🇺🇸 United States
Country Total Rank 12,979
Country Ranking

HCL 39

Makefile 586

Scala
699

Python
7,377

beginner_de_project

Beginner data engineering project - batch edition

data_engineering_project_template

A template repository to create a data project with IAC, CI/CD, Data migrations, & testing

simple_dbt_project

Code for dbt tutorial

bitcoinMonitor

Near real time ETL to populate a dashboard.

online_store

End to end data engineering project

beginner_de_project_stream

Simple stream processing pipeline

spark_submit_airflow

Simple repo to demonstrate how to submit a spark job to EMR from Airflow

analytical_dp_with_sql

Code for my "Analytical Data Processing in SQL" book.

socialetl

Project for "Data pipeline design patterns" blog.

e2e_datapipeline_test

Example repo to create end to end tests for data pipeline.

local_dev

Local development environment for python data projects, with Docker

data_test_ci

Repository showing how to automate data testing as part of CI

change_data_capture

Repo for CDC with debezium blog post

idempotent-data-pipeline

Making data pipelines idempotent

dbt_development

Repo to explain development, CI/CD cycle in dbt

data-engineering-interview-series

WIP repository for Data Engineering Interview Series

Jupyter Notebook

trigger_spark_with_lambda

Simple example showing how to trigger a spark job with AWS Lambda

adv_data_transformation_in_sql

Code for "Advanced data transformations in SQL" free live workshop

data_engineering_best_practices

spark_submit_airflow-

Simple repo to demonstrate how to submit a spark job

python_essentials_for_data_engineers

recipes

personal how-tos for common DE tasks

josephmachado

unit_test_dbt

unit test example in DBT

sde_superset_demo

Apache Superset Demp

docker_for_data_engineers

data_engineering_best_practices_log

Code to demonstrate data engineering metadata & logging best practices

how-to-slash-dbt-cost-w-duckdb-