josephmachado/trigger_spark_with_lambda josephmachado/trigger_spark_with_lambda

  • Stars
    star
    3
  • Rank 3,963,521 (Top 79 %)
  • Language
    Shell
  • Created over 3 years ago
  • Updated almost 2 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
josephmachado/trigger_spark_with_lambda
github

josephmachado

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Simple example showing how to trigger a spark job with AWS Lambda

More Repositories

1

beginner_de_project

Beginner data engineering project - batch edition
HCL
287
star
2

data_engineering_project_template

A template repository to create a data project with IAC, CI/CD, Data migrations, & testing
HCL
85
star
3

simple_dbt_project

Code for dbt tutorial
50
star
4

bitcoinMonitor

Near real time ETL to populate a dashboard.
Python
29
star
5

online_store

End to end data engineering project
Python
25
star
6

beginner_de_project_stream

Simple stream processing pipeline
Scala
24
star
7

spark_submit_airflow

Simple repo to demonstrate how to submit a spark job to EMR from Airflow
Python
20
star
8

analytical_dp_with_sql

Code for my "Analytical Data Processing in SQL" book.
Makefile
16
star
9

socialetl

Project for "Data pipeline design patterns" blog.
Python
14
star
10

e2e_datapipeline_test

Example repo to create end to end tests for data pipeline.
Python
12
star
11

local_dev

Local development environment for python data projects, with Docker
Python
10
star
12

data_test_ci

Repository showing how to automate data testing as part of CI
Python
6
star
13

change_data_capture

Repo for CDC with debezium blog post
Python
4
star
14

idempotent-data-pipeline

Making data pipelines idempotent
Python
4
star
15

dbt_development

Repo to explain development, CI/CD cycle in dbt
4
star
16

data-engineering-interview-series

WIP repository for Data Engineering Interview Series
Jupyter Notebook
3
star
17

adv_data_transformation_in_sql

Code for "Advanced data transformations in SQL" free live workshop
2
star
18

data_engineering_best_practices

WIP
Python
2
star
19

spark_submit_airflow-

Simple repo to demonstrate how to submit a spark job
2
star
20

python_essentials_for_data_engineers

WIP
Python
2
star
21

recipes

personal how-tos for common DE tasks
2
star
22

josephmachado

Profile readme
1
star
23

unit_test_dbt

unit test example in DBT
Shell
1
star
24

sde_superset_demo

Apache Superset Demp
1
star
25

docker_for_data_engineers

C
1
star
26

data_engineering_best_practices_log

Code to demonstrate data engineering metadata & logging best practices
Python
1
star
27

how-to-slash-dbt-cost-w-duckdb-

JavaScript
1
star