andresmaopal/data-lake-staging-engine andresmaopal/data-lake-staging-engine

  • Stars
    star
    6
  • Rank 2,531,573 (Top 51 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created almost 5 years ago
  • Updated over 4 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
andresmaopal/data-lake-staging-engine
github

andresmaopal

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

S3 event-based engine to process files (microbatches), transform them (parquet) and sync the source to Glue Data Catalog - (Multicountry support)

More Repositories

1

aws-etl-orchestrator-v2

A serverless architecture for orchestrating ETL jobs in arbitrarily-complex workflows using AWS Step Functions and AWS Lambda. Version 2 Allow Glue Spark and Python shell Jobs
Python
4
star
2

hudi_palacan_demos

Hudi on AWS Notebook
Jupyter Notebook
1
star
3

Excel_to_LakeFormation_permissions

Lambda to Grant and Revoke permissions in bulk to Lake Formation using an user-friendly Excel template
Python
1
star