Discover bennyaustin/elt-framework Open Source project

Stars
17
Rank 1,257,181 (Top 25 %)
Language TSQL
License
GNU General Publi...
Created almost 4 years ago
Updated 3 months ago

bennyaustin

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Extract Load Transform (ELT) framework is a metadata based batch orchestration framework for modern data platforms. Implemented using Azure PaaS data services. Common ingestion and transformation patterns available out of box. Reusable code can be easily extended to cater to custom patterns.

pyspark-utils

Reusable Python classes that extend open source PySpark capabilities. Examples of implementation is available under notebooks of repo https://github.com/bennyaustin/synapse-dataplatform

Python

synapse-dataplatform

A modern data platform implemented on Azure Synapse Analytics using ELT Framework - https://github.com/bennyaustin/elt-framework. Data platform infrastructure provisioned using https://github.com/bennyaustin/iac-synapse-dataplatform

TSQL

logicapp-documentIntelligence

Azure Logic App that scores a custom Azure Form Recognizer model in response to an Azure Event Grid event

PowerShell

ADFCustomActivity-AASPartition

Repository for Azure Data Factory (ADF) Custom Activity to dynamically create and process Azure Analysis Service (AAS) Tabular Model Partitions