Discover bennyaustin/pyspark-utils Open Source project

Stars
9
Rank 1,939,727 (Top 39 %)
Language
Python
License
Apache License 2.0
Created about 4 years ago
Updated 8 months ago

bennyaustin

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Reusable Python classes that extend open source PySpark capabilities. Examples of implementation is available under notebooks of repo https://github.com/bennyaustin/synapse-dataplatform

elt-framework

Extract Load Transform (ELT) framework is a metadata based batch orchestration framework for modern data platforms. Implemented using Azure PaaS data services. Common ingestion and transformation patterns available out of box. Reusable code can be easily extended to cater to custom patterns.

TSQL

synapse-dataplatform

A modern data platform implemented on Azure Synapse Analytics using ELT Framework - https://github.com/bennyaustin/elt-framework. Data platform infrastructure provisioned using https://github.com/bennyaustin/iac-synapse-dataplatform

TSQL

logicapp-documentIntelligence

Azure Logic App that scores a custom Azure Form Recognizer model in response to an Azure Event Grid event

PowerShell

ADFCustomActivity-AASPartition

Repository for Azure Data Factory (ADF) Custom Activity to dynamically create and process Azure Analysis Service (AAS) Tabular Model Partitions

logicapp-azure-resourcemgmt

Logic App that calls Azure Resource Management REST API to get information about resources in a subscription

PowerShell

iac-synapse-dataplatform

Infrastructure as Code (IaC) for synapse-dataplatform repo

Bicep

bennyaustin/pyspark-utils

bennyaustin

Reviews

Repository Details

More Repositories