rebremer/blog-datapipeline-cicd rebremer/blog-datapipeline-cicd

  • Stars
    star
    11
  • Rank 1,694,829 (Top 34 %)
  • Language
    Shell
  • Created about 4 years ago
  • Updated almost 2 years ago
Twitter logo Share LinkedIn logo Share Facebook logo Share
rebremer/blog-datapipeline-cicd
github

rebremer

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Data pipeline project using Data Factory, Databricks and Cosmosdb Graph, deployed using Azure DevOps, secured using firewalls and Azure AD

More Repositories

1

azure-function-selenium

Azure Function running selenium webdriver using a custom docker image
Python
56
star
2

devopsai_databricks

DevOps for AI project using Azure Databricks, Azure DevOps and Azure Machine Learning Service
Python
16
star
3

azure-synapseserverless-python-rest-api

Creating a REST API with Python on Synapse Serverless pools using external tables
Python
12
star
4

ms-identity-python-webapp-backend

Python web app using delegated permissions from signed-in user or application permissions to authenticate to backend. Backend can be Azure Function or SQLDB.
PowerShell
12
star
5

realtime_video_processing

Realtime video processing project using IoT Hub, Azure Functions, Cognitive Services and Power BI
Python
9
star
6

adfv2_cdm_metadata

Azure Data Factory (ADFv2) copy pipeline adding metadata using Common Data Model (CDM) structure
Python
9
star
7

blog-adfv2unittest-git

Data Factory unit test using Azure DevOps. ARM is used to deploy ADFv2, BACPAC SQLDB to standardize input/output data and pytest to run tests
Python
9
star
8

expose-deltatable-via-restapi

Python
5
star
9

azure_function_mi_sqldb

Project to secure Azure Functions with Azure AD, Key Vault and VNETs. Then connect to Azure SQL using firewall rules and Managed Identity of Function.
PowerShell
4
star
10

azure-functions-datalake-recovery-pitr

Scalable point-in-time recovery solution for an ADLSgen2 account with hierarchical namespace enabled using Azure Durable Functions
Python
4
star
11

databricks-mlflow-azureml-moe-vnet

Develop models in Databricks, use MLFlow to register models in AzureML with public access disabled, deploy models as private managed online endpoints
Jupyter Notebook
3
star
12

blog-snapshotbackup-azuredatalake

Datalake snapshots and incremental backups to different storage account
Python
3
star
13

data-factory-managed-identity-connection-aws-s3

Repository that uses ADF MI to connect to AWS S3 without need of storing credentials in key vault
2
star
14

blog-databrickshubspoke-git

Private link connectivity of 1 central ADLSgen2 account to multiple decentral Databricks workspaces
Shell
2
star
15

blog-sqldbalwaysencrypted-git

Azure DevOps yml pipeline that deploys a dacpac to a SQLDB using always encrypted
Rich Text Format
2
star
16

managed_identity_authentication

Authentication from Azure Data Factory to Azure Function using Managed Identity
PowerShell
1
star
17

python_swiss_army_knife

python_swiss_army_knife
Jupyter Notebook
1
star
18

pareto-distribution-calculator

Hobby statistics Python project that computes expected value E[x], variation Var[x] and kurtosis Kur[x] of fat tailed Pareto distribution
Python
1
star
19

application-gateway-aadauth

How to set up an Application Gateway in front of Web App that has Azure AD authentication enabled
1
star
20

azure-miscellaneous-scripts

Miscellaneous scripts to get things done on azure
Jupyter Notebook
1
star
21

azure-functions-scrapy

Azure Functions in Python that uses Scrapy to crawl websites
Shell
1
star
22

blog-mlopsapim-git

Jupyter Notebook
1
star
23

sqlmi_linkedserver_synapse_sqldb

PowerShell script that creates linked server in SQLMI. Linked server can either point to Synapse on demand or SQLDB
PowerShell
1
star
24

databricks_aad_token

Azure Databricks client script in Python using Service Principals
Python
1
star