AdamPaternostro/Azure-Data-Factory-CI-CD-Source-Control

Stars
39
Rank 693,563 (Top 14 %)
Language
PowerShell
Created over 4 years ago
Updated about 4 years ago

AdamPaternostro/Azure-Data-Factory-CI-CD-Source-Control

AdamPaternostro

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

How do to CI/CD with Azure Data Factory

Azure-Databricks-Dev-Ops

Complete end to end sample of doing DevOps with Azure Databricks

Azure-Power-BI-Embedded-Sample

Shows how to setup and configure Power BI Embedded quickly. It also provides an Azure Function to get the Embedded tokens for a pure JavaScript sample.

Azure-Databricks-Log4J-To-AppInsights

Connect your Spark Databricks clusters Log4J output to the Application Insights Appender

Azure-Synapse-SQL-Serverless-Generate-From-Hive

This is a script that will read a Hive metastore and generate SQL Serverless create view statements.

Jupyter Notebook

Architecture-Azure

Sample Azure Architecture

Training-Azure-DevOps

Using Azure Dev Ops to deploy Infrastructure as Code, Web application, Staging Slots and Docker images. Also, contains my best practices when using DevOps in the cloud.

Azure-App-Insights-Distrubuted-Tracing

How to use Application Insights to do distributed tracing through a Web App, REST API, Function App, Service Bus, Databricks and Data Factory.

Azure-Dev-Ops-Single-CI-CD-Dev-To-Production-Pipeline

Create a single CI-C pipeline

Azure-Big-Data-and-Machine-Learning-Architecture

A ready to use architecture for processing data and performing machine learning in Azure

DevOps-Best-Practices

DevOps-Best-Practices

Azure-Sample-ARM-Template-Architecture

Shows how to build a set of linked templates with conditions, loops, array and complex parameters

Azure-Web-App-Nginx-Reverse-Proxy-Dotnet-Core-Linux

Dotnet Core MVC web app running on Linux using Docker and nginx as a reverse proxy

Azure-SQL-DW-Synapse-Test-Case-Runner

Cross platform tool for running test cases in a serial or concurrently and log the results to a set of tables to see the results.

Azure-Databricks-Token-Rotation

A script that will rotate Azure Databricks tokens utilizing Azure Key Vault

Azure-Cloud-Services-To-App-Service-Web-Apps

Best practices and lessons learned converting Azure Classic Cloud services to Azure App Service Web Apps

Azure-ARM-Terraform

This shows some examples of ARM and Terraform templates along with a feature comparison.

Azure-Relay-Connect-To-SQL-PaaS-Without-Firewall-Rule

Azure-Relay-Connect-To-SQL-PaaS-Without-Firewall-Rule

Azure-Databricks-With-Spline

Using Spline with Azure Databricks

Azure-Deploy-VSTS-On-VNET-App-Service-Environment

Shows how to use Visual Studio Team Services to deploy an application to an Application Service Environment that is on a VNET.

Azure-Shipyard-with-InfiniBand-and-Intel-MPI

Lightning fast performance in Azure with InfiniBand, Azure Storage and Intel MPI

Azure-Databricks-External-Hive-and-ADLS

Shows how to use an External Hive (SQL Server) along with ADLS Gen 1 as part of a Databricks initialization script that runs when the cluster is created.

Azure-Verify-Required-Tags

Checks to see if resources and/or resource groups have required tags.

Azure-Docker-Shipyard

Demo of using Docker contatiners in Azure Batch (using Shipyard)

Azure-Databricks-CI-CD-Initial-Token

How to do CI/CD with Azure Databricks and get the initial Databricks token.

Azure-Dual-Region-Deployment-Approach

This shows my typically deployment utilizes two regions in Azure.

Azure-Get-All-Storage-Accounts-With-Public-Blob-Containers

Loops through all your subscriptions and checks each storage account for public blob containers.

Azure-Tomcat-Web-App-Container

Creates a Docker container with Tomcat installed behind Apache which acts as a reverse proxy so Tomcat can be warmed up before being added to the Azure load balancer. Solves the problems of web apps needing time to warm-up as well as locking of WAR files that can occur on Azure Window Web Apps.

Azure-DevOps-Common-Templates

Stores common re-usable YAML files for Azure Pipelines across an enterprise.

Azure-Databricks-HDInsight-Hive-Metastore

How to share an HDInsight Hive Metastore with Azure Databricks

Azure-Create-Well-Named-VMs

Create a well named set of Azure VMs instead of letting Azure give default names to your Azure disks and NIC cards

Azure-Spark-Livy-Application-Insights-External-Dependency

Use Spark with Livy along with Application Insights. Learn to host your external dependencies in data lake.

Azure-Big-Data-Dev-Ops

A set of Azure ARM templates for big data architectures in Azure

Azure-DataFactory-REST-Call

This Node.js script shows how to call a Data Factory on demand via REST

Azure-ADLS-Blob-Data-Copy

Queues up files to copy from one ADLS account to be copied to another ADLS account. You can also use this for on-prem and/or blob.

Azure-Create-HDInsight-Service-Principle

Creates a PFX for HDInsights. Also, creates the Azure Application.

Azure-Blob-Concurrency

Docker container that runs on Azure blob using Alpine Linux with Azure Python SDK to put a load on Azure Storage to see the download times.

Azure-HDInsight-ARM-Template

Creates an HDInsight cluster that has an external Hive metastore and access to Azure Data Lake Store

Azure-Synapse-REST-API

A REST API for Azure Synapse (SQL DW).

Azure-HDI-DistCP

Creates a HDInsight cluster then runs distcp remotely to copy data between blob and/or data lake (ADLS)

Azure-DataLake-Folder-Upload

Upload a folder to Azure Data Lake Store

Azure-Databricks-auto-sklearn

Shows how to install auto-sklearn on an Azure Databricks cluster

GitHub-Azure-Function-Proxy-for-ARM-Template

Allows you to use your private GitHub repo for Azure Linked ARM templates.

Azure-DevOps-Integration

Tests using a YAML template that resides in a different repository.

Azure-SQL-Database-Backup-Restore-CLI

Shows how to backup and restore a SQL Database using Azure CLI. Performs the restore process in a loop (optional) for restoring over several destinations in an easy manner.

Azure-EA-Billing-Data-for-Budget-Alerts

Uses PowerShell to download your EA billing data. It then creates a CSV by Resource Group so you can then trigger alerts at this level.