AdamPaternostro/Azure-Databricks-Dev-Ops

Stars
66
Rank 466,488 (Top 10 %)
Language
Shell
Created about 4 years ago
Updated over 2 years ago

AdamPaternostro/Azure-Databricks-Dev-Ops

AdamPaternostro

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Complete end to end sample of doing DevOps with Azure Databricks

Azure-Data-Factory-CI-CD-Source-Control

How do to CI/CD with Azure Data Factory

Azure-Power-BI-Embedded-Sample

Shows how to setup and configure Power BI Embedded quickly. It also provides an Azure Function to get the Embedded tokens for a pure JavaScript sample.

Azure-Databricks-Log4J-To-AppInsights

Connect your Spark Databricks clusters Log4J output to the Application Insights Appender

Azure-Synapse-SQL-Serverless-Generate-From-Hive

This is a script that will read a Hive metastore and generate SQL Serverless create view statements.

Jupyter Notebook

Architecture-Azure

Sample Azure Architecture

Training-Azure-DevOps

Using Azure Dev Ops to deploy Infrastructure as Code, Web application, Staging Slots and Docker images. Also, contains my best practices when using DevOps in the cloud.

Azure-App-Insights-Distrubuted-Tracing

How to use Application Insights to do distributed tracing through a Web App, REST API, Function App, Service Bus, Databricks and Data Factory.

Azure-Dev-Ops-Single-CI-CD-Dev-To-Production-Pipeline

Create a single CI-C pipeline

Azure-Big-Data-and-Machine-Learning-Architecture

A ready to use architecture for processing data and performing machine learning in Azure

DevOps-Best-Practices

DevOps-Best-Practices

Azure-Sample-ARM-Template-Architecture

Shows how to build a set of linked templates with conditions, loops, array and complex parameters

Azure-Web-App-Nginx-Reverse-Proxy-Dotnet-Core-Linux

Dotnet Core MVC web app running on Linux using Docker and nginx as a reverse proxy

Azure-SQL-DW-Synapse-Test-Case-Runner

Cross platform tool for running test cases in a serial or concurrently and log the results to a set of tables to see the results.

Azure-Databricks-Token-Rotation

A script that will rotate Azure Databricks tokens utilizing Azure Key Vault

Azure-Cloud-Services-To-App-Service-Web-Apps

Best practices and lessons learned converting Azure Classic Cloud services to Azure App Service Web Apps

Azure-ARM-Terraform

This shows some examples of ARM and Terraform templates along with a feature comparison.

Azure-Relay-Connect-To-SQL-PaaS-Without-Firewall-Rule

Azure-Relay-Connect-To-SQL-PaaS-Without-Firewall-Rule

Azure-Databricks-With-Spline

Using Spline with Azure Databricks

Azure-Deploy-VSTS-On-VNET-App-Service-Environment

Shows how to use Visual Studio Team Services to deploy an application to an Application Service Environment that is on a VNET.

Azure-Shipyard-with-InfiniBand-and-Intel-MPI

Lightning fast performance in Azure with InfiniBand, Azure Storage and Intel MPI

Azure-Databricks-External-Hive-and-ADLS

Shows how to use an External Hive (SQL Server) along with ADLS Gen 1 as part of a Databricks initialization script that runs when the cluster is created.

Azure-Verify-Required-Tags

Checks to see if resources and/or resource groups have required tags.

Azure-Docker-Shipyard

Demo of using Docker contatiners in Azure Batch (using Shipyard)

Azure-Databricks-CI-CD-Initial-Token

How to do CI/CD with Azure Databricks and get the initial Databricks token.

Azure-Dual-Region-Deployment-Approach

This shows my typically deployment utilizes two regions in Azure.

Azure-Get-All-Storage-Accounts-With-Public-Blob-Containers

Loops through all your subscriptions and checks each storage account for public blob containers.

Azure-Tomcat-Web-App-Container

Creates a Docker container with Tomcat installed behind Apache which acts as a reverse proxy so Tomcat can be warmed up before being added to the Azure load balancer. Solves the problems of web apps needing time to warm-up as well as locking of WAR files that can occur on Azure Window Web Apps.

Azure-DevOps-Common-Templates

Stores common re-usable YAML files for Azure Pipelines across an enterprise.

Azure-Databricks-HDInsight-Hive-Metastore

How to share an HDInsight Hive Metastore with Azure Databricks

Azure-Create-Well-Named-VMs

Create a well named set of Azure VMs instead of letting Azure give default names to your Azure disks and NIC cards

Azure-Spark-Livy-Application-Insights-External-Dependency

Use Spark with Livy along with Application Insights. Learn to host your external dependencies in data lake.

Azure-Big-Data-Dev-Ops

A set of Azure ARM templates for big data architectures in Azure

Azure-DataFactory-REST-Call

This Node.js script shows how to call a Data Factory on demand via REST

Azure-ADLS-Blob-Data-Copy

Queues up files to copy from one ADLS account to be copied to another ADLS account. You can also use this for on-prem and/or blob.

Azure-Create-HDInsight-Service-Principle

Creates a PFX for HDInsights. Also, creates the Azure Application.

Azure-Blob-Concurrency

Docker container that runs on Azure blob using Alpine Linux with Azure Python SDK to put a load on Azure Storage to see the download times.

Azure-HDInsight-ARM-Template

Creates an HDInsight cluster that has an external Hive metastore and access to Azure Data Lake Store

Azure-Synapse-REST-API

A REST API for Azure Synapse (SQL DW).

Azure-HDI-DistCP

Creates a HDInsight cluster then runs distcp remotely to copy data between blob and/or data lake (ADLS)

Azure-DataLake-Folder-Upload

Upload a folder to Azure Data Lake Store

Azure-Databricks-auto-sklearn

Shows how to install auto-sklearn on an Azure Databricks cluster

GitHub-Azure-Function-Proxy-for-ARM-Template

Allows you to use your private GitHub repo for Azure Linked ARM templates.

Azure-DevOps-Integration

Tests using a YAML template that resides in a different repository.

Azure-SQL-Database-Backup-Restore-CLI

Shows how to backup and restore a SQL Database using Azure CLI. Performs the restore process in a loop (optional) for restoring over several destinations in an easy manner.

Azure-EA-Billing-Data-for-Budget-Alerts

Uses PowerShell to download your EA billing data. It then creates a CSV by Resource Group so you can then trigger alerts at this level.