Explore @databricks Open Source projects

Databricks (@databricks)

databricks

Stars
28,485
Global Org. Rank 644 (Top 0.3 %)
Registered almost 11 years ago
Most used languages

Python
31.4 %

Scala
26.7 %

HCL 9.3 %

Java
5.8 %

Go
4.7 %

TypeScript
4.7 %

JavaScript
3.5 %

Jupyter Notebook
3.5 %

C
2.3 %

HTML
2.3 %

Rust
1.2 %

C++
1.2 %

Dockerfile
1.2 %

Shell
1.2 %

Ruby
1.2 %
Location 🇺🇸 United States
Country Total Rank 301
Country Ranking

Scala
4

HCL 101

Dockerfile
115

Java
148

Python
153

Rust
177

HTML
372

Go
954

TypeScript
1,829

C
4,559

Jupyter Notebook
6,987

JavaScript
8,060

learning-spark

Example code from Learning Spark book

koalas

Koalas: pandas API on Apache Spark

Spark-The-Definitive-Guide

Spark: The Definitive Guide's Code Repository

scala-style-guide

Databricks Scala Coding Style Guide

spark-deep-learning

Deep Learning Pipelines for Apache Spark

click

The "Command Line Interactive Controller for Kubernetes"

LearningSparkV2

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]

spark-sklearn

(Deprecated) Scikit-learn integration package for Apache Spark

spark-csv

CSV Data Source for Apache Spark 1.x

tensorframes

[DEPRECATED] Tensorflow wrapper for DataFrames on Apache Spark

devrel

This repository contains the notebooks and presentations we use for our Databricks Tech Talks

reference-apps

Spark reference applications

spark-redshift

Redshift data source for Apache Spark

spark-sql-perf

spark-avro

Avro Data Source for Apache Spark

spark-xml

XML data source for Spark SQL and DataFrames

spark-corenlp

Stanford CoreNLP wrapper for Apache Spark

spark-training

Apache Spark training material

databricks-cli

(Legacy) Command Line Interface for Databricks

spark-perf

Performance tests for Apache Spark

terraform-provider-databricks

Databricks Terraform Provider

spark-knowledgebase

Spark Knowledge Base

delta-live-tables-notebooks

databricks-ml-examples

sjsonnet

mlops-stacks

This repo provides a customizable stack for starting new ML projects on Databricks that follow production best-practices out of the box.

jsonnet-style-guide

Databricks Jsonnet Coding Style Guide

databricks-sdk-py

Databricks SDK for Python (Beta)

dbt-databricks

A dbt adapter for Databricks.

containers

Sample base images for Databricks Container Services

sbt-spark-package

Sbt plugin for Spark packages

databricks-sql-python

Databricks SQL Connector for Python

benchmarks

A place in which we publish scripts for reproducible benchmarks.

databricks-vscode

VS Code extension for Databricks

terraform-databricks-examples

Examples of using Terraform to deploy Databricks resources

notebook-best-practices

An example showing how to apply software engineering best practices to Databricks notebooks.

spark-tfocs

A Spark port of TFOCS: Templates for First-Order Conic Solvers (cvxr.com/tfocs)

intellij-jsonnet

Intellij Jsonnet Plugin

sbt-databricks

An sbt plugin for deploying code to Databricks Cloud

spark-integration-tests

Integration tests for Spark

terraform-databricks-lakehouse-blueprints

Set of Terraform automation templates and quickstart demos to jumpstart the design of a Lakehouse on Databricks. This project has incorporated best practices across the industries we work with to deliver composable modules to build a workspace to comply with the highest platform security and governance standards.

spark-pr-dashboard

Dashboard to aid in Spark pull request reviews

run-notebook

simr

Spark In MapReduce (SIMR) - launching Spark applications on existing Hadoop MapReduce infrastructure

ide-best-practices

Best practices for working with Databricks from an IDE

devbox

unity-catalog-setup

Notebooks, terraform, tools to enable setting up Unity Catalog

diviner

Grouped time series forecasting engine

cli

security-bucket-brigade

databricks-sdk-go

Databricks SDK for Go

pig-on-spark

proof-of-concept implementation of Pig-on-Spark integrated at the logical node level

databricks-sql-cli

CLI for querying Databricks SQL

automl

databricks-sql-go

Golang database/sql driver for Databricks SQL.

tpch-dbgen

Patched version of dbgen

als-benchmark-scripts

Scripts to benchmark distributed Alternative Least Squares (ALS)

databricks-sql-nodejs

Databricks SQL Connector for Node.js

spark-package-cmd-tool

A command line tool for Spark packages

python-interview

Databricks Python interview setup instructions

xgb-regressor

MLflow XGBoost Regressor

databricks-accelerators

Accelerate the use of Databricks for customers [public repo]

tableau-connector

files_in_repos

upload-dbfs-temp

spark-sklearn-docs

genomics-pipelines

secondary analysis pipelines parallelized with apache spark

workflows-examples

databricks-sdk-java

Databricks SDK for Java

sqltools-databricks-driver

SQLTools driver for Databricks SQL

xgboost-linux64

Databricks Private xgboost Linux64 fork

tmm

mlflow-example-sklearn-elasticnet-wine

Jupyter Notebook

databricks-ttyd

dais-cow-bff

Code for the "Bridging the Production Gap" DAIS 2023 talk

Jupyter Notebook

setup-cli

Sets up the Databricks CLI in your GitHub Actions workflow.

terraform-databricks-mlops-aws-project

This module creates and configures service principals with appropriate permissions and entitlements to run CI/CD for a project, and creates a workspace directory as a container for project-specific resources for the Databricks AWS staging and prod workspaces.

jenkins-job-builder

Fork of https://docs.openstack.org/infra/jenkins-job-builder/ to include unmerged patches

terraform-databricks-mlops-azure-project-with-sp-creation

This module creates and configures service principals with appropriate permissions and entitlements to run CI/CD for a project, and creates a workspace directory as a container for project-specific resources for the Azure Databricks staging and prod workspaces. It also creates the relevant Azure Active Directory (AAD) applications for the service principals.

terraform-databricks-sra

The Security Reference Architecture (SRA) implements typical security features as Terraform Templates that are deployed by most high-security organizations, and enforces controls for the largest risks that customers ask about most often.

databricks-empty-ide-project

Empty IDE project used by the VSCode extension for Databricks

databricks-repos-proxy

databricks-asset-bundles-dais2023

pex

Fork of pantsbuild/pex with a few Databricks-specific changes

SnpEff

Databricks snpeff fork

databricks-dbutils-scala

The Scala SDK for Databricks.

notebook_gallery

Jupyter Notebook

terraform-databricks-mlops-aws-infrastructure

This module sets up multi-workspace model registry between a Databricks AWS development (dev) workspace, staging workspace, and production (prod) workspace, allowing READ access from dev/staging workspaces to staging & prod model registries.

terraform-databricks-aws-workspace

Terraform module to create Databricks AWS E2 workspace

homebrew-tap

Homebrew Tap for the Databricks CLI

terraform-databricks-mlops-azure-infrastructure-with-sp-creation

This module sets up multi-workspace model registry between an Azure Databricks development (dev) workspace, staging workspace, and production (prod) workspace, allowing READ access from dev/staging workspaces to staging & prod model registries. It also creates the relevant Azure Active Directory (AAD) applications for the service principals.

mfg_dlt_workshop

DLT Manufacturing Workshop

terraform-databricks-mlops-azure-project-with-sp-linking

This module creates and configures service principals with appropriate permissions and entitlements to run CI/CD for a project, and creates a workspace directory as a container for project-specific resources for the Azure Databricks staging and prod workspaces. It also links pre-existing Azure Active Directory (AAD) applications to the service principals.

terraform-databricks-mlops-azure-infrastructure-with-sp-linking

This module sets up multi-workspace model registry between an Azure Databricks development (dev) workspace, staging workspace, and production (prod) workspace, allowing READ access from dev/staging workspaces to staging & prod model registries. It also links pre-existing Azure Active Directory (AAD) applications to the service principals.