• Stars
    star
    63
  • Rank 485,082 (Top 10 %)
  • Language
    Python
  • License
    Other
  • Created 5 months ago
  • Updated about 1 month ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

More Repositories

1

learning-spark

Example code from Learning Spark book
Java
3,864
star
2

koalas

Koalas: pandas API on Apache Spark
Python
3,330
star
3

Spark-The-Definitive-Guide

Spark: The Definitive Guide's Code Repository
Scala
2,678
star
4

scala-style-guide

Databricks Scala Coding Style Guide
2,673
star
5

spark-deep-learning

Deep Learning Pipelines for Apache Spark
Python
1,993
star
6

click

The "Command Line Interactive Controller for Kubernetes"
Rust
1,416
star
7

LearningSparkV2

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]
Scala
1,158
star
8

megablocks

Python
1,147
star
9

spark-sklearn

(Deprecated) Scikit-learn integration package for Apache Spark
Python
1,080
star
10

spark-csv

CSV Data Source for Apache Spark 1.x
Scala
1,051
star
11

tensorframes

[DEPRECATED] Tensorflow wrapper for DataFrames on Apache Spark
Scala
749
star
12

devrel

This repository contains the notebooks and presentations we use for our Databricks Tech Talks
HTML
687
star
13

reference-apps

Spark reference applications
Scala
648
star
14

spark-redshift

Redshift data source for Apache Spark
Scala
598
star
15

spark-sql-perf

Scala
543
star
16

spark-avro

Avro Data Source for Apache Spark
Scala
538
star
17

spark-xml

XML data source for Spark SQL and DataFrames
Scala
501
star
18

spark-corenlp

Stanford CoreNLP wrapper for Apache Spark
Scala
424
star
19

mlops-stacks

This repo provides a customizable stack for starting new ML projects on Databricks that follow production best-practices out of the box.
Python
415
star
20

spark-training

Apache Spark training material
Scala
396
star
21

databricks-cli

(Legacy) Command Line Interface for Databricks
Python
384
star
22

spark-perf

Performance tests for Apache Spark
Scala
372
star
23

delta-live-tables-notebooks

Python
334
star
24

terraform-provider-databricks

Databricks Terraform Provider
Go
333
star
25

spark-knowledgebase

Spark Knowledge Base
328
star
26

databricks-ml-examples

Python
284
star
27

sjsonnet

Scala
266
star
28

jsonnet-style-guide

Databricks Jsonnet Coding Style Guide
205
star
29

dbt-databricks

A dbt adapter for Databricks.
Python
199
star
30

databricks-sdk-py

Databricks SDK for Python (Beta)
Python
185
star
31

containers

Sample base images for Databricks Container Services
Dockerfile
163
star
32

databricks-sql-python

Databricks SQL Connector for Python
Python
158
star
33

sbt-spark-package

Sbt plugin for Spark packages
Scala
150
star
34

notebook-best-practices

An example showing how to apply software engineering best practices to Databricks notebooks.
Python
116
star
35

databricks-vscode

VS Code extension for Databricks
TypeScript
114
star
36

benchmarks

A place in which we publish scripts for reproducible benchmarks.
Python
106
star
37

terraform-databricks-examples

Examples of using Terraform to deploy Databricks resources
HCL
103
star
38

spark-tfocs

A Spark port of TFOCS: Templates for First-Order Conic Solvers (cvxr.com/tfocs)
Scala
88
star
39

intellij-jsonnet

Intellij Jsonnet Plugin
Java
82
star
40

sbt-databricks

An sbt plugin for deploying code to Databricks Cloud
Scala
71
star
41

terraform-databricks-lakehouse-blueprints

Set of Terraform automation templates and quickstart demos to jumpstart the design of a Lakehouse on Databricks. This project has incorporated best practices across the industries we work with to deliver composable modules to build a workspace to comply with the highest platform security and governance standards.
Python
71
star
42

spark-integration-tests

Integration tests for Spark
Scala
68
star
43

spark-pr-dashboard

Dashboard to aid in Spark pull request reviews
JavaScript
54
star
44

run-notebook

TypeScript
47
star
45

ide-best-practices

Best practices for working with Databricks from an IDE
Python
47
star
46

unity-catalog-setup

Notebooks, terraform, tools to enable setting up Unity Catalog
45
star
47

simr

Spark In MapReduce (SIMR) - launching Spark applications on existing Hadoop MapReduce infrastructure
Java
44
star
48

devbox

Scala
38
star
49

databricks-sql-go

Golang database/sql driver for Databricks SQL.
Go
35
star
50

diviner

Grouped time series forecasting engine
Python
33
star
51

cli

Databricks CLI
Go
32
star
52

tmm

Python
30
star
53

security-bucket-brigade

JavaScript
30
star
54

databricks-sdk-go

Databricks SDK for Go
Go
29
star
55

pig-on-spark

proof-of-concept implementation of Pig-on-Spark integrated at the logical node level
Scala
28
star
56

databricks-sql-cli

CLI for querying Databricks SQL
Python
27
star
57

automl

Python
26
star
58

databricks-sql-nodejs

Databricks SQL Connector for Node.js
TypeScript
24
star
59

tpch-dbgen

Patched version of dbgen
C
22
star
60

als-benchmark-scripts

Scripts to benchmark distributed Alternative Least Squares (ALS)
Scala
22
star
61

spark-package-cmd-tool

A command line tool for Spark packages
Python
18
star
62

congruity

The goal of this library is to provide a compatibility layer that makes it easier to adopt Spark Connect. The library is designed to be simply imported in your application and will then monkey-patch the existing API to provide the legacy functionality.
Python
16
star
63

python-interview

Databricks Python interview setup instructions
15
star
64

xgb-regressor

MLflow XGBoost Regressor
Python
15
star
65

databricks-accelerators

Accelerate the use of Databricks for customers [public repo]
Python
15
star
66

tableau-connector

Scala
12
star
67

files_in_repos

Python
12
star
68

upload-dbfs-temp

TypeScript
12
star
69

spark-sklearn-docs

HTML
11
star
70

sqltools-databricks-driver

SQLTools driver for Databricks SQL
TypeScript
11
star
71

genomics-pipelines

secondary analysis pipelines parallelized with apache spark
Scala
10
star
72

workflows-examples

10
star
73

databricks-sdk-java

Databricks SDK for Java
Java
10
star
74

dais-cow-bff

Code for the "Path to Production" DAIS 2024 and 2023 talks
Jupyter Notebook
8
star
75

xgboost-linux64

Databricks Private xgboost Linux64 fork
C++
8
star
76

mlflow-example-sklearn-elasticnet-wine

Jupyter Notebook
7
star
77

databricks-ttyd

C
6
star
78

setup-cli

Sets up the Databricks CLI in your GitHub Actions workflow.
Shell
4
star
79

terraform-databricks-mlops-aws-project

This module creates and configures service principals with appropriate permissions and entitlements to run CI/CD for a project, and creates a workspace directory as a container for project-specific resources for the Databricks AWS staging and prod workspaces.
HCL
4
star
80

jenkins-job-builder

Fork of https://docs.openstack.org/infra/jenkins-job-builder/ to include unmerged patches
Python
4
star
81

terraform-databricks-mlops-azure-project-with-sp-creation

This module creates and configures service principals with appropriate permissions and entitlements to run CI/CD for a project, and creates a workspace directory as a container for project-specific resources for the Azure Databricks staging and prod workspaces. It also creates the relevant Azure Active Directory (AAD) applications for the service principals.
HCL
4
star
82

terraform-databricks-sra

The Security Reference Architecture (SRA) implements typical security features as Terraform Templates that are deployed by most high-security organizations, and enforces controls for the largest risks that customers ask about most often.
HCL
4
star
83

databricks-empty-ide-project

Empty IDE project used by the VSCode extension for Databricks
3
star
84

databricks-repos-proxy

Python
2
star
85

databricks-asset-bundles-dais2023

Python
2
star
86

pex

Fork of pantsbuild/pex with a few Databricks-specific changes
Python
2
star
87

SnpEff

Databricks snpeff fork
Java
2
star
88

notebook_gallery

Jupyter Notebook
2
star
89

terraform-databricks-mlops-aws-infrastructure

This module sets up multi-workspace model registry between a Databricks AWS development (dev) workspace, staging workspace, and production (prod) workspace, allowing READ access from dev/staging workspaces to staging & prod model registries.
HCL
2
star
90

expectations

Python
1
star
91

homebrew-tap

Homebrew Tap for the Databricks CLI
Ruby
1
star
92

terraform-databricks-mlops-azure-infrastructure-with-sp-creation

This module sets up multi-workspace model registry between an Azure Databricks development (dev) workspace, staging workspace, and production (prod) workspace, allowing READ access from dev/staging workspaces to staging & prod model registries. It also creates the relevant Azure Active Directory (AAD) applications for the service principals.
HCL
1
star
93

mfg_dlt_workshop

DLT Manufacturing Workshop
Python
1
star
94

databricks-dbutils-scala

The Scala SDK for Databricks.
Scala
1
star
95

kdd24-forecasting-anomaly-detection

Python
1
star
96

terraform-databricks-mlops-azure-project-with-sp-linking

This module creates and configures service principals with appropriate permissions and entitlements to run CI/CD for a project, and creates a workspace directory as a container for project-specific resources for the Azure Databricks staging and prod workspaces. It also links pre-existing Azure Active Directory (AAD) applications to the service principals.
HCL
1
star
97

terraform-databricks-mlops-azure-infrastructure-with-sp-linking

This module sets up multi-workspace model registry between an Azure Databricks development (dev) workspace, staging workspace, and production (prod) workspace, allowing READ access from dev/staging workspaces to staging & prod model registries. It also links pre-existing Azure Active Directory (AAD) applications to the service principals.
HCL
1
star