Discover astronomer/airflow-provider-kafka Open Source project

Guides and docs to help you get up and running with Apache Airflow.

1,154

airflow-guides

JavaScript

797

astronomer-cosmos

Run your dbt Core projects as Apache Airflow DAGs and Task Groups with a few lines of code

Helm Charts for the Astronomer Platform, Apache Airflow as a Service on Kubernetes

589

astronomer

CLI that makes it easy to create, test and deploy Airflow DAGs to Astronomer

444

astro-cli

348

astro-sdk

Astro SDK allows rapid and clean development of {Extract, Load, Transform} workflows using Python and SQL, powered by Apache Airflow.

A Helm chart to install Apache Airflow on Kubernetes

338

airflow-chart

An end-to-end LLM reference implementation providing a Q&A interface for Airflow and Astronomer

252

ask-astro

A repository of sample code to accompany our blog post on Airflow and dbt.

187

airflow-dbt-demo

Great Expectations Airflow operator

165

airflow-provider-great-expectations

Airflow Providers containing Deferrable Operators & Sensors from Astronomer

158

astronomer-providers

Astronomer Core Docker Images

134

ap-airflow

Jinja

102

airflow-data-quality-demo

A repository of sample code to show data quality checking best practices using Airflow.

A template repo for building and releasing Airflow provider packages.

airflow-provider-sample

airflow-example-dags

Sample Airflow DAGs

webinar-dag-writing-best-practices

Get started with Apache Airflow. Check the README for instructions on how to run your first DAGs today. 🚀

airflow-quickstart

This repository contains all content and code for Astro and Astronomer Software documentation.

docs

telescope

ray-airflow-demo

dynamic-dags-tutorial

Demo DAGs that show how to run dbt Core in Airflow using Cosmos

cosmos-demo

An MLflow Provider Package for Apache Airflow

airflow-provider-mlflow

airflow-ui

TypeScript

starship

Orchestrate your Databricks notebooks in Airflow and execute them as Databricks Workflows

astro-provider-databricks

airflow-testing-guide

A skeleton project for testing Airflow code

webinar-demos

deploy-action

Custom Github Actions

airflow-testing-skeleton

A new Airflow Provider for Fivetran, maintained by Astronomer and Fivetran

airflow-provider-fivetran-async

Sample Airflow DAGs to load data from the CovidTracking API to Snowflake via an AWS S3 intermediary.

airflow-covid-data

A provider package for DuckDB

airflow-provider-duckdb

Security Manager for the Astronomer Airflow distribution

astronomer-fab-securitymanager

apache-airflow-providers-transfers

This repo contains DAGs demonstrating a variety of ELT patterns using Airflow along with dbt.

airflow-dbt-elt

ap-vendor

Astronomer Vendor Images

Dockerfile

intro-to-airflow-webinar

airflow-guide-passing-data-between-tasks

Intended for internal use: deploys all infrastructure required for Astronomer to run on GCP

terraform-google-astronomer-gcp

Example DAGs demonstrating how to implement alerting and notifications via Microsoft Teams

cs-tutorial-msteams-callbacks

Easily create and use Python Virtualenvs in Apache Airflow

astro-provider-venv

astronomer-airflow-scripts

Waits for Apache Airflow database migrations to complete.

Deploys all infrastructure required for Astronomer to run on AWS. For a complete deployment, see https://github.com/astronomer/terraform-aws-astronomer-enterprise

terraform-aws-astronomer-aws

airflow-scheduling-tutorial

cosmos-example

2-4-example-dags

mlflow-example

2-6-example-dags

airflow-provider-pulumi

academy-genai

kedro-ge-airflow

A template repository for contributing DAGs to the Astronomer Registry.

registry-dag-template

dynamic-task-mapping-tutorial

apache-airflow-providers-alembic

webinar-secrets-management

custom-xcom-backend-tutorial

airflow-sql-tutorial

Deploy Astronomer on Kubernetes

terraform-kubernetes-astronomer

Example DAGs demonstrating how to implement alerting and notifications via Slack

cs-tutorial-slack-callbacks

Example DAGs for Airflow 2.9

airflow-dags

airflow-analytics-plugin

terraform-provider-astro

Astro Terraform Provider

airflow-databricks-tutorial

azure-operator-tutorials

Codespace with Airflow and the Astro CLI

education-sandbox

Getting phased out - please use astronomer/terraform-* modules to track issues

terraform

cross-dag-dependencies-tutorial

apache-airflow-providers-isolation

A repository of sample code to accompany our blog post on Change Data Capture and CloudSQL

cdc-cloudsql-airflow-demo

Example project for configuring opern source Airflow version with LDAP. Includes prepopulated OpenLDAP server

airflow-ldap-example

airflow-wandb-demo

Reusable UI components for Astronomer projects.

spectra

JavaScript

airflow-llm-demo

2-7-example-dags

airflow-snowpark-containers-demo

astro-example-dags

Tutorial for how to use Astronomer+Airflow with Talend. Contains reference DAGs and other supporting materials.

airflow-talend-tutorial

azure_demo

The script needed to set up a customer's Google Cloud Project for an Astro activation

astro-gcp-onboarding

Shell

greenplum-airflow-demo

cs-astro-onboarding

Homebrew Formulae to @astronomer binaries, powered by @astronomer

homebrew-tap

Ruby

debugging-dags-webinar

A repository containing the DAGs shown in the Debugging DAGs webinar on 2023-01-31.

Customer facing utilities to help customers migrating from Software/Nebula to Astro

migrate-to-astro

Guides and structured metadata about Airflow connections

airflow-connection-docs

The repository for example DAGs shown in the 2023-04-11 Astronomer webinar on passing data between tasks.

pass-data-between-tasks-webinar

pyconuk2022

ds-ml-example-dags

Code challenge for Front-End developers applying at Astronomer

react-graphql-code-challenge

JavaScript

airflow-sagemaker-tutorial

databricks-ml-example

Repository for the Airflow 101 webinar on June 6th 2023.

airflow_101_webinar

Repo hosting PagerDuty + Airflow Integration Benefits Doc

pagerduty_airflow_integration_benefits

sagemaker-batch-inference

An example DAG for orchestrating Azure Data Factory pipelines with Apache Airflow.

airflow-adf-integration

Example code for the dbt core Learn tutorial. The Astro dbt provider, also known as Cosmos, is a tool automatically integrate dbt models into your Airflow DAGs.

astro-dbt-provider-tutorial-example

How to Load reporting database for Airflow DAGS, DAG Runs, and Task Instances

cs-tutorial-reporting

cosmos-dev