blockchain-etl/public-datasets

Stars
187
Rank 206,464 (Top 5 %)
Language
License
MIT License
Created over 4 years ago
Updated 10 months ago

blockchain-etl/public-datasets

blockchain-etl

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

The list of public blockchain datasets in BigQuery

Public Blockchain Datasets

Network	BigQuery	Lag	PubSub	Node repo(s)	Indexer repo(s)
Band	🔍 (Examples)	0	💧	band-kubernetes	band-etl
Bitcoin	🔍 (Examples)	3	💧	docker-bitcoind	bitcoin-etl, bitcoin-etl-airflow, bitcoin-etl-airflow-neo4j, bitcoin-etl-streaming
Bitcoin Cash	🔍 (Examples)	6	💧	docker-bitcoin-cashd	see Bitcoin
Celo	🔍	10	💧	?	celo-etl
Dash	🔍 (Examples)	24	💧	docker-dashd	see Bitcoin
Dogecoin	🔍 (Examples)	60	💧	docker-dogecoind	see Bitcoin
Ethereum	🔍 (Examples)	18	💧	?	ethereum-etl, ethereum-etl-airflow, ethereum-etl-neo4j, ethereum-etl-postgres
Ethereum 2	🔍	?	❌	?	ethereum2-etl, ethereum2-etl-airflow
Ethereum Classic	🔍 (Examples)	?	❌	see Ethereum	see Ethereum
Fantom	🔍	?	💧	?	?
Hedera Hashgraph	🔍	?	?	?	hedera-etl
IoTeX	🔍 (Examples)	10	💧	iotex-kubernetes	iotex-etl
Litecoin	🔍 (Examples)	12	💧	docker-litecoind	see Bitcoin
Polygon	🔍 (Examples)	80	💧	?	polygon-etl
Tezos	🔍 (Examples)	?	?	tezos-kubernetes	tezos-etl, tezos-etl-airflow
Theta	🔍 (Examples)	?	?	?	theta-etl
XRP	🔍	?	?	?	fetch-xrpl-transactions
Zcash	🔍 (Examples)	24	💧	docker-zcashd	see Bitcoin
Zilliqa	🔍 (Examples)	?	?	?	zilliqa-etl

Planning / Building

Network	BigQuery	Lag	PubSub	Node repo(s)	Indexer repo(s)	Notes
Algorand	⏳	?	⏳	?	?	Needs help
Aptos	⏳	?	⏳	?	?	Needs help
Arbitrum	⏳	?	⏳	?	?	Needs help
Avalanche	⏳	?	⏳	?	?	Needs help
BNB Chain	⏳	?	⏳	?	?	Needs help
Cronos	⏳	?	⏳	?	?	Needs help
Eos	❌	?	❌	?	eos-etl, eos-etl-airflow	Needs help
Icon	❌	?	❌	?	icon-etl, icon-etl-airflow	Needs help
Klaytn	👷	?	👷	?	klaytn-etl	In progress
Optimism	⏳	?	⏳	?	?	Needs help
Solana	👷	?	👷	?	solana-etl	In progress
Sui	⏳	?	⏳	?	?	Needs help

ethereum-etl

Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

bitcoin-etl

ETL scripts for Bitcoin, Litecoin, Dash, Zcash, Doge, Bitcoin Cash. Available in Google BigQuery https://goo.gl/oY5BCQ

ethereum-etl-airflow

Airflow DAGs for exporting, loading, and parsing the Ethereum blockchain data. How to get any Ethereum smart contract into BigQuery https://towardsdatascience.com/how-to-get-any-ethereum-smart-contract-into-bigquery-in-8-mins-bab5db1fdeee

awesome-bigquery-views

Useful SQL queries for Blockchain ETL datasets in BigQuery.

ethereum-etl-postgres

ETL for moving Ethereum data to PostgreSQL database

polygon-etl

ETL (extract, transform and load) tools for ingesting Polygon blockchain data to Google BigQuery and Pub/Sub

blockchain-etl-streaming

Streaming Ethereum and Bitcoin blockchain data to Google Pub/Sub or Postgres in Kubernetes

ethereum2-etl

Python scripts for ETL (extract, transform and load) jobs for Ethereum 2.0 beacon blocks, attestations, deposits, slashings, validators, committees. Data is available in Google BigQuery

blockchain-etl-architecture

Blockchain ETL Architecture

ethers.js-bigquery

ethers.js library, compiled for use in Google BigQuery

solana-etl-airflow

ETL for Solana. Contributions are welcome. Join the Telegram channel https://t.me/joinchat/GsMpbA3mv1OJ6YMp3T5ORQ

bitcoin-etl-airflow

Airflow DAGs for https://github.com/blockchain-etl/bitcoin-etl

blockchain-kubernetes

Kubernetes manifests for running blockchain nodes

ethereum-etl-neo4j

ETL for moving Ethereum data to Neo4j database

bigquery-to-pubsub

A tool for streaming time series data from a BigQuery table to a Pub/Sub topic

bitcoin-etl-airflow-neo4j

Airflow DAGs for ingesting Bitcoin blockchain data to Neo4j

tezos-etl

Python scripts for ETL (extract, transform and load) jobs for Tezos blocks, balance updates, and operations

blockchain-etl-table-definition-cli

CLI for generating table definitions for https://github.com/blockchain-etl/ethereum-etl-airflow

hedera-etl

ETL scripts for Hedera Hashgraph

blockchain-streaming-analytics

Blockchain streaming analytics

eos-etl

ETL scripts for EOS.

ethereum-export-pipeline

UNMAINTAINED! AWS CloudFormation scripts for Ethereum ETL export pipeline

blockchain-etl-dataflow

Dataflow pipelines for Blockchain ETL. Connects Pub/Sub topics with BigQuery tables.

data-studio-connectors

Connect Google BigQuery crypto public datasets to Google Data Studio

abi-functions

blockchain-terraform-deployment

Template repository for deploying https://github.com/blockchain-etl/blockchain-terraform

icon-etl

Python scripts for ETL (extract, transform and load) jobs for ICON blocks, transactions, receipts, and logs.

ethereum2-etl-airflow

Airflow DAGs for exporting Ethereum 2.0 blockchain data to Google BigQuery

blockchain-terraform

Terraform configuration files for running blockchain nodes

blockchain-etl-common

Common utils for blockchain-etl

abi

EVM public good - pull requests welcome for any ABI from any EVM

band-etl

ETL (extract, transform and load) tools for ingesting Band Protocol blockchain data to Google BigQuery and Pub/Sub

tezos-kubernetes

Kubernetes manifests for running Tezos node

abi-parser

Web app which parses smart contracts and outputs queries and tables for Ethereum-ETL

iotex-etl

ETL (extract, transform and load) tools for ingesting IoTeX blockchain data to Google BigQuery and Pub/Sub

ordinals-etl

anomalous-transactions-detector-dataflow

Dataflow pipeline for detecting anomalous transactions on the Ethereum and Bitcoin blockchains

solana-etl

etl-rust

twitter-bot-cloud-function

Google Cloud Function for tweeting Blockchain ETL alerts

zilliqa-etl

Python scripts for ETL (extract, transform and load) jobs for Zilliqa blockchain data

pubsub-to-firestore-dataflow

Dataflow pipeline that pulls messages from a Pub/Sub topic and saves them in a Firestore collection

eos-etl-airflow

Airflow DAGs for https://github.com/blockchain-etl/eos-etl

bitcoin-rpc

Bitcoin JSON RPC client in Python

icon-etl-airflow

Airflow DAGs for exporting, loading, and parsing the ICON blockchain data.

tezos-etl-airflow

Airflow DAGs for exporting and loading the Tezos blockchain data to Google BigQuery

throttle-pubsub-cloud-function

Google Cloud Function that can throttle messages in a Pub/Sub topic

theta-etl

iotex-kubernetes

Helm charts for running IoTeX node