apache/arrow-datafusion

Stars
2,998
Rank 15,077 (Top 0.3 %)
Language
Rust
License
Apache License 2.0
Created over 3 years ago
Updated almost 2 years ago

apache/arrow-datafusion

apache

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Apache Arrow DataFusion SQL Query Engine

DataFusion

DataFusion is a very fast, extensible query engine for building high-quality data-centric systems in Rust, using the Apache Arrow in-memory format. Python Bindings are also available.

DataFusion offers SQL and Dataframe APIs, excellent performance, built-in support for CSV, Parquet, JSON, and Avro, extensive customization, and a great community.

https://arrow.apache.org/datafusion/ contains the project's documentation.

Using DataFusion

The example usage section in the user guide and the datafusion-examples code in the crate contain information on using DataFusion.

Contributing to DataFusion

The developer’s guide contains information on how to contribute.

echarts

Apache ECharts is a powerful, interactive charting and data visualization library for browser

superset

Apache Superset is a Data Visualization and Data Exploration Platform

dubbo

The java implementation of Apache Dubbo. An RPC and microservice framework.

spark

Apache Spark - A unified analytics engine for large-scale data processing

airflow

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

kafka

Mirror of Apache Kafka

incubator-seata

🔥 Seata is an easy-to-use, high-performance, open source distributed transaction solution.

skywalking

APM, Application Performance Monitoring System

flink

mxnet

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

shardingsphere

Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.

rocketmq

Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.

arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

brpc

brpc is an Industrial-grade RPC framework using C++ Language, which is often used in high performance system such as Search, Storage, Machine learning, Advertisement, Recommendation etc. "brpc" means "better RPC".

incubator-weex

Apache Weex (Incubating)

hadoop

pulsar

Apache Pulsar - distributed pub-sub messaging system

druid

Apache Druid: a high performance real-time analytics database.

predictionio

PredictionIO, a machine learning server for developers and ML engineers.

doris

Apache Doris is an easy-to-use, high performance and unified analytics database.

zookeeper

Apache ZooKeeper

incubator-answer

A Q&A platform software for teams at any scales. Whether it's a community forum, help center, or knowledge management platform, you can always count on Apache Answer.

apisix

The Cloud-Native API Gateway

thrift

dolphinscheduler

Apache DolphinScheduler is the modern data workflow orchestration platform with powerful user interface, dedicated to solving complex task dependencies in the data pipeline and providing various types of jobs available `out of the box`

cassandra

Mirror of Apache Cassandra

shardingsphere-elasticjob

Distributed scheduled job

seatunnel

SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.

tvm

Open deep learning compiler stack for cpu, gpu and specialized accelerators

beam

Apache Beam is a unified programming model for Batch and Streaming data processing.

shenyu

Apache ShenYu is a Java native API Gateway for service proxy, protocol conversion and API governance.

jmeter

Apache JMeter open-source load testing tool for analyzing and measuring the performance of a variety of services

tomcat

storm

iceberg

zeppelin

Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.

openwhisk

Apache OpenWhisk is an open source serverless cloud platform

couchdb

Seamless multi-master syncing database with an intuitive HTTP/JSON API, designed for reliability

iotdb

incubator-kie-drools

Drools is a rule engine, DMN engine and complex event processing (CEP) engine for Java.

pinot

Apache Pinot - A realtime distributed OLAP datastore

dubbo-spring-boot-project

Spring Boot Project for Apache Dubbo

mesos

hive

camel

Apache Camel is an open source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data.

groovy

Apache Groovy: A powerful multi-faceted programming language for the JVM platform

hbase

incubator-weex-ui

🏄 A rich interaction, lightweight, high performance UI library based on Weex.

ignite

rocketmq-externals

Mirror of Apache RocketMQ (Incubating)

incubator-pagespeed-ngx

Automatic PageSpeed optimization module for Nginx

lucene-solr

Apache Lucene and Solr open-source search software

dubbo-go

Go Implementation For Apache Dubbo

shiro

calcite

nifi

maven

Apache Maven core

hudi

Upserts, Deletes And Incremental Processing on Big Data.

dubbo-admin

The ops and reference implementation for Apache Dubbo

incubator-heron

Apache Heron (Incubating) is a realtime, distributed, fault-tolerant stream processing engine from Twitter

cordova-android

Apache Cordova Android

kylin

httpd

Mirror of Apache HTTP Server. Issues: http://issues.apache.org

linkis

Apache Linkis builds a computation middleware layer to facilitate connection, governance and orchestration between the upper applications and the underlying data engines.

incubator-kie-optaplanner

AI constraint solver in Java to optimize the vehicle routing problem, employee rostering, task assignment, maintenance scheduling, conference scheduling and other planning problems.

logging-log4j2

Apache Log4j 2 is a versatile, feature-rich, efficient logging API and backend for Java.

fury

A blazingly fast multi-language serialization framework powered by JIT and zero-copy.

curator

singa

a distributed deep learning platform

avro

Apache Avro is a data serialization system.

incubator-streampark

StreamPark, Make stream processing easier! easy-to-use streaming application development framework and operation platform

nutch

Apache Nutch is an extensible and scalable web crawler

netbeans

Apache NetBeans

guacamole-server

Mirror of Apache Guacamole Server

incubator-devlake

Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.

commons-lang

Apache Commons Lang

flume

Mirror of Apache Flume

geode

gobblin

A distributed data integration framework that simplifies common aspects of big data integration such as data ingestion, replication, organization and lifecycle management for both streaming and batch data ecosystems.

incubator-seata-samples

maven-mvnd

Apache Maven Daemon

activemq

Mirror of Apache ActiveMQ

incubator-hugegraph

A graph database that supports more than 100+ billion data, high performance and scalability (Include OLTP Engine & REST-API & Backends)

parquet-mr

kvrocks

Kvrocks is a distributed key value NoSQL database that uses RocksDB as storage engine and is compatible with Redis protocol.

pdfbox

Mirror of Apache PDFBox

cordova-ios

Apache Cordova iOS

mahout

Mirror of Apache Mahout

lucenenet

Apache Lucene.NET

ambari

Apache Ambari simplifies provisioning, managing, and monitoring of Apache Hadoop clusters.

incubator-pegasus

Apache Pegasus - A horizontally scalable, strongly consistent and high-performance key-value store

libcloud

Apache Libcloud is a Python library which hides differences between different cloud provider APIs and allows you to manage different cloud resources through a unified and easy to use API.

cloudstack

Apache CloudStack is an opensource Infrastructure as a Service (IaaS) cloud computing platform

tika

The Apache Tika toolkit detects and extracts metadata and text from over a thousand different file types (such as PPT, XLS, and PDF).

drill

Apache Drill is a distributed MPP query layer for self describing data

dubbo-samples

samples for Apache Dubbo

servicecomb-java-chassis

ServiceComb Java Chassis is a Software Development Kit (SDK) for rapid development of microservices in Java, providing service registration, service discovery, dynamic routing, and service management features

tinkerpop

Apache TinkerPop - a graph computing framework

trafficserver

Apache Traffic Server™ is a fast, scalable and extensible HTTP/1.1 and HTTP/2 compliant caching proxy server.

bookkeeper

Apache BookKeeper - a scalable, fault tolerant and low latency storage service optimized for append-only workloads