• This repository has been archived on 06/Jan/2023
  • Stars
    star
    2,044
  • Rank 22,628 (Top 0.5 %)
  • Language
    Clojure
  • License
    Eclipse Public Li...
  • Created almost 11 years ago
  • Updated about 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Distributed, masterless, high performance, fault tolerant data processing

Logo Onyx

Join the chat at https://gitter.im/onyx-platform/onyx

What is it?

  • a masterless, cloud scale, fault tolerant, high performance distributed computation system
  • batch and stream hybrid processing model
  • exposes an information model for the description and construction of distributed workflows
  • Competes against Storm, Flink, Cascading, Cascalog, Spark, Map/Reduce, Sqoop, etc
  • written in pure Clojure

What would I use this for?

  • Realtime event stream processing
  • CQRS
  • Continuous computation
  • Extract, transform, load
  • Data transformation à la map-reduce
  • Data ingestion and storage medium transfer
  • Data cleaning

Installation

Available on Clojars:

[org.onyxplatform/onyx "0.14.6"]

Changelog

Changelog can be found at changes.md.

Quick Lookup Doc

A searchable set of documentation for the Onyx data model is available.

Project Template

A project template can be found at onyx-template.

Plugins and Libraries

Plugin Template

We provide a plugin template for use in building new plugins. This can be found at onyx-plugin.

Plugin Use

To use the supported plugins, please use version coordinates such as [org.onyxplatform/onyx-amazon-sqs "0.14.6.SNAPSHOT.0"], and read the READMEs on the 0.14.x branches linked above.

Build Status

Component release unstable
onyx core Circle CI Circle CI
onyx-local-rt Circle CI Circle CI
onyx-kafka Circle CI Circle CI
onyx-datomic Circle CI Circle CI
onyx-redis Circle CI Circle CI
onyx-sql Circle CI Circle CI
onyx-bookkeeper Circle CI Circle CI
onyx-amazon-sqs Circle CI Circle CI
onyx-amazon-s3 Circle CI Circle CI
onyx-http Circle CI Circle CI
learn-onyx Circle CI -
onyx-examples Circle CI Circle CI
onyx-peer-http-query Circle CI Circle CI
lib-onyx Circle CI Circle CI
onyx-plugin Circle CI Circle CI
onyx-template Circle CI Circle CI
  • release: stable, released content
  • unstable: unreleased content

Unsupported plugins

Some plugins are currently unsupported in onyx 0.14.x. These are:

Companies Running Onyx in Production

LockedOn                                                                                               

Quick Start Guide

Feeling impatient? Hit the ground running ASAP with the onyx-starter repo and walkthrough. You can also boot into preloaded a Leiningen application template.

User Guide 0.14.6

Developer's Guide 0.14.6

API Docs 0.14.6

Code level API documentation can be found here.

Official plugin listing

Official plugins are vetted by Michael Drogalis. Ensure in your project that plugin versions directly correspond to the same Onyx version (e.g. onyx-kafka version 0.14.6.0-SNAPSHOT goes with onyx version 0.14.6). Fixes to plugins can be applied using a 4th versioning identifier (e.g. 0.14.6.1-SNAPSHOT).

Generate plugin templates through Leiningen with onyx-plugin.

3rd Party plugin listing

Unofficial plugins have not been vetted.

Need help?

Check out the Onyx Google Group.

Want the logo?

Feel free to use it anywhere. You can find a few different versions here.

Running the tests

A simple lein test will run the full suite for Onyx core.

Contributor list

Acknowledgements

Some code has been incorporated from the following projects:

License

Copyright © 2017 Michael Drogalis

Distributed under the Eclipse Public License, the same as Clojure.

More Repositories

1

learn-onyx

Learn Onyx with an interactive workshop.
Clojure
77
star
2

onyx-dashboard

Dashboard for the Onyx distributed processing system
JavaScript
76
star
3

onyx-starter

Starter tutorial for working with Onyx
Clojure
74
star
4

onyx-examples

Examples of using Onyx
Clojure
52
star
5

onyx-kafka

Onyx plugin for Kafka
Clojure
48
star
6

onyx-commander-example

An example of the Commander pattern with Kafka, Onyx, and Datomic.
Clojure
46
star
7

onyx-datomic

Onyx plugin for Datomic
Clojure
37
star
8

onyx-local-rt

A pure, deterministic Clojure(Script) runtime for Onyx
Clojure
32
star
9

onyx-template

Onyx Leiningen application template
Clojure
25
star
10

onyx-elasticsearch

Onyx plugin for ElasticSearch
Clojure
17
star
11

engraver

[Inactive project] A tool for deploying and managing Onyx clusters in the cloud
Python
17
star
12

empathy

[Work in progress] Sometimes things go wrong. We understand.
Clojure
15
star
13

onyx-twitter-sample

Example of an Onyx application using the live Twitter stream
Clojure
12
star
14

onyx-metrics

Instrument Onyx workflows.
Clojure
11
star
15

onyx-sql

Onyx plugin for JDBC-supported SQL databases
Clojure
10
star
16

ansible-onyx

[Inactive] Ansible playbook for Onyx
Clojure
9
star
17

onyx-benchmark

Benchmarking suite used internally by the Onyx team.
Clojure
8
star
18

lib-onyx

A library to support additional functionality in Onyx.
Clojure
8
star
19

beginners-guide

A guide for getting comfortable with Onyx
7
star
20

onyx-redis

Onyx plugin for Redis
Clojure
6
star
21

onyx-http

An Onyx output plugin for HTTP
Clojure
6
star
22

slides

Slides from talks about Onyx
6
star
23

onyx-bookkeeper

Onyx plugin for Apache BookKeeper
Clojure
6
star
24

onyx-console-dashboard

A console dashboard for Onyx
Clojure
5
star
25

charts

Helm charts and images for Onyx/Pyroclast
Shell
5
star
26

onyx-java

The Java API for Onyx
Java
5
star
27

onyx-twitter

Clojure
4
star
28

onyx-plugin

A Leiningen template for Onyx plugins
Clojure
4
star
29

onyx-visualization

Clojure
4
star
30

onyx-durable-queue

Onyx plugin for Factual's durable-queue
Clojure
4
star
31

docker-bookkeeper

Docker image for Apache BookKeeper.
Shell
3
star
32

onyx-peer-http-query

A library that supports pull-based health checks of Onyx peers
Clojure
3
star
33

onyx-platform.github.io

Website for the Onyx platform.
JavaScript
3
star
34

onyx-etl

[Not actively maintained] Extract, transform, load convenience Onyx package
Clojure
3
star
35

onyx-amazon-sqs

Onyx plugin for Amazon SQS
Clojure
3
star
36

onyx-cep

Complex event processing for Onyx via Metamorphic.
Clojure
3
star
37

docker-zookeeper

docker-zookeeper *SEE CHARTS*
Shell
3
star
38

onyx-amazon-kinesis

Onyx plugin for Amazon Kinesis
Clojure
2
star
39

onyx-spec

Clojure specs for Onyx core.
Clojure
2
star
40

jepsen-bookkeeper

Shell
2
star
41

onyx-seq

Onyx plugin to read from seqs (lazy or otherwise)
Clojure
2
star
42

onyx-jepsen

Jepsen test Onyx
Clojure
2
star
43

onyx-cheat-sheet

Project to generate the Cheat Sheet website.
CSS
2
star
44

pyroclast-support

Feature requests and bug tracking for Pyroclast.
1
star
45

onyx-amazon-s3

Amazon S3 plugin for Onyx
Clojure
1
star
46

onyx-ruby

Ruby API for Onyx
C++
1
star
47

engraver-kafka

Ansible Kafka role for Engraver
1
star