Join us on slack 
Lenses for Apache Kafka
Lenses offers SQL (for data browsing and Kafka Streams), Kafka Connect connector management, cluster monitoring and more.
You can find more on lenses.io
Stream Reactor
A collection of components to build a real time ingestion pipeline.
Kafka Compatibility
- Kafka 2.8 -> 3.3 (Confluent 2.8 -> 7.3) - Upcoming Stream Reactor 4.1.0
- Kafka 3.1 (Confluent 7.1) - Stream Reactor 4.0.0 (Kafka 3.1 Build)
- Kafka 2.8 (Confluent 6.2) - Stream Reactor 4.0.0 (Kafka 2.8 Build)
- Kafka 2.5 (Confluent 5.5) - Stream reactor 2.0.0+
- Kafka 2.0 -> 2.4 (Confluent 5.4) - Stream reactor 1.2.7
DEPRECATION NOTICE
The following connectors have been deprecated and are no longer included in future releases:
- Kudu
- Hive
The following connectors have been deprecated and are no longer included in any release from 3.0.
- Blockchain
- Bloomberg
- Rethink
- VoltDB
The following connectors have been deprecated and are no longer included in any release from 4.0.
- Coap
- Hive 1.1
Connectors
Please take a moment and read the documentation and make sure the software prerequisites are met!!
| Connector | Type | Description | Docs |
|---|---|---|---|
| AWS S3 | Sink | Copy data from Kafka to AWS S3. | Docs |
| AzureDocumentDb | Sink | Copy data from Kafka and Azure Document Db. | Docs |
| Cassandra | Source | Copy data from Cassandra to Kafka. | Docs |
| *Cassandra | Sink | Certified DSE Cassandra, copy data from Kafka to Cassandra. | Docs |
| Elastic 6 | Sink | Copy data from Kafka to Elastic Search 6.x w. tcp or http | Docs |
| FTP/HTTP | Source | Copy data from FTP/HTTP to Kafka. | Docs |
| Hazelcast | Sink | Copy data from Kafka to Hazelcast. | Docs |
| HBase | Sink | Copy data from Kafka to HBase. | Docs |
| Hive | Source | Copy data from Hive/HDFS to Kafka. | Docs |
| Hive | Sink | Copy data from Kafka to Hive/HDFS | Docs |
| InfluxDb | Sink | Copy data from Kafka to InfluxDb. | Docs |
| Kudu | Sink | Copy data from Kafka to Kudu. | Docs |
| JMS | Source | Copy data from JMS topics/queues to Kafka. | Docs |
| JMS | Sink | Copy data from Kafka to JMS. | Docs |
| MongoDB | Sink | Copy data from Kafka to MongoDB. | Docs |
| MQTT | Source | Copy data from MQTT to Kafka. | Docs |
| MQTT | Sink | Copy data from Kafka to MQTT. | Docs |
| Pulsar | Source | Copy data from Pulsar to Kafka. | Docs |
| Pulsar | Sink | Copy data from Kafka to Pulsar. | Docs |
| Redis | Sink | Copy data from Kafka to Redis. | Docs |
Release Notes
Please see the Stream Reactor Release Notes at Lenses Documentation.
Building
To build:
sbt clean compileTo test:
sbt testTo create assemblies:
sbt assemblyTo build a particular project:
sbt "project cassandra" compileTo test a particular project:
sbt "project cassandra" testTo create a jar of a particular project:
sbt "project cassandra" assemblyRunning E2E tests
If not already built, you must first build the connector archives:
sbt "project cassandra" assembly
sbt "project elastic6" assembly
sbt "project mongodb" assembly
sbt "project redis" assemblyTo run the tests:
sbt e2e:testContributing
We'd love to accept your contributions! Please use GitHub pull requests: fork the repo, develop and test your code, semantically commit and submit a pull request. Thanks!
