• Stars
    star
    193
  • Rank 201,081 (Top 4 %)
  • Language
    Java
  • License
    Apache License 2.0
  • Created over 6 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

The Open source Resource as Code framework for Apache Kafka. Jikkou helps you implement GitOps for Kafka at scale!

Jikkou!

Introduction

Jikkou (jikkล / ๅฎŸ่กŒ) is an open-source tool designed to provide an efficient and easy way to manage, automate, and provision resource configurations for Kafka, Schema Registry, etc.

Developed by Kafka โค๏ธ, Jikkou aims to streamline daily operations on Apache Kafka, ensuring that platform governance is no longer a boring and tedious task for both Developers and Administrators.

Jikkou enables a declarative management approach of Topics, ACLs, Quotas, Schemas and even more with the use of YAML files called Resource Definitions.

Taking inspiration from kubectl and Kubernetes resource definition files, Jikkou offers an intuitive and user-friendly approach to configuration management.

Jikkou can be used with Apache Kafka, Aiven, MSK, Confluent Cloud, Redpanda.

Motivation

See this blog post for more details.

Installation

The latest stable release of jikkou (x86) for Linux, and macOS can be downloaded from GitHub Releases

Below are the convenience links for the base downloads of Jikkou.

Platform Link

Linux

โฌ‡๏ธ download (jikkou-0.29.0-linux-x86_64.zip)

macOS

โฌ‡๏ธ download (jikkou-0.29.0-osx-x86_64.zip)

Alternatively, jikkou is also available:

Download the jikkou binary from the releases page, uncompress and copy to the desired location.

# Download the latest stable release
wget https://github.com/streamthoughts/jikkou/releases/download/v0.29.0/jikkou-0.29.0-linux-x86_64.zip

# Uncompress
unzip jikkou-0.29.0-linux-x86_64.zip

# Copy to the desired location
cp jikkou-0.29.0-linux-x86_64/bin/jikkou $HOME/.local/bin

It is recommended to install the bash/zsh completion script jikkou_completion:

wget https://raw.githubusercontent.com/streamthoughts/jikkou/main/jikkou_completion . jikkou_completion

or alternatively, run the following command for generation the completion script.

source <(jikkou generate-completion)
Warning
If you are using macOS you may need to remove the quarantine attribute from the bits before you can use them To do this, run the following: sudo xattr -r -d com.apple.quarantine path/to/jikkou/folder/

Overview

Here is an example of how to create and manage a Kafka topic using Jikkou:

  • Create a resource file kafka-topics.yaml:

    kafka-topics.yaml
# file:./kafka-topics.yaml
apiVersion: 'kafka.jikkou.io/v1beta2'
kind: 'KafkaTopic'
metadata:
  name: 'my-first-topic-with-jikkou'
  labels: {}
  annotations: {}
spec:
  partitions: 12
  replicas: 3
  configs:
    min.insync.replicas: 2
  • Then run the following command:

$ jikkou apply --files ./kafka-topics.yaml

Jikkou will then take care of computing and applying the necessary changes directly to your cluster.

(output):

TASK [CREATE] Create a new topic my-first-topic-with-jikkou (partitions=12, replicas=3) - CHANGED **********************
{
  "changed" : true,
  "end" : 1634071489773,
  "resource" : {
    "name" : "my-first-topic-with-jikkou",
    "operation" : "ADD",
    "partitions" : {
      "after" : 12,
      "operation" : "ADD"
    },
    "replicas" : {
      "after" : 3,
      "operation" : "ADD"
    },
    "configs" : {
      "min.insync.replicas" : {
        "after" : "2",
        "operation" : "ADD"
      }
    }
  },
  "failed" : false,
  "status" : "CHANGED"
}
EXECUTION in 2s 661ms (DRY_RUN)
ok : 0, created : 1, altered : 0, deleted : 0 failed : 0

Documentation

Check the official documentation for further installation and usage instructions.

๐Ÿญ Developers

You need to have Java and Docker installed.

Dependencies

Jikkou CLI is built with Micronaut and Picocli

To build the project you will need:

  • Java 17

  • GraalVM 22.1.0 or newer to create native executable

  • TestContainer to run integration tests

Build project

This project includes Maven Wrapper.

Below are the commands commonly used to build the project:

# Build and run all tests
./mvnw clean verify

# Build and skip integration tests
./mvnw clean verify -DskipTests

Create Native Executable

# Build and run all tests
./mvnw clean verify -Pnative

You can then execute the native executable with: ./jikkou-cli/target/jikkou-$PROJECT_VERSION-runner

Create Debian Package (on Linux)

# Build and run all tests
./mvnw clean package -Pnative
./mvnw package -Pdeb

You can then install the package with: sudo dpkg -i ./dist/jikkou-$PROJECT_VERSION-linux-x86_64.deb

Note
Jikkou will install itself in the directory : /opt/jikkou

Formats

This project uses the Maven plugin Spotless to format all Java classes and to apply some code quality checks.

Bugs

This project uses the Maven plugin SpotBugs and FindSecBugs to run some static analysis to look for bugs in Java code.

Reported bugs can be analysed using SpotBugs GUI:

$ ./mvnw spotbugs:gui

๐Ÿ’ก Contributions

Any feedback, bug reports and PRs are greatly appreciated!

๐Ÿ™ Show your support

You think this project can help you or your team to manage your Apache Kafka Cluster ? Please โญ this repository to support us!

Licence

This code base is available under the Apache License, version 2.

More Repositories

1

kafka-connect-file-pulse

๐Ÿ”— A multipurpose Kafka Connect connector that makes it easy to parse, transform and stream any file, in any format, into Apache Kafka
Java
321
star
2

azkarra-streams

๐Ÿš€ Azkarra is a lightweight java framework to make it easy to develop, deploy and manage cloud-native streaming microservices based on Apache Kafka Streams.
HTML
186
star
3

kafka-monitoring-stack-docker-compose

This project provides Docker compose files to deploy an Apache Kafka platform with a monitoring stack using Prometheus and Grafana
Shell
134
star
4

awesome-opensource-contribs-kafka

A list of all awesome open-source contributions for the Apache Kafka project
95
star
5

kafka-clients-kotlin

This projects gives Kotlin bindings and several extensions for Apache Kafka Clients.
Kotlin
40
star
6

apache-pulsar-cheat-sheet

This cheat sheet is filled with some handy tips, commands and code snippets to get you streaming data using Apache Pulsar in no time!
Java
28
star
7

demo-twitter-ksqldb-clickhouse

Demonstration Project : Fast Data Analytic platform with Clickhouse, Apache Kafka and ksqlDB
Java
20
star
8

kafka-connect-transform-grok

Grok Expression Transform for Kafka Connect.
Java
16
star
9

kafka-consumer-partition-assignor

This project contains examples to implement a custom PartitionAssignor for Apache Kafka consumer-client.
Java
12
star
10

kafka-connect-client

Java client library for Apache Kafka Connect REST Interface
Java
6
star
11

kc4streams

A collection of common utilities and reusable classes for Kafka Streams applications.
Java
6
star
12

pulsar-quickstart-java

This project demonstrates how to write simple producers and consumers for Apache Pulsar
Java
3
star
13

demo-kafka-streams-scottify

Scottify is the official media services provider of Starfleet ๐Ÿ––
Java
3
star
14

azkarra-streaming-count

This project demonstrates how to build and containerize a Kafka Streams ap using Akzarra and Jib
Java
1
star
15

spark-streaming-examples

This project contains simple code skeleton for developing a Spark Streaming application using Kafka integration.
Java
1
star
16

setup-jikkou

Jikkou GitHub Actions
JavaScript
1
star
17

kafka-streams-examples

This project contains code examples that demonstrate how to use the KafkaStreams Streams DSL and Processor API.
Java
1
star