• Stars
    star
    195
  • Rank 194,764 (Top 4 %)
  • Language
    Go
  • License
    The Unlicense
  • Created almost 4 years ago
  • Updated 10 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Example code for how to get hashicorp/raft running with gRPC

raft-grpc-example

This is some example code for how to use Hashicorp's Raft implementation with gRPC.

Start your own cluster

$ mkdir /tmp/my-raft-cluster
$ mkdir /tmp/my-raft-cluster/node{A,B,C}
$ ./raft-grpc-example --raft_bootstrap --raft_id=nodeA --address=localhost:50051 --raft_data_dir /tmp/my-raft-cluster
$ ./raft-grpc-example --raft_id=nodeB --address=localhost:50052 --raft_data_dir /tmp/my-raft-cluster
$ ./raft-grpc-example --raft_id=nodeC --address=localhost:50053 --raft_data_dir /tmp/my-raft-cluster
$ go install github.com/Jille/raftadmin/cmd/raftadmin@latest
$ raftadmin localhost:50051 add_voter nodeB localhost:50052 0
$ raftadmin --leader multi:///localhost:50051,localhost:50052 add_voter nodeC localhost:50053 0
$ go run cmd/hammer/hammer.go &
$ raftadmin --leader multi:///localhost:50051,localhost:50052,localhost:50053 leadership_transfer
$ wait

You start up three nodes, and bootstrap one of them. Then you tell the bootstrapped node where to find peers. Those peers sync up to the state of the bootstrapped node and become members of the cluster. Once your cluster is running, you never need to pass --raft_bootstrap again.

raftadmin is used to communicate with the cluster and add the other nodes.

This example uses Jille/raft-grpc-transport to communicate between nodes using gRPC.

This example uses Jille/raft-grpc-leader-rpc to send RPCs to the leader.

Hammer is a client that connects to your raft cluster and sends a bunch of requests. Trigger some leadership failovers to show that it's unaffected.

What's what

Raft uses logs to synchronize changes. Every change submitted to a Raft cluster is a log entry, which gets stored and replicated to the followers in the cluster. In this example, we use raft-boltdb to store these logs. Once in a while Raft decides the logs have grown too large, and makes a snapshot. Your code is asked to write out its state. That state captures all previous logs. Now Raft can delete all the old logs and just use the snapshot. These snapshots are stored using the FileSnapshotStore, which means they'll just be files in your disk. Raft also needs a way to talk to other nodes, that's called a Transport. This example uses Jille/raft-grpc-transport to communicate between nodes using gRPC.

You can see all this happening in NewRaft() in main.go.

Your application

See application.go. You'll need to implement a raft.FSM, and you probably want a gRPC RPC interface.

More Repositories

1

raft-grpc-transport

Make hashicorp/raft communicate over gRPC
Go
42
star
2

dockpin

A tool for pinning Docker image and apt package versions
Go
31
star
3

raftadmin

gRPC wrapper for Hashicorp's raft.Raft object
Go
22
star
4

grpc-multi-resolver

Multiresolver allows you to Dial to multiple hosts/IPs as a single ClientConn.
Go
18
star
5

raft-grpc-leader-rpc

Send gRPCs to your Raft leader
Go
15
star
6

distwasm

Distributed system to run WebAssembly over many computers
Go
8
star
7

contextcond

A condvar like sync.Cond that can also wait for context cancellation.
Go
5
star
8

rufs

RUFS - easy multi-server file sharing
Go
4
star
9

rcf-m18

Library for controlling your RCF M18 headless mixer
Go
4
star
10

karafun-fair-scheduler

Tool to automatically order your karafun queue to give everyone their fair share.
Python
4
star
11

billy-grpc

billy-grpc implements the billy.Filesystem interface and calls another billy.Filesystem over gRPC.
Go
4
star
12

grpc-filesystem

A gRPC interface for a filesystem with client and server code.
Go
4
star
13

mpdswitcher

A proxy for MPD which switches between multiple MPD instances
Go
3
star
14

alarm-reporter

Simple Android app that sends an HTTP request whenever you set your alarm.
Java
2
star
15

FBP

File Broadcast Protocol
C
2
star
16

tiny-tcp-balancer

A tiny TCP balancer that sends requests to the first backend that acks the TCP connection.
Go
2
star
17

letsencrypt-with-etcd

A letsencrypt client that uses etcd as its storage.
Go
2
star
18

throughputbuffer

An efficient high throughput buffer in Go
Go
2
star
19

bindlink

A daemon to combine multiple uplinks together
Go
2
star
20

easymutex

A trivial mutex wrapper that allows you to unlock only once
Go
2
star
21

hrmoptimizer

Code optimizer for Human Resource Machine
Python
1
star
22

darn

Distributed Alerting and Reporting Nodes
Python
1
star
23

xenrescuemgr

[alpha] A rescue utility for people hosting virtual servers for others
C
1
star
24

billy-bazilfuse

Wrapper between bazil.org/fuse and go-billy.
Go
1
star
25

billy-logger

Debugging library that logs all calls made to a Billy filesystem.
Go
1
star
26

php-fdpass

[alpha] File-descriptor passing extension for PHP
C
1
star
27

rpcz

A library to show RPC samples for gRPC
Go
1
star
28

MiXML

Mixing songs using XML scripts
Python
1
star
29

voorstel-ict-opleiding

Een voorstel voor een nieuwe informatica opleiding
1
star
30

vvszaalrooster

Zaalrooster Villa van Schaeck
Python
1
star
31

splparser

Een parser voor SPL voor Compiler Construction op de RU
C
1
star
32

bitmpc

bitMPC for Android. Written at Bitendian Ltd.
Java
1
star
33

etcd-config-dispenser

Binary that fetches keys from etcd and writes them to disk and reloads its subprocess.
Go
1
star
34

billy-cgofuse

Wrapper between billziss-gh/cgofuse and go-billy.
Go
1
star
35

etcd-auth-dump

A library and binary to dump authentication configuration from etcd.
Go
1
star
36

etcd-client-from-env

A library to construct etcd/clientv3 connection params from environment variables
Go
1
star
37

human-nurikabe-solver

A Nurikabe solver that makes human-understandable steps.
Go
1
star
38

convreq

Library to make writing webservers in Go more convenient.
Go
1
star
39

mysql-rerep

MySQL replication setup script
PHP
1
star
40

dfr

dfr is a Go framework to help making defer more powerful/usable
Go
1
star
41

etcd-postgresql-sync

A simple tool to sync your etcd cluster to PostgreSQL in realtime.
Go
1
star
42

kbdwebmpc

A keyboard-only web interface to MPD
JavaScript
1
star
43

dingesform

Forms class
PHP
1
star
44

oid

Overwrite If Different
C
1
star
45

billy-router

billy-router allows to virtually mount billy.Filesystems over another to create the tree structure you need.
Go
1
star
46

trxwrap

WIP - Database transactions library for use with sqlc
Go
1
star
47

mysqlusersync

Manage MySQL-privileges through a config file
PHP
1
star
48

maandagseries

Software die random 8 mensen prikt voor de maandagseries
Python
1
star
49

deregexp

Deregexp deconstructs a regexp to an expression with substring filters.
Go
1
star