goraft

A minimal implementation of Raft in Go. Demo video.

This blog post walks through the code.

NOT FOR PRODUCTION USE.

Things that are missing:

• Client serial identifier for each message (maybe the most obvious missing function/actual bug)
  • Search for "session" in Diego Ongaro's thesis for more details.
• Snapshotting/checkpointing and associated state transfer
• Configuration change protocol
• Rigged up to Jepsen
• Surely much else

Stress

Not particularly aggressive yet but does basic correctness and stress testing.

Read more about the details of what it tests in the README.

$ cd cmd/stress
$ go run main.go util.go

With the go-deadlock package turned off and the default sync package on, I get throughput of around 20k-40k entries/second with this stress test.

Distributed Key-Value Store API

Try out the builtin distributed key-value API.

$ cd cmd/kvapi && go build
$ rm *.dat

Terminal 1

$ ./kvapi --node 1 --http :2021 --cluster "0,:3030;1,:3031;2,:3032"

Terminal 2

$ ./kvapi --node 1 --http :2021 --cluster "0,:3030;1,:3031;2,:3032"

Terminal 3

$ ./kvapi --node 2 --http :2021 --cluster "0,:3030;1,:3031;2,:3032"

Terminal 4

Remember that requests will go through the leader (except for if we turn that off in the /get request). So you'll have to try sending a message to each server until you find the leader.

To set a key:

$ curl http://localhost:2020/set?key=y&value=hello

To get a key:

$ curl http://localhost:2020/get\?key\=y

References

• In Search of an Understandable Consensus Algorithm: The Raft paper.
• raft.tla: Diego Ongaro's TLA+ spec for Raft.
• Jon Gjengset's Students' Guide to Raft
• Jack Vanlightly's Detecting Bugs in Data Infrastructure using Formal Methods (TLA+ Series Part 1): An intro to TLA+.

And useful implementations I looked at for inspiration and clarity.

• Hashicorp's Raft implementation in Go: Although it's often quite complicated to learn from since it actually is intended for production.
• Eli Bendersky's Raft implementation in Go: Although it gets confusing because it uses negative numbers for terms whereas the paper does not.
• Jing Yang's Raft implementation in Rust.