riak_id
Overview
riak_id is a very simple application that serves both to
duplicate functionality from Twitter’s “Snowflake” and to teach,
via a very simple example, how to build a riak_core application.
Design
The core functionality of riak_id is to provide k-sorted unique
identifiers. The structure of the identifiers is very similar to
Snowflake. They are 64-bit integers with the following layout
(from highest to lowest):
| Bits | Description |
|---|---|
| 1 | Signedness flag, always 0 |
| 41 | Unix timestamp, down to the millisecond |
| 10 | Top 10 bits of partition number (see Caveats) |
| 12 | Per-partition strictly increasing counter |
Caveats
- Decent NTP synchronization between machines is needed for
riak_idto work as expected. - Request failures (e.g. from partition, timeout) should simply be retried. You will eventually get an ID.
- Because only 10 bits are reserved for the partition number, there is a practical ring-size limit of 1024. This should not be a problem because the work is mostly CPU-bound.
- So that we can give reasonable guarantees of uniqueness, some requests may have a minimum latency of 1ms (to allow the timestamp to increment).
- It will run out of ID space within the century. Snowflake solves this using the “Twepoch”. I didn’t bother with that, but it would be a simple feature to add.
- I wrote
riak_idto learn, I hope it will be useful to others.
Quick Start
riak_id requires Erlang/OTP, rebar and a sane GNU build system.
- Clone the repository, copy the
rebarscript into the repository. - Run
make relfrom the command line. rel/riak_id/bin/riak_id consoleriak_id:next_id().from the console to generate IDs.
Acknowledgments
Thanks to Jesse Newland, Andy Gross, Ryan Zezeski for their work on the rebar template for riak_core apps, without which I would be lost. Special thanks to Andy for pointing out some egregious errors in an early version of the vnode code.
License
Copyright 2011, Sean Cribbs. Licensed under Apache License 2.0.