• Stars
    star
    12,072
  • Rank 2,611 (Top 0.06 %)
  • Language
    C
  • License
    Apache License 2.0
  • Created over 12 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A fast, light-weight proxy for memcached and redis

twemproxy (nutcracker) Build Status

twemproxy (pronounced "two-em-proxy"), aka nutcracker is a fast and lightweight proxy for memcached and redis protocol. It was built primarily to reduce the number of connections to the caching servers on the backend. This, together with protocol pipelining and sharding enables you to horizontally scale your distributed caching architecture.

Build

To build twemproxy 0.5.0+ from distribution tarball:

$ ./configure
$ make
$ sudo make install

To build twemproxy 0.5.0+ from distribution tarball in debug mode:

$ CFLAGS="-ggdb3 -O0" ./configure --enable-debug=full
$ make
$ sudo make install

To build twemproxy from source with debug logs enabled and assertions enabled:

$ git clone [email protected]:twitter/twemproxy.git
$ cd twemproxy
$ autoreconf -fvi
$ ./configure --enable-debug=full
$ make
$ src/nutcracker -h

A quick checklist:

  • Use newer version of gcc (older version of gcc has problems)
  • Use CFLAGS="-O1" ./configure && make
  • Use CFLAGS="-O3 -fno-strict-aliasing" ./configure && make
  • autoreconf -fvi && ./configure needs automake and libtool to be installed

make check will run unit tests.

Older Releases

Distribution tarballs for older twemproxy releases (<= 0.4.1) can be found on Google Drive. The build steps are the same (./configure; make; sudo make install).

Features

  • Fast.
  • Lightweight.
  • Maintains persistent server connections.
  • Keeps connection count on the backend caching servers low.
  • Enables pipelining of requests and responses.
  • Supports proxying to multiple servers.
  • Supports multiple server pools simultaneously.
  • Shard data automatically across multiple servers.
  • Implements the complete memcached ascii and redis protocol.
  • Easy configuration of server pools through a YAML file.
  • Supports multiple hashing modes including consistent hashing and distribution.
  • Can be configured to disable nodes on failures.
  • Observability via stats exposed on the stats monitoring port.
  • Works with Linux, *BSD, OS X and SmartOS (Solaris)

Help

Usage: nutcracker [-?hVdDt] [-v verbosity level] [-o output file]
                  [-c conf file] [-s stats port] [-a stats addr]
                  [-i stats interval] [-p pid file] [-m mbuf size]

Options:
  -h, --help             : this help
  -V, --version          : show version and exit
  -t, --test-conf        : test configuration for syntax errors and exit
  -d, --daemonize        : run as a daemon
  -D, --describe-stats   : print stats description and exit
  -v, --verbose=N        : set logging level (default: 5, min: 0, max: 11)
  -o, --output=S         : set logging file (default: stderr)
  -c, --conf-file=S      : set configuration file (default: conf/nutcracker.yml)
  -s, --stats-port=N     : set stats monitoring port (default: 22222)
  -a, --stats-addr=S     : set stats monitoring ip (default: 0.0.0.0)
  -i, --stats-interval=N : set stats aggregation interval in msec (default: 30000 msec)
  -p, --pid-file=S       : set pid file (default: off)
  -m, --mbuf-size=N      : set size of mbuf chunk in bytes (default: 16384 bytes)

Zero Copy

In twemproxy, all the memory for incoming requests and outgoing responses is allocated in mbuf. Mbuf enables zero-copy because the same buffer on which a request was received from the client is used for forwarding it to the server. Similarly the same mbuf on which a response was received from the server is used for forwarding it to the client.

Furthermore, memory for mbufs is managed using a reuse pool. This means that once mbuf is allocated, it is not deallocated, but just put back into the reuse pool. By default each mbuf chunk is set to 16K bytes in size. There is a trade-off between the mbuf size and number of concurrent connections twemproxy can support. A large mbuf size reduces the number of read syscalls made by twemproxy when reading requests or responses. However, with a large mbuf size, every active connection would use up 16K bytes of buffer which might be an issue when twemproxy is handling large number of concurrent connections from clients. When twemproxy is meant to handle a large number of concurrent client connections, you should set chunk size to a small value like 512 bytes using the -m or --mbuf-size=N argument.

Configuration

Twemproxy can be configured through a YAML file specified by the -c or --conf-file command-line argument on process start. The configuration file is used to specify the server pools and the servers within each pool that twemproxy manages. The configuration files parses and understands the following keys:

  • listen: The listening address and port (name:port or ip:port) or an absolute path to sock file (e.g. /var/run/nutcracker.sock) for this server pool.
  • client_connections: The maximum number of connections allowed from redis clients. Unlimited by default, though OS-imposed limitations will still apply.
  • hash: The name of the hash function. Possible values are:
    • one_at_a_time
    • md5
    • crc16
    • crc32 (crc32 implementation compatible with libmemcached)
    • crc32a (correct crc32 implementation as per the spec)
    • fnv1_64
    • fnv1a_64 (default)
    • fnv1_32
    • fnv1a_32
    • hsieh
    • murmur
    • jenkins
  • hash_tag: A two character string that specifies the part of the key used for hashing. Eg "{}" or "$$". Hash tag enable mapping different keys to the same server as long as the part of the key within the tag is the same.
  • distribution: The key distribution mode for choosing backend servers based on the computed hash value. Possible values are:
  • timeout: The timeout value in msec that we wait for to establish a connection to the server or receive a response from a server. By default, we wait indefinitely.
  • backlog: The TCP backlog argument. Defaults to 512.
  • tcpkeepalive: A boolean value that controls if tcp keepalive is enabled for connections to servers. Defaults to false.
  • preconnect: A boolean value that controls if twemproxy should preconnect to all the servers in this pool on process start. Defaults to false.
  • redis: A boolean value that controls if a server pool speaks redis or memcached protocol. Defaults to false.
  • redis_auth: Authenticate to the Redis server on connect.
  • redis_db: The DB number to use on the pool servers. Defaults to 0. Note: Twemproxy will always present itself to clients as DB 0.
  • server_connections: The maximum number of connections that can be opened to each server. By default, we open at most 1 server connection.
  • auto_eject_hosts: A boolean value that controls if server should be ejected temporarily when it fails consecutively server_failure_limit times. See liveness recommendations for information. Defaults to false.
  • server_retry_timeout: The timeout value in msec to wait for before retrying on a temporarily ejected server, when auto_eject_hosts is set to true. Defaults to 30000 msec.
  • server_failure_limit: The number of consecutive failures on a server that would lead to it being temporarily ejected when auto_eject_hosts is set to true. Defaults to 2.
  • servers: A list of server address, port and weight (name:port:weight or ip:port:weight) for this server pool.

For example, the configuration file in conf/nutcracker.yml, also shown below, configures 5 server pools with names - alpha, beta, gamma, delta and omega. Clients that intend to send requests to one of the 10 servers in pool delta connect to port 22124 on 127.0.0.1. Clients that intend to send request to one of 2 servers in pool omega connect to unix path /tmp/gamma. Requests sent to pool alpha and omega have no timeout and might require timeout functionality to be implemented on the client side. On the other hand, requests sent to pool beta, gamma and delta timeout after 400 msec, 400 msec and 100 msec respectively when no response is received from the server. Of the 5 server pools, only pools alpha, gamma and delta are configured to use server ejection and hence are resilient to server failures. All the 5 server pools use ketama consistent hashing for key distribution with the key hasher for pools alpha, beta, gamma and delta set to fnv1a_64 while that for pool omega set to hsieh. Also only pool beta uses nodes names for consistent hashing, while pool alpha, gamma, delta and omega use 'host:port:weight' for consistent hashing. Finally, only pool alpha and beta can speak the redis protocol, while pool gamma, delta and omega speak memcached protocol.

alpha:
  listen: 127.0.0.1:22121
  hash: fnv1a_64
  distribution: ketama
  auto_eject_hosts: true
  redis: true
  server_retry_timeout: 2000
  server_failure_limit: 1
  servers:
   - 127.0.0.1:6379:1

beta:
  listen: 127.0.0.1:22122
  hash: fnv1a_64
  hash_tag: "{}"
  distribution: ketama
  auto_eject_hosts: false
  timeout: 400
  redis: true
  servers:
   - 127.0.0.1:6380:1 server1
   - 127.0.0.1:6381:1 server2
   - 127.0.0.1:6382:1 server3
   - 127.0.0.1:6383:1 server4

gamma:
  listen: 127.0.0.1:22123
  hash: fnv1a_64
  distribution: ketama
  timeout: 400
  backlog: 1024
  preconnect: true
  auto_eject_hosts: true
  server_retry_timeout: 2000
  server_failure_limit: 3
  servers:
   - 127.0.0.1:11212:1
   - 127.0.0.1:11213:1

delta:
  listen: 127.0.0.1:22124
  hash: fnv1a_64
  distribution: ketama
  timeout: 100
  auto_eject_hosts: true
  server_retry_timeout: 2000
  server_failure_limit: 1
  servers:
   - 127.0.0.1:11214:1
   - 127.0.0.1:11215:1
   - 127.0.0.1:11216:1
   - 127.0.0.1:11217:1
   - 127.0.0.1:11218:1
   - 127.0.0.1:11219:1
   - 127.0.0.1:11220:1
   - 127.0.0.1:11221:1
   - 127.0.0.1:11222:1
   - 127.0.0.1:11223:1

omega:
  listen: /tmp/gamma 0666
  hash: hsieh
  distribution: ketama
  auto_eject_hosts: false
  servers:
   - 127.0.0.1:11214:100000
   - 127.0.0.1:11215:1

Finally, to make writing a syntactically correct configuration file easier, twemproxy provides a command-line argument -t or --test-conf that can be used to test the YAML configuration file for any syntax error.

Observability

Observability in twemproxy is through logs and stats.

Twemproxy exposes stats at the granularity of server pool and servers per pool through the stats monitoring port by responding with the raw data over TCP. The stats are essentially JSON formatted key-value pairs, with the keys corresponding to counter names. By default stats are exposed on port 22222 and aggregated every 30 seconds. Both these values can be configured on program start using the -c or --conf-file and -i or --stats-interval command-line arguments respectively. You can print the description of all stats exported by using the -D or --describe-stats command-line argument.

$ nutcracker --describe-stats

pool stats:
  client_eof          "# eof on client connections"
  client_err          "# errors on client connections"
  client_connections  "# active client connections"
  server_ejects       "# times backend server was ejected"
  forward_error       "# times we encountered a forwarding error"
  fragments           "# fragments created from a multi-vector request"

server stats:
  server_eof          "# eof on server connections"
  server_err          "# errors on server connections"
  server_timedout     "# timeouts on server connections"
  server_connections  "# active server connections"
  requests            "# requests"
  request_bytes       "total request bytes"
  responses           "# responses"
  response_bytes      "total response bytes"
  in_queue            "# requests in incoming queue"
  in_queue_bytes      "current request bytes in incoming queue"
  out_queue           "# requests in outgoing queue"
  out_queue_bytes     "current request bytes in outgoing queue"

See notes/debug.txt for examples of how to read the stats from the stats port.

Logging in twemproxy is only available when twemproxy is built with logging enabled. By default logs are written to stderr. Twemproxy can also be configured to write logs to a specific file through the -o or --output command-line argument. On a running twemproxy, we can turn log levels up and down by sending it SIGTTIN and SIGTTOU signals respectively and reopen log files by sending it SIGHUP signal.

Pipelining

Twemproxy enables proxying multiple client connections onto one or few server connections. This architectural setup makes it ideal for pipelining requests and responses and hence saving on the round trip time.

For example, if twemproxy is proxying three client connections onto a single server and we get requests - get key\r\n, set key 0 0 3\r\nval\r\n and delete key\r\n on these three connections respectively, twemproxy would try to batch these requests and send them as a single message onto the server connection as get key\r\nset key 0 0 3\r\nval\r\ndelete key\r\n.

Pipelining is the reason why twemproxy ends up doing better in terms of throughput even though it introduces an extra hop between the client and server.

Deployment

If you are deploying twemproxy in production, you might consider reading through the recommendation document to understand the parameters you could tune in twemproxy to run it efficiently in the production environment.

Utils

Companies using Twemproxy in Production

Issues and Support

Have a bug or a question? Please create an issue here on GitHub!

https://github.com/twitter/twemproxy/issues

Committers

Thank you to all of our contributors!

License

Copyright 2012 Twitter, Inc.

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

More Repositories

1

the-algorithm

Source code for Twitter's Recommendation Algorithm
Scala
61,569
star
2

twemoji

Emoji for everyone. https://twemoji.twitter.com/
HTML
16,673
star
3

typeahead.js

typeahead.js is a fast and fully-featured autocomplete library
JavaScript
16,528
star
4

the-algorithm-ml

Source code for Twitter's Recommendation Algorithm
Python
9,938
star
5

finagle

A fault tolerant, protocol-agnostic RPC system
Scala
8,752
star
6

hogan.js

A compiler for the Mustache templating language
JavaScript
5,142
star
7

labella.js

Placing labels on a timeline without overlap.
JavaScript
3,875
star
8

scala_school

Lessons in the Fundamentals of Scala
HTML
3,700
star
9

AnomalyDetection

Anomaly Detection with R
R
3,534
star
10

scalding

A Scala API for Cascading
Scala
3,478
star
11

twitter-text

Twitter Text Libraries. This code is used at Twitter to tokenize and parse text to meet the expectations for what can be used on the platform.
HTML
3,059
star
12

TwitterTextEditor

A standalone, flexible API that provides a full-featured rich text editor for iOS applications.
Swift
2,966
star
13

opensource-website

Twitter's open source website, identifying projects we've released, organizations we support, and the work we do to support open source.
SCSS
2,957
star
14

util

Wonderful reusable code from Twitter
Scala
2,679
star
15

algebird

Abstract Algebra for Scala
Scala
2,284
star
16

finatra

Fast, testable, Scala services built on TwitterServer and Finagle
Scala
2,273
star
17

effectivescala

Twitter's Effective Scala Guide
HTML
2,242
star
18

summingbird

Streaming MapReduce with Scalding and Storm
Scala
2,139
star
19

pelikan

Pelikan is Twitter's unified cache backend
C
1,933
star
20

ios-twitter-image-pipeline

Twitter Image Pipeline is a robust and performant image loading and caching framework for iOS clients
C
1,852
star
21

twurl

OAuth-enabled curl for the Twitter API
Ruby
1,789
star
22

twitter-server

Twitter-Server defines a template from which services at Twitter are built
Scala
1,559
star
23

rezolus

Systems performance telemetry
Rust
1,545
star
24

communitynotes

Documentation and source code powering Twitter's Community Notes
Python
1,362
star
25

activerecord-reputation-system

An Active Record Reputation System for Rails
Ruby
1,335
star
26

compose-rules

Static checks to aid with a healthy adoption of Compose
Kotlin
1,335
star
27

fatcache

Memcache on SSD
C
1,299
star
28

rsc

Experimental Scala compiler focused on compilation speed
Scala
1,243
star
29

elephant-bird

Twitter's collection of LZO and Protocol Buffer-related Hadoop, Pig, Hive, and HBase code.
Java
1,137
star
30

cassovary

Cassovary is a simple big graph processing library for the JVM
Scala
1,040
star
31

Serial

Light-weight, fast framework for object serialization in Java, with Android support.
Java
991
star
32

hbc

A Java HTTP client for consuming Twitter's realtime Streaming API
Java
961
star
33

twemcache

Twemcache is the Twitter Memcached
C
926
star
34

vireo

Vireo is a lightweight and versatile video processing library written in C++11
C++
924
star
35

innovators-patent-agreement

Innovators Patent Agreement (IPA)
921
star
36

twitter-korean-text

Korean tokenizer
Scala
856
star
37

scrooge

A Thrift parser/generator
Scala
787
star
38

BreakoutDetection

Breakout Detection via Robust E-Statistics
C++
753
star
39

GraphJet

GraphJet is a real-time graph processing library.
Java
699
star
40

twitter-cldr-rb

Ruby implementation of the ICU (International Components for Unicode) that uses the Common Locale Data Repository to format dates, plurals, and more.
Ruby
669
star
41

bijection

Reversible conversions between types
Scala
657
star
42

chill

Scala extensions for the Kryo serialization library
Scala
609
star
43

ios-twitter-network-layer

Twitter Network Layer is a scalable and feature rich network layer built on top of NSURLSession for Apple platforms
Objective-C
573
star
44

hadoop-lzo

Refactored version of code.google.com/hadoop-gpl-compression for hadoop 0.20
Shell
545
star
45

storehaus

Storehaus is a library that makes it easy to work with asynchronous key value stores
Scala
465
star
46

rpc-perf

A tool for benchmarking RPC services
Rust
458
star
47

d3kit

D3Kit is a set tools to speed D3 related project development
JavaScript
428
star
48

scoot

Scoot is a distributed task runner, supporting both a proprietary API and Bazel's Remote Execution.
Go
347
star
49

twitter-cldr-js

JavaScript implementation of the ICU (International Components for Unicode) that uses the Common Locale Data Repository to format dates, plurals, and more. Based on twitter-cldr-rb.
JavaScript
347
star
50

rustcommon

Common Twitter Rust lib
Rust
341
star
51

scala_school2

Scala School 2
Scala
340
star
52

wordpress

The official Twitter plugin for WordPress. Embed Twitter content and grow your audience on Twitter.
PHP
315
star
53

ios-twitter-logging-service

Twitter Logging Service is a robust and performant logging framework for iOS clients
Objective-C
299
star
54

nodes

A library to implement asynchronous dependency graphs for services in Java
Java
246
star
55

SentenTree

A novel text visualization technique
JavaScript
227
star
56

interactive

Twitter interactive visualization
HTML
214
star
57

joauth

A Java library for authenticating HTTP Requests using OAuth
Java
213
star
58

thrift_client

A Thrift client wrapper that encapsulates some common failover behavior
Ruby
196
star
59

hpack

Header Compression for HTTP/2
Java
192
star
60

cache-trace

A collection of Twitter's anonymized production cache traces.
Shell
166
star
61

zktraffic

ZooKeeper protocol analyzer and stats gathering daemon
Python
166
star
62

twemoji-parser

A simple library for identifying emoji entities within a string in order to render them as Twemoji.
Scala
163
star
63

sbf

Java
161
star
64

tormenta

Scala extensions for Storm
Scala
132
star
65

whiskey

HTTP library for Android (beta)
Java
131
star
66

hraven

hRaven collects run time data and statistics from MapReduce jobs in an easily queryable format
Java
127
star
67

netty-http2

HTTP/2 for Netty
Java
120
star
68

sqrl

A Safe, Stateful Rules Language for Event Streams
TypeScript
100
star
69

ccommon

Cache Commons
C
99
star
70

focus

Focus aligns Git worktree content based on outlines of a repository's Bazel build graph. Focused repos are sparse, shallow, and thin and unlock markedly better performance in large repos.
Rust
91
star
71

metrics

78
star
72

dict_minimize

Access scipy optimizers from your favorite deep learning framework.
Python
77
star
73

twitter.github.io

HTML
74
star
74

diffusion-rl

Python
68
star
75

go-bindata

Go
68
star
76

birdwatch

67
star
77

cloudhopper-commons

Cloudhopper Commons
Java
57
star
78

.github

Twitter GitHub Organization-wide files
49
star
79

twitter-cldr-npm

TwitterCldr npm package
JavaScript
49
star
80

bazel-multiversion

Bazel rules to resolve, fetch and manage 3rdparty JVM dependencies with support for multiple parallel versions of the same dependency. Powered by Coursier.
Scala
47
star
81

libwatchman

A C interface to watchman
C
44
star
82

sslconfig

Twitter's OpenSSL Configuration
43
star
83

ios-twitter-apache-thrift

A thrift encoding and decoding library for Swift
Swift
42
star
84

gatekeeper-service

GateKeeper is a service built to automate the manual steps involved in onboarding, offboarding, and lost asset scenarios.
Python
36
star
85

dodo

The Twitter OSS Project Builder
Shell
35
star
86

repo-scaffolding

Tools for creating repos based on open source standards and best practices
34
star
87

iago2

A load generator, built for engineers
Scala
25
star
88

caladrius

Performance modelling system for Distributed Stream Processing Systems (DSPS) such as Apache Heron and Apache Storm
Python
22
star
89

ossdecks

Repository for Twitter Open Source Decks
10
star
90

curation-style-guide

Document Repository for Twitter's Curation Style Guide
10
star
91

analytics-infra-governance

Description of the process for how to commit, review, and release code to the Scalding OSS family (Scalding, Summingbird, Algebird, Bijection, Storehaus, etc)
9
star
92

gpl-commitment

Twitter's GPL Cooperation Commitment
5
star
93

second-control-probability-distributions

4
star
94

google-tag-manager-event-tag

Smarty
3
star
95

google-tag-manager-base-tag

Smarty
2
star