kni/redis-sharding

Redis Sharding
**************

Nota bene!

The  "Redis Sharding on Perl" project has been closed in favor of "Redis Sharding on Haskell" because the latter is 4 times more productive (http://github.com/kni/redis-sharding-hs). Standard ML version (https://github.com/kni/redis-sharding-sml) is even more productive.

Sharding
--------

https://github.com/kni/redis-sharding

Redis Sharding is a multiplexed proxy-server, designed to work with the database divided to several servers.
It's a temporary substitution of Redis Cluster (http://redis.io) that is under development.

Redis Sharding is used for horizontal Redis database scaling (with connecting of additional servers) as long as load distribution between the cores on the multiprocessor servers (as Redis server is single-threaded, several copies of the server can be run, one for each free core).

                              /- Redis (node 1)
 Client 1 ---                /-- Redis (node 2)
              Redis Sharding --- Redis (node 3)
 Client 2 ---                \-- Redis (node 4)
                              \- Redis (node 5)

Sharding is done based on the CRC32 checksum of a key or key tag ("key{key_tag}").

The majority of the commands are supported except the key renaming commands, 
commands of working with sets, transactions, subscription and, of course, server configure. 

For BLPOP and BRPOP commands all the keys should be on one node, that is done with the key tag.
Also for this commands Redis timeout 0 is recommended. 

Perl 5.10 or higher is needed with the following modules installed: String::CRC32, EV.

Launch:

 perl redis_sharding.pl                             --nodes=10.1.1.2:6380,10.1.1.3:6380,...
 perl redis_sharding.pl                 --port=6379 --nodes=10.1.1.2:6380,10.1.1.3:6380,...
 perl redis_sharding.pl --host=10.1.1.1 --port=6379 --nodes=10.1.1.2:6380,10.1.1.3:6380,...


Resharding
----------

Cluster configuration change on the fly is not supported. We are looking forward for Redis Cluster release. 

To copy data to a new cluster use a new utility resharding.pl.

For example, there is a cluster consisting of two servers: A1 and A2. It is served with redis_sharding, that is launched as

 perl redis_sharding.pl --nodes=A1,A2

We want to copy data from the base 9 to a new cluster of 5 servers: B1, B2, B3, B4 and B5.

To do this we need to stop redis_sharding.pl and run resharding.pl for each server of the cluster: 

 perl resharding.pl --db=9 --from=A1 --nodes=B1,B2,B3,B4,B5
 perl resharding.pl --db=9 --from=A2 --nodes=B1,B2,B3,B4,B5

After copying have been done, run redis_sharding.pl for a new cluster: 

perl redis_sharding.pl --nodes=B1,B2,B3,B4,B5

If a flag --flushdb is set, the FLUSHDB command is sent to all cluster nodes before data copying.