rufus-tokyo¶ ↑

ruby-ffi based interface to Tokyo Cabinet and Tokyo Tyrant.

NOTE : this library is in “maintenance mode”, see

jmettraux.wordpress.com/2010/02/26/retiring-rufus-tokyo/

for the explanation

installation¶ ↑

sudo gem install rufus-tokyo --source http://gemcutter.org

(see after 'usage' for how to install Tokyo Cabinet (and Tyrant) if required)

Rufus::Edo¶ ↑

Note : Rufus::Tokyo focuses on leveraging Hirabayashi-san’s C libraries via ruby-ffi, but the gem rufus-tokyo also contains Rufus::Edo which wraps the Tokyo Cabinet/Tyrant author’s [native] C bindings :

http://github.com/jmettraux/rufus-tokyo/tree/master/lib/rufus/edo

usage¶ ↑

hereafter TC references Tokyo Cabinet, while TT references Tokyo Tyrant.

the rdoc is at rufus.rubyforge.org/rufus-tokyo/

version [mis]match¶ ↑

this ‘master’ branch/version of rufus-tokyo works with

ffi 0.3.1, 0.3.5, 0.4.0, 0.5.0, 0.5.1

ruby 1.8.6
ruby 1.9.1p129
jruby 1.1.6, 1.2.0, 1.3.1
ree 1.8.7 2009.10 (thanks to http://rvm.beginrescueend.com)

TC 1.4.27 / 29 and TT 1.1.29 / 33

tc-ruby 1.26 / 29 and tt-ruby 1.10 / 12 (Rufus::Edo)

Rufus::Tokyo : pointing to the right library¶ ↑

By default Rufus::Tokyo (the FFI side) searches for Tokyo Cabinet and Tyrant dynamic libraries under /usr/lib/, /usr/lib64/, /opt/local/lib/ and /usr/local/lib/

If you had to install Tokyo Cabinet (and Tokyo Tyrant) in another location, you can set the environment variables TOKYO_CABINET_LIB and TOKYO_TYRANT_LIB to point to the absolute path of the dynamic library, like in :

export TOKYO_CABINET_LIB = /home/toto/tc/libtokyocabinet.so
export TOKYO_TYRANT_LIB = /home/toto/tc/libtokyotyrant.so

So that your rufus-tokyo enabled Ruby application can see the libraries.

TC Abstract API¶ ↑

tokyocabinet.sourceforge.net/spex-en.html#tcadbapi

to create a hash (file named ‘data.tch’)

require 'rubygems'
require 'rufus/tokyo'

db = Rufus::Tokyo::Cabinet.new('data.tch')

db['nada'] = 'surf'

p db['nada'] # => 'surf'
p db['lost'] # => nil

5000.times { |i| db[i.to_s] = "x" }

p db.inject { |r, (k, v)| k } # => 4999

db.close

TC Table API¶ ↑

tokyocabinet.sourceforge.net/spex-en.html#tctdbapi

require 'rubygems'
require 'rufus/tokyo'

t = Rufus::Tokyo::Table.new('table.tct')

t['pk0'] = { 'name' => 'alfred', 'age' => '22' }
t['pk1'] = { 'name' => 'bob', 'age' => '18' }
t['pk2'] = { 'name' => 'charly', 'age' => '45' }
t['pk3'] = { 'name' => 'doug', 'age' => '77' }
t['pk4'] = { 'name' => 'ephrem', 'age' => '32' }

p t.query { |q|
  q.add_condition 'age', :numge, '32'
  q.order_by 'age'
}
  # => [ {"name"=>"ephrem", :pk=>"pk4", "age"=>"32"},
  #      {"name"=>"charly", :pk=>"pk2", "age"=>"45"} ]

t.close

Note that the Tokyo Cabinet Table API does support transactions :

p t.size
  # => 0

t.transaction do
  t['pk0'] = { 'name' => 'alfred', 'age' => '22' }
  t['pk1'] = { 'name' => 'bob', 'age' => '18' }
  t.abort
end

p t.size
  # => 0

TT remote db¶ ↑

tokyocabinet.sourceforge.net/tyrantdoc/

to start a ttserver (backed by a hash), on the command line

ttserver -port 45001 data.tch

then, in Ruby :

require 'rubygems'
require 'rufus/tokyo/tyrant'

db = Rufus::Tokyo::Tyrant.new('localhost', 45001)

db['nada'] = 'surf'

p db['nada'] # => 'surf'
p db['lost'] # => nil

db.close

Rufus::Tokyo::Tyrant instances have a #stat method :

puts db.stat.inject('') { |s, (k, v)| s << "#{k} => #{v}\n" }
  # =>
  #   pid => 7566
  #   loadavg => 0.398438
  #   size => 528736
  #   rnum => 0
  #   time => 1234764065.305923
  #   sid => 898521513
  #   type => hash
  #   bigend => 0
  #   ru_sys => 3.398698
  #   version => 1.1.15
  #   ru_user => 2.155215
  #   ru_real => 3218.451152
  #   fd => 7

Note that it’s also OK to make a Tokyo Tyrant server listen on a unix socket :

ttserver -host /tmp/ttsocket -port 0 data.tch

and then :

require 'rubygems'
require 'rufus/tokyo/tyrant'
db = Rufus::Tokyo::Tyrant.new('/tmp/ttsocket')
db['a'] = 'alpha'
db.close

TT remote table¶ ↑

to start a ttserver (backed by a table), on the command line :

ttserver -port 45002 data.tct

then, in Ruby, much like a local table :

require 'rubygems'
require 'rufus/tokyo/tyrant'

t = Rufus::Tokyo::TyrantTable.new('localhost', 45002)

t['pk0'] = { 'name' => 'alfred', 'age' => '22' }
t['pk1'] = { 'name' => 'bob', 'age' => '18' }
t['pk2'] = { 'name' => 'charly', 'age' => '45' }
t['pk3'] = { 'name' => 'doug', 'age' => '77' }
t['pk4'] = { 'name' => 'ephrem', 'age' => '32' }

p t.query { |q|
  q.add_condition 'age', :numge, '32'
  q.order_by 'age'
}
  # => [ {"name"=>"ephrem", :pk=>"pk4", "age"=>"32"},
  #      {"name"=>"charly", :pk=>"pk2", "age"=>"45"} ]

t.close

Rufus::Tokyo::TyrantTable instances have a #stat method :

puts t.stat.inject('') { |s, (k, v)| s << "#{k} => #{v}\n" }
  # =>
  #   pid => 7569
  #   loadavg => 0.295410
  #   size => 935792
  #   rnum => 0
  #   time => 1234764228.942014
  #   sid => 1027604232
  #   type => table
  #   bigend => 0
  #   ru_sys => 5.966750
  #   version => 1.1.15
  #   ru_user => 2.601947
  #   ru_real => 3382.084479
  #   fd => 10

Note that it’s also OK to make a Tokyo Tyrant server listen on a unix socket :

ttserver -host /tmp/tttsocket -port 0 data.tct

and then :

require 'rubygems'
require 'rufus/tokyo/tyrant'
t = Rufus::Tokyo::TyrantTable.new('/tmp/tttsocket')
t['customer0'] = { 'name' => 'Heike no Kyomori', 'age' => '75' }
t.close

rdoc¶ ↑

more in the rdoc

• rufus.rubyforge.org/rufus-tokyo/

• rufus.rubyforge.org/rufus-tokyo/classes/Rufus/Tokyo/Cabinet.html

• rufus.rubyforge.org/rufus-tokyo/classes/Rufus/Tokyo/Table.html

• rufus.rubyforge.org/rufus-tokyo/classes/Rufus/Tokyo/Tyrant.html

• rufus.rubyforge.org/rufus-tokyo/classes/Rufus/Tokyo/TyrantTable.html

don’t hesitate to “man ttserver” on the command line.

or directly in the source

• github.com/jmettraux/rufus-tokyo/blob/master/lib/rufus/tokyo/cabinet/abstract.rb

• github.com/jmettraux/rufus-tokyo/blob/master/lib/rufus/tokyo/cabinet/table.rb

• github.com/jmettraux/rufus-tokyo/blob/master/lib/rufus/tokyo/tyrant/abstract.rb

• github.com/jmettraux/rufus-tokyo/blob/master/lib/rufus/tokyo/tyrant/table.rb

Tokyo Cabinet / Tyrant install¶ ↑

a compilation of notes is available at :

http://openwferu.rubyforge.org/tokyo.html

dependencies¶ ↑

the ruby gem ‘ffi’

mailing list¶ ↑

On the rufus-ruby list :

http://groups.google.com/group/rufus-ruby

Note : Flinn Mueller started an english speaking mailing list for TC / TT / TD

http://groups.google.com/group/tokyocabinet-users

issue tracker¶ ↑

http://github.com/jmettraux/rufus-tokyo/issues/

irc¶ ↑

irc.freenode.net #ruote

source¶ ↑

github.com/jmettraux/rufus-tokyo

git clone git://github.com/jmettraux/rufus-tokyo.git

credits¶ ↑

many thanks to the author of Tokyo Cabinet, Mikio Hirabayashi, and to the authors of ruby-ffi

• 1978th.net/tokyocabinet/

• 1978th.net/tokyotyrant/

• 1978th.net/tokyodystopia/

• wiki.github.com/ffi/ffi

dystopia interface developed with and sponsored by Seven Scale <sevenscale.com/>, creators of Open Syslog.

authors¶ ↑

• John Mettraux, [email protected], jmettraux.wordpress.com

• Zev Blut, www.iknow.co.jp/users/zev

• Jeremy Hinegardner github.com/copiousfreetime (Tokyo Dystopia)

• James Edward Gray II blog.grayproductions.net/

the rest of Rufus¶ ↑

rufus.rubyforge.org

license¶ ↑

MIT