There are no reviews yet. Be the first to send feedback to the community and the maintainers!
leveldb: A key-value store Authors: Sanjay Ghemawat ([email protected]) and Jeff Dean ([email protected]) The original Google README is now README.GOOGLE. ** Introduction This repository contains the Google source code as modified to benefit the Riak environment. The typical Riak environment has two attributes that necessitate leveldb adjustments, both in options and code: - production servers: Riak often runs in heavy Internet environments: servers with many CPU cores, lots of memory, and 24x7 disk activity. Basho's leveldb takes advantage of the environment by adding hardware CRC calculation, increasing Bloom filter accuracy, and defaulting to integrity checking enabled. - multiple databases open: Riak opens 8 to 128 databases simultaneously. Google's leveldb supports this, but its background compaction thread can fall behind. leveldb will "stall" new user writes whenever the compaction thread gets too far behind. Basho's leveldb modification include multiple thread blocks that each contain prioritized threads for specific compaction activities. Details for Basho's customizations exist in the leveldb wiki: http://github.com/basho/leveldb/wiki ** Branch pattern This repository follows the Basho standard for branch management as of November 28, 2013. The standard is found here: https://github.com/basho/riak/wiki/Basho-repository-management In summary, the "develop" branch contains the most recently reviewed engineering work. The "master" branch contains the most recently released work, i.e. distributed as part of a Riak release. ** Basic options needed Those wishing to truly savor the benefits of Basho's modifications need to initialize a new leveldb::Options structure similar to the following before each call to leveldb::DB::Open: leveldb::Options * options; options=new Leveldb::Options; options.filter_policy=leveldb::NewBloomFilterPolicy2(16); options.write_buffer_size=62914560; // 60Mbytes options.total_leveldb_mem=2684354560; // 2.5Gbytes (details below) options.env=leveldb::Env::Default(); ** Memory plan Basho's leveldb dramatically departed from Google's original internal memory allotment plan with Riak 2.0. Basho's leveldb uses a methodology called flexcache. The technical details are here: https://github.com/basho/leveldb/wiki/mv-flexcache The key points are: - options.total_leveldb_mem is an allocation for the entire process, not a single database - giving different values to options.total_leveldb_mem on subsequent Open calls causes memory to rearrange to current value across all databases - recommended minimum for Basho's leveldb is 340Mbytes per database. - performance improves rapidly from 340Mbytes to 2.5Gbytes per database (3.0Gbytes if using Riak's active anti-entropy). Even more is nice, but not as helpful. - never assign more than 75% of available RAM to total_leveldb_mem. There is too much unaccounted memory overhead (worse if you use tcmalloc library). - options.max_open_files and options.block_cache should not be used.
riak
Riak is a decentralized datastore from Basho Technologies.riak_core
Distributed systems infrastructure used by Riak.bitcask
because you need another a key/value storage enginerebar
ATTENTION: Please find the canonical repository here:riak_kv
Riak Key/Value Storeriak_cs
Riak CS is simple, available cloud storage built on Riak.erlang_protobuffs
An implementation of Google's Protocol Buffers for Erlang, based on ngerakines/erlang_protobuffs.riak_dt
Convergent replicated datatypes in Erlangriak-python-client
The Riak client for Python.riak-erlang-client
The Riak client for Erlang.basho_bench
A load-generation and testing tool for basically whatever you can write a returning Erlang function for.eleveldb
Erlang LevelDB APIriak-java-client
The Riak client for Java.yokozuna
Riak + Solrerlang_js
A linked-in driver for Erlang to Mozilla's Spidermonkey Javascript runtime.riak-ruby-client
The Riak client for Ruby.cuttlefish
never lose your childlike sense of wonder baby cuttlefish, promise me?basho_docs
Basho Products Documentationriak_ensemble
Multi-Paxos framework in Erlangriak-php-client
PHP clients for Riakriak_pipe
Riak Pipelinesclique
CLI Framework for Erlangriak_search
Full-text search engine based on Riakriak_control
Webmachine-based administration interface for Riak.machi
Machi file storeenm
Erlang driver for nanomsgsidejob
Parallel worker and capacity limiting library for Erlangriak-go-client
The Riak client for Go.node_package
RPM/Debian/FreeBSD/SmartOS/Solaris/OSX packaging templates for Erlang Nodesmerge_index
MergeIndex is an Erlang library for storing ordered sets on disk. It is very similar to an SSTable (in Google's Bigtable) or an HFile (in Hadoop).riak_function_contrib
Riak Function Contribmochiweb
a branch of Mochi Media's excellent HTTP library -- their canonical source can be found at https://github.com/mochi/mochiwebriak-dotnet-client
The Riak client for .NETriak_sysmon
Simple OTP app for managing Erlang VM system_monitor event messagesriak-nodejs-client
The Riak client for Node.js.riak_pb
Riak Protocol Buffers Messagesriak_test
I'm in your cluster, testing your riakscluster_info
Fork of Hibari's nifty cluster_info OTP appebloom
A NIF wrapper around a basic bloom filter.spark-riak-connector
The official Riak Spark Connector for Apache Spark with Riak TS and Riak KVinnertube
A thread-safe re-entrant resource pool for Ruby, extracted from the Riak Ruby Client.lager_syslog
Syslog backend for lagerriaknostic
A diagnostic tool for Riak installations, to find common errors asapriaktant
A node.js sample app that stores syslog messages in Riak Searchriak_repl
Riak DC Replicationgiddyup
Visual scorecard for riak_test.riak-erlang-http-client
Riak Erlang client using the HTTP interfaceriak_crdt_cookbook
A Cookbook full of Tutorials to get Developers started with Riak's CRDTsriak_err
Enhanced SASL Error Logger for Riakriak-hadoop
Riak data as input to hadoop m/r and output of hadoop m/rcloudformation-riak
Create Riak clusters with AWS CloudFormationinnostore
Innostore is a simple Erlang API to Embedded InnoDB.riak_ql
SQL query language for Riakriak_api
Riak Client APIsluke
Dataflow / MapReduce coordination framework.taste-of-riak
Source code for all the taste of riak examplesbasho_metrics
Fast performance metrics for Erlangskerl
Skein hash function for Erlang, via NIFsjam
Erlang time/date processingstanchion
Stanchion is an application to enforce the serialization of requests for Riak CS.dactyl
String templating library for Erlangfaulterl
Erlang glue & control code for dynamic library-level fault injectiontools.mk
A small makefile library for working with erlang toolsnifwait
Utility to test effect of blocking NIFs on Erlang schedulerrebar_raw_resource
A rebar3 resource wrapper to accommodate non-OTP-app dependenciescolumbo
Columbo - the dependency detective - will highlight 3rd party dependency problems in your Erlang programs.canola
Simple PAM port driver for erlangbashubot
Our own little hubot.riak_cs_control
Webmachine-based administration interface for Riak CS.bench_shim
A Jinterface shim between basho_bench and riak-java-clientriak_on_azure
basho-dtrace
DTrace scripts that Basho Team has found usefulriak-client-tools
Tools for Riak Client librariesriak-nodejs-client-examples
Riak Node.js Client example codestableboy
basho_harness VM / Harness provisionerriak_shell
A Repl for Riakriak-phppb-client
Official PHP Protocol Buffers Client for Riakbashobot
A friendly little bot for the #riak IRC channelriak-zabbix
A set of tools and templates to help monitor Riak with the Zabbix monitoring tool.riak_cs_auth
Authentication schemes for Riak CSsystest
Miscellaneous system test scriptsrecap-blog
[depreciated] The Riak Recap Blogcasbench
tools for benchmarking cassandrazdgrab
Zdgrab is a utility for downloading attachments to tickets from Zendesk.planet_riak
Riak Planetriak_cs_core
Core functionality for RIak CSdata_platform
Basho Data Platformdata_platform_core
Basho Data Platform Coreriak_auth_mods
Standard interface for security auth modules for Riakriak_cs_web
HTTP APIs for Riak Cloud Storagerfc
cluster_info_browser
In-browser app to easily flip through a cluster_info reportriak_ee-issues
Issue tracking for Riak Enterpriseriak_cs_lfs
Large file support for Riak CSripple-encryption
congruent
Generic test suite for Riak clientsriak_cs_multibag
Riak CS Multi-cluster moduleriak_cs_acl
ACL support for Riak CSriak_cs_report
Reporting functionality for Riak CSvelvet
Client library for StanchionLove Open Source and this site? Check out how you can help us