KVDK (Key-Value Development Kit) is a key-value store library implemented in C++ language. It is designed for supporting DRAM, Optane persistent memory and CXL memory pool. It also demonstrates several optimization methods for high performance with tiered memory. Besides providing the basic APIs of key-value store, it offers several advanced features, like read-modify-write, checkpoint, etc.
Features
- Rich data types
- string, sorted, hash, list, hash
- Basic KV operations
- get/put/update/delete/scan
- Read-Modify-Write
- Support TTL
- Atomic Batch Write
- Snapshot based Scan
- Consistent Dump & Restore to/from storage
- Consistent Checkpoint
- Transaction
- C/C++/Java APIs
Limitations
- The maximum supported key-value size is 64KB-4GB.
- No approach to key-value compression.
- Users can't expand the persistent memory space on the fly.
Getting the Source
git clone --recurse-submodules https://github.com/pmem/kvdk.gitBuilding
Install dependent tools and libraries on Ubuntu 18.04
sudo apt install make clang-format-9 pkg-config g++ autoconf libtool asciidoctor libkmod-dev libudev-dev uuid-dev libjson-c-dev libkeyutils-dev pandoc libhwloc-dev libgflags-dev libtext-diff-perl bash-completion systemd wget git curl
git clone https://github.com/pmem/ndctl.git
cd ndctl
git checkout v70.1
./autogen.sh
./configure CFLAGS='-g -O2' --prefix=/usr --sysconfdir=/etc --libdir=/usr/lib
make
sudo make install
git clone https://github.com/pmem/pmdk.git
cd pmdk
git checkout 1.11.1
make
sudo make install
wget https://github.com/Kitware/CMake/releases/download/v3.12.4/cmake-3.12.4.tar.gz
tar vzxf cmake-3.12.4.tar.gz
cd cmake-3.12.4
./bootstrap
make
sudo make install
Install dependent tools and libraries on CentOS 8
yum config-manager --add-repo /etc/yum.repos.d/CentOS-Linux-PowerTools.repo
yum config-manager --set-enabled PowerTools
yum install -y git gcc gcc-c++ autoconf automake asciidoc bash-completion xmlto libtool pkgconfig glib2 glib2-devel libfabric libfabric-devel doxygen graphviz pandoc ncurses kmod kmod-devel libudev-devel libuuid-devel json-c-devel keyutils-libs-devel gem make cmake libarchive clang-tools-extra hwloc-devel perl-Text-Diff gflags-devel curl
git clone https://github.com/pmem/ndctl.git
cd ndctl
git checkout v70.1
./autogen.sh
./configure CFLAGS='-g -O2' --prefix=/usr --sysconfdir=/etc --libdir=/usr/lib
make
sudo make install
git clone https://github.com/pmem/pmdk.git
cd pmdk
git checkout 1.11.1
make
sudo make install
wget https://github.com/Kitware/CMake/releases/download/v3.12.4/cmake-3.12.4.tar.gz
tar vzxf cmake-3.12.4.tar.gz
cd cmake-3.12.4
./bootstrap
make
sudo make installCompile KVDK
mkdir -p build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DCHECK_CPP_STYLE=ON && make -jHow to test it on a system without PMEM
# set the correct path for pmdk library
export LD_LIBRARY_PATH=/usr/local/lib64
# setup a tmpfs for test
mkdir /mnt/pmem0
mount -t tmpfs -o size=2G tmpfs /mnt/pmem0
# force the program work on non-pmem directory
export PMEM_IS_PMEM_FORCE=1
cd kvdk/build/examples
# Note: this requires CPU supporting AVX512
./cpp_api_tutorial
Benchmarks
Here are the examples of how to benchmark the performance of KVDK on your systems.
Documentations
User Guide
Please refer to User guide for API introductions of KVDK.
Architecture
Support
Welcome to join the wechat group or slack channel for KVDK tech discussion.