A totally proof-of-concept FoundationDB based NBD backend
I wanted to play around with FoundationDB a bit more and building a network block device (NBD) backend seemed like a good fit for that.
Do not use this in production, unless you like data loss, kernel crashes and you don't mind if your house burns down.
If you're interested in a more serious implementation, have a look at https://github.com/spullara/nbd.
Installation
- Install FoundationDB.
- Run
fdb-nbd.py
. It will start a local TCP server on port 10809 (default NBD port). - Load the
nbd
kernel module. - Run
nbd-client -N example 127.0.0.1 /dev/nbd0
. This will initialize the network block device/dev/nbd0
and point it to the started python server. - You might now format
/dev/nbd0
with any filesystem and mount it.
Cleaning up
- Unmount your filesystem
- Run
nbd-client -d /dev/nbd0
to disconnect the block device from the server
Worth noting
- The server can handle multiple block device "stores" at once. In the above example,
example
got selected with the-N
argument ofnbd-client
. Have a look at the source code offdb-nbd.py
to see how this is initialized. - You can list all other available "stores" with
nbd-client -l 127.0.0.1
- The server is hardcoded to use a blocksize of
1024
. Each block is stored in its own FoundationDB key('dev', 'example', 'blocks', block_nr)
. Partial reads or writes of blocks are not supported. - Since it's possible, I just compress/decompress each key before set/get. Yay.
- It's interesting to see how block device caching works. File system actions often don't directly cause block device operations. Play around with
sync
and flushing the cacheecho 3 > /proc/sys/vm/drop_caches
for maximum effect. - Don't suddenly stop the server or disconnect with
nbd-client
. The kernel can be a bit sensitive about this. I've had unkillable processes as a result and a kernel OOPS. You have been warned. - Performance isn't too good, at least in my tests. It's around 10MB/s or so with a locally running FoundationDB. But hey: It works :-)