• Stars
    star
    1,018
  • Rank 45,191 (Top 0.9 %)
  • Language
    Rust
  • License
    Apache License 2.0
  • Created over 3 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A developer-friendly document database that grows with you, written in Rust

BonsaiDb

BonsaiDb forbids unsafe code BonsaiDb is considered alpha crate version Live Build Status HTML Coverage Report for main Documentation for main

BonsaiDb is a developer-friendly document database for Rust that grows with you. It offers many features out of the box that many developers need:

  • ACID-compliant, transactional storage of Collections
  • Atomic Key-Value storage with configurable delayed persistence (similar to Redis)
  • At-rest Encryption
  • Backup/Restore
  • Role-Based Access Control (RBAC)
  • Local-only access, networked access via QUIC, or networked access via WebSockets
  • And much more.

⚠️ Status of this project

BonsaiDb is considered alpha software. It is under active development (GitHub commit activity). There may still be bugs that result in data loss. All users should regularly back up their data and test that restoring from backup works correctly.

Around May 2022, a bug and a mistake in benchmarking were discovered. The bug was promptly fixed, but the net result is that BonsaiDb's transactional write performance is significantly slower than other databases. Unless you're buliding a very write-heavy application, the performance will likely still be acceptable. Issue #251 on GitHub is where progress of the performance updates are being tracked. From a developer's perspective, migration is expected to be painless beyond the IO needed to copy the old database into the new format.

Example

To get an idea of how it works, let's review the view-examples example. See the examples README for a list of all available examples.

The view-examples example shows how to define a simple schema containing a single collection (Shape), a view to query the Shapes by their number_of_sides (ShapesByNumberOfSides), and demonstrates multiple ways to query that view.

First, here's how the schema is defined:

#[derive(Debug, Serialize, Deserialize, Collection)]
#[collection(name = "shapes", views = [ShapesByNumberOfSides])]
struct Shape {
    pub sides: u32,
}

#[derive(Debug, Clone, View, ViewSchema)]
#[view(collection = Shape, key = u32, value = usize, name = "by-number-of-sides")]
struct ShapesByNumberOfSides;

impl CollectionMapReduce for ShapesByNumberOfSides {
    fn map<'doc>(&self, document: CollectionDocument<Shape>) -> ViewMapResult<'doc, Self::View> {
        document
            .header
            .emit_key_and_value(document.contents.sides, 1)
    }

    fn reduce(
        &self,
        mappings: &[ViewMappedValue<'_, Self>],
        _rereduce: bool,
    ) -> ReduceResult<Self::View> {
        Ok(mappings.iter().map(|m| m.value).sum())
    }
}

After you have your collection(s) and view(s) defined, you can open up a database and insert documents:

let db = Database::open::<Shape>(StorageConfiguration::new("view-examples.bonsaidb"))?;

// Insert a new document into the Shape collection.
Shape { sides: 3 }.push_into(&db)?;

And query data using the Map-Reduce-powered view:

let triangles = ShapesByNumberOfSides::entries(&db).with_key(&3).query()?;
println!("Number of triangles: {}", triangles.len());

You can review the full example in the repository, or see all available examples in the examples README.

User's Guide

Our user's guide is early in development, but is available at: https://dev.bonsaidb.io/main/guide/

Minimum Supported Rust Version (MSRV)

While this project is alpha, we are actively adopting the current version of Rust. The current minimum version is 1.70.

Feature Flags

No feature flags are enabled by default in the bonsaidb crate. This is because in most Rust executables, you will only need a subset of the functionality. If you'd prefer to enable everything, you can use the full feature:

[dependencies]
bonsaidb = { version = "*", features = "full" }
  • full: Enables the features below and local-full, server-full, and client-full.
  • cli: Enables the bonsaidb executable.
  • files: Enables file storage support with bonsaidb-files
  • password-hashing: Enables the ability to use password authentication using Argon2 via AnyConnection.
  • token-authentication: Enables the ability to authenticate using authentication tokens, which are similar to API keys.

All other feature flags, listed below, affect each crate individually, but can be safely combined.

Local databases only

[dependencies]
bonsaidb = { version = "*", features = "local-full" }

All Cargo features that affect local databases:

  • local-full: Enables all the flags below
  • local: Enables the local module, which re-exports the crate bonsaidb-local.
  • async: Enables async support with Tokio.
  • cli: Enables the clap structures for embedding database management commands into your own command-line interface.
  • compression: Enables support for compressed storage using lz4.
  • encryption: Enables at-rest encryption.
  • instrument: Enables instrumenting with tracing.
  • password-hashing: Enables the ability to use password authentication using Argon2.
  • token-authentication: Enables the ability to authenticate using authentication tokens, which are similar to API keys.

BonsaiDb server

[dependencies]
bonsaidb = { version = "*", features = "server-full" }

All Cargo features that affect networked servers:

  • server-full: Enables all the flags below,
  • server: Enables the server module, which re-exports the crate bonsaidb-server.
  • acme: Enables automtic certificate acquisition through ACME/LetsEncrypt.
  • cli: Enables the cli module.
  • compression: Enables support for compressed storage using lz4.
  • encryption: Enables at-rest encryption.
  • hyper: Enables convenience functions for upgrading websockets using hyper.
  • instrument: Enables instrumenting with tracing.
  • pem: Enables the ability to install a certificate using the PEM format.
  • websockets: Enables WebSocket support.
  • password-hashing: Enables the ability to use password authentication using Argon2.
  • token-authentication: Enables the ability to authenticate using authentication tokens, which are similar to API keys.

Client for accessing a BonsaiDb server

[dependencies]
bonsaidb = { version = "*", features = "client-full" }

All Cargo features that affect networked clients:

  • client-full: Enables all flags below.
  • client: Enables the client module, which re-exports the crate bonsaidb-client.
  • trusted-dns: Enables using trust-dns for DNS resolution. If not enabled, all DNS resolution is done with the OS's default name resolver.
  • websockets: Enables WebSocket support for bonsaidb-client.
  • password-hashing: Enables the ability to use password authentication using Argon2.
  • token-authentication: Enables the ability to authenticate using authentication tokens, which are similar to API keys.

Developing BonsaiDb

Writing Unit Tests

Unless there is a good reason not to, every feature in BonsaiDb should have thorough unit tests. Many tests are implemented in bonsaidb_core::test_util via a macro that allows the suite to run using various methods of accessing BonsaiDb.

Some features aren't able to be tested using the Connection, StorageConnection, KeyValue, and PubSub traits only. If that's the case, you should add tests to whichever crates makes the most sense to test the code. For example, if it's a feature that only can be used in bonsaidb-server, the test should be somewhere in the bonsaidb-server crate.

Tests that require both a client and server can be added to the core-suite test file in the bonsaidb crate.

Checking Syntax

We use clippy to give additional guidance on our code. Clippy should always return with no errors, regardless of feature flags being enabled:

cargo clippy --all-features

Running all tests

Our CI processes require that some commands succeed without warnings or errors. These checks can be performed manually by running:

cargo xtask test --fail-on-warnings

Or, if you would like to run all these checks before each commit, you can install the check as a pre-commit hook:

cargo xtask install-pre-commit-hook

Formatting Code

We have a custom rustfmt configuration that enables several options only available in nightly builds:

cargo +nightly fmt

Open-source Licenses

This project, like all projects from Khonsu Labs, is open-source. This repository is available under the MIT License or the Apache License 2.0.

To learn more about contributing, please see CONTRIBUTING.md.

More Repositories

1

cushy

An experimental cross-platform graphical user interface (GUI) crate for Rust.
Rust
462
star
2

nebari

A pure Rust database implementation using an append-only B-Tree file format.
Rust
266
star
3

okaywal

A Write Ahead Log (WAL) implementation in Rust
Rust
214
star
4

justjson

An efficient JSON Value parser
Rust
75
star
5

sediment

A low-level MVCC file format for storing blobs.
Rust
63
star
6

kludgine

A kludgey 2d game engine written in Rust atop wgpu
Rust
62
star
7

pot

A concise, self-describing binary format written in Rust for Serde
Rust
53
star
8

budlang

A safe, fast, lightweight embeddable scripting language written in Rust.
Rust
20
star
9

cosmicverge

A systematic, sandbox MMO still in the concept phase. Will be built with Rust atop BonsaiDb and Gooey
HTML
18
star
10

lrumap

A set of safe Least Recently Used (LRU) map/cache types for Rust
Rust
16
star
11

easygpu

A small wrapper around wgpu to make some things simpler
Rust
16
star
12

fabruic

An easy-to-use QUIC-based protocol that supports reliable payload delivery.
Rust
15
star
13

basws

A basis for building an async client-server Websocket protocol
Rust
15
star
14

circulate

Lightweight async PubSub framework
Rust
12
star
15

rsn

Rust
11
star
16

refuse

An easy-to-use, incremental, multi-threaded garbage collector for Rust
Rust
9
star
17

pulldown-cmark-frontmatter

A Frontmatter extractor for Markdown documents built atop pulldown-cmark
Rust
8
star
18

ordered-varint

Variable-length signed and unsigned integer encoding that is byte-orderable for Rust
Rust
8
star
19

transmog

Utilities for serializing with multiple formats of data in Rust.
Rust
8
star
20

figures

A math library specialized for 2d screen graphics.
Rust
8
star
21

delve-rs

A crate search engine powered by Rust and BonsaiDb
Rust
7
star
22

watchable

An observable RwLock-like type for Rust that is compatible with both multi-threaded and async code
Rust
7
star
23

ncog

Rust
7
star
24

alot

A forbid-unsafe, generational slot map with usize-sized IDs
Rust
5
star
25

interner

Rust
5
star
26

nominals

Formatting of nominal identifiers in various languages (ordered list headings)
Rust
5
star
27

muse

Rust
5
star
28

minority-game

A mini-game demo of BonsaiDb + Gooey
Rust
4
star
29

englishid

English formatting for unsigned integer IDs
Rust
4
star
30

rustme

A tool for managing a Rust project's README-like files.
Rust
4
star
31

cityhasher

Rust
4
star
32

custodian

End-user secret management with Rust. Early WIP.
Rust
3
star
33

kempt

A #[forbid_unsafe] ordered collection crate for Rust
Rust
3
star
34

dossier

An artifact server built with BonsaiDb
Rust
3
star
35

gooey-attempt-2

The second attempt at the `gooey` crate for Rust. Never published to crates.io.
Rust
3
star
36

magrathea

A pixel-art-style planet generator
Rust
2
star
37

muse-archive

Virtual instrument synthesizer library for rust built atop cpal
Rust
2
star
38

actionable

An enum-based async framework for building permission-driven APIs
Rust
2
star
39

async-handle

A reference-counted async RwLock for Rust
Rust
2
star
40

easing-function

Rust
2
star
41

pixaxe

A scriptable, indexed pixel art editor written in Rust
Rust
2
star
42

appit

An opinionated wrapper for `winit` utilizing a single-thread-per-window model
Rust
2
star
43

watchfile

An easy-to-use automatic file reloader.
Rust
1
star
44

async-locking-benchmarks

Rust
1
star
45

khonsubase

Khonsubase is a lightweight project management and knowledgebase collaboration tool written in Rust.
HTML
1
star
46

budget-executor

An approach to "throttling" async tasks in Rust using manual instrumentation
Rust
1
star
47

pot-diff

Rust
1
star
48

yew-bulma

Yew wrappers for the Bulma CSS framework
Rust
1
star
49

ncog-archived

A consumer-centric, privacy and safety-forward authentication service written with game developers in mind
CSS
1
star
50

console-thingy

Rust
1
star
51

approximint

A large integer Rust crate using an approximate representation inspired by scientific notation
Rust
1
star
52

FunnyBones

A simple 2D kinematics library for Rust
Rust
1
star
53

cushy-clicker

A proof-of-concept library for creating clicker/incremental games in Rust using Cushy
Rust
1
star
54

gooey-canvas

A Canvas widget for the `Gooey` UI framework
Rust
1
star
55

file-manager

Rust
1
star
56

gooey-attempt-1

The first attempt at the `gooey` crate for Rust. Never published to crates.io.
Rust
1
star