• Stars
    star
    4,079
  • Rank 10,649 (Top 0.3 %)
  • Language
    JavaScript
  • License
    MIT License
  • Created about 12 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A wrapper for abstract-leveldown compliant stores, for Node.js and browsers.

levelup

📌 This module will soon be deprecated, because it is superseded by abstract-level.

level badge npm Node version Test Coverage Standard Common Changelog Donate

Table of Contents

Click to expand

Introduction

Fast and simple storage. A Node.js wrapper for abstract-leveldown compliant stores, which follow the characteristics of LevelDB.

LevelDB is a simple key-value store built by Google. It's used in Google Chrome and many other products. LevelDB supports arbitrary byte arrays as both keys and values, singular get, put and delete operations, batched put and delete, bi-directional iterators and simple compression using the very fast Snappy algorithm.

LevelDB stores entries sorted lexicographically by keys. This makes the streaming interface of levelup - which exposes LevelDB iterators as Readable Streams - a very powerful query mechanism.

The most common store is leveldown which provides a pure C++ binding to LevelDB. Many alternative stores are available such as level.js in the browser or memdown for an in-memory store. They typically support strings and Buffers for both keys and values. For a richer set of data types you can wrap the store with encoding-down.

The level package is the recommended way to get started. It conveniently bundles levelup, leveldown and encoding-down. Its main export is levelup - i.e. you can do var db = require('level').

Supported Platforms

We aim to support Active LTS and Current Node.js releases as well as browsers. For support of the underlying store, please see the respective documentation.

Sauce Test Status

Usage

If you are upgrading: please see UPGRADING.md.

First you need to install levelup! No stores are included so you must also install leveldown (for example).

$ npm install levelup leveldown

All operations are asynchronous. If you do not provide a callback, a Promise is returned.

var levelup = require('levelup')
var leveldown = require('leveldown')

// 1) Create our store
var db = levelup(leveldown('./mydb'))

// 2) Put a key & value
db.put('name', 'levelup', function (err) {
  if (err) return console.log('Ooops!', err) // some kind of I/O error

  // 3) Fetch by key
  db.get('name', function (err, value) {
    if (err) return console.log('Ooops!', err) // likely the key was not found

    // Ta da!
    console.log('name=' + value)
  })
})

API

levelup(db[, options[, callback]])

The main entry point for creating a new levelup instance.

  • db must be an abstract-leveldown compliant store.
  • options is passed on to the underlying store when opened and is specific to the type of store being used

Calling levelup(db) will also open the underlying store. This is an asynchronous operation which will trigger your callback if you provide one. The callback should take the form function (err, db) {} where db is the levelup instance. If you don't provide a callback, any read & write operations are simply queued internally until the store is fully opened, unless it fails to open, in which case an error event will be emitted.

This leads to two alternative ways of managing a levelup instance:

levelup(leveldown(location), options, function (err, db) {
  if (err) throw err

  db.get('foo', function (err, value) {
    if (err) return console.log('foo does not exist')
    console.log('got foo =', value)
  })
})

Versus the equivalent:

// Will throw if an error occurs
var db = levelup(leveldown(location), options)

db.get('foo', function (err, value) {
  if (err) return console.log('foo does not exist')
  console.log('got foo =', value)
})

db.supports

A read-only manifest. Might be used like so:

if (!db.supports.permanence) {
  throw new Error('Persistent storage is required')
}

if (db.supports.bufferKeys && db.supports.promises) {
  await db.put(Buffer.from('key'), 'value')
}

db.open([options][, callback])

Opens the underlying store. In general you shouldn't need to call this method directly as it's automatically called by levelup(). However, it is possible to reopen the store after it has been closed with close().

If no callback is passed, a promise is returned.

db.close([callback])

close() closes the underlying store. The callback will receive any error encountered during closing as the first argument.

You should always clean up your levelup instance by calling close() when you no longer need it to free up resources. A store cannot be opened by multiple instances of levelup simultaneously.

If no callback is passed, a promise is returned.

db.put(key, value[, options][, callback])

put() is the primary method for inserting data into the store. Both key and value can be of any type as far as levelup is concerned.

options is passed on to the underlying store.

If no callback is passed, a promise is returned.

db.get(key[, options][, callback])

Get a value from the store by key. The key can be of any type. If it doesn't exist in the store then the callback or promise will receive an error. A not-found err object will be of type 'NotFoundError' so you can err.type == 'NotFoundError' or you can perform a truthy test on the property err.notFound.

db.get('foo', function (err, value) {
  if (err) {
    if (err.notFound) {
      // handle a 'NotFoundError' here
      return
    }
    // I/O or other error, pass it up the callback chain
    return callback(err)
  }

  // .. handle `value` here
})

The optional options object is passed on to the underlying store.

If no callback is passed, a promise is returned.

db.getMany(keys[, options][, callback])

Get multiple values from the store by an array of keys. The optional options object is passed on to the underlying store.

The callback function will be called with an Error if the operation failed for any reason. If successful the first argument will be null and the second argument will be an array of values with the same order as keys. If a key was not found, the relevant value will be undefined.

If no callback is provided, a promise is returned.

db.del(key[, options][, callback])

del() is the primary method for removing data from the store.

db.del('foo', function (err) {
  if (err)
    // handle I/O or other error
});

options is passed on to the underlying store.

If no callback is passed, a promise is returned.

db.batch(array[, options][, callback]) (array form)

batch() can be used for very fast bulk-write operations (both put and delete). The array argument should contain a list of operations to be executed sequentially, although as a whole they are performed as an atomic operation inside the underlying store.

Each operation is contained in an object having the following properties: type, key, value, where the type is either 'put' or 'del'. In the case of 'del' the value property is ignored. Any entries with a key of null or undefined will cause an error to be returned on the callback and any type: 'put' entry with a value of null or undefined will return an error.

const ops = [
  { type: 'del', key: 'father' },
  { type: 'put', key: 'name', value: 'Yuri Irsenovich Kim' },
  { type: 'put', key: 'dob', value: '16 February 1941' },
  { type: 'put', key: 'spouse', value: 'Kim Young-sook' },
  { type: 'put', key: 'occupation', value: 'Clown' }
]

db.batch(ops, function (err) {
  if (err) return console.log('Ooops!', err)
  console.log('Great success dear leader!')
})

options is passed on to the underlying store.

If no callback is passed, a promise is returned.

db.batch() (chained form)

batch(), when called with no arguments will return a Batch object which can be used to build, and eventually commit, an atomic batch operation. Depending on how it's used, it is possible to obtain greater performance when using the chained form of batch() over the array form.

db.batch()
  .del('father')
  .put('name', 'Yuri Irsenovich Kim')
  .put('dob', '16 February 1941')
  .put('spouse', 'Kim Young-sook')
  .put('occupation', 'Clown')
  .write(function () { console.log('Done!') })

batch.put(key, value[, options])

Queue a put operation on the current batch, not committed until a write() is called on the batch. The options argument, if provided, must be an object and is passed on to the underlying store.

This method may throw a WriteError if there is a problem with your put (such as the value being null or undefined).

batch.del(key[, options])

Queue a del operation on the current batch, not committed until a write() is called on the batch. The options argument, if provided, must be an object and is passed on to the underlying store.

This method may throw a WriteError if there is a problem with your delete.

batch.clear()

Clear all queued operations on the current batch, any previous operations will be discarded.

batch.length

The number of queued operations on the current batch.

batch.write([options][, callback])

Commit the queued operations for this batch. All operations not cleared will be written to the underlying store atomically, that is, they will either all succeed or fail with no partial commits.

The optional options object is passed to the .write() operation of the underlying batch object.

If no callback is passed, a promise is returned.

db.status

A readonly string that is one of:

  • new - newly created, not opened or closed
  • opening - waiting for the underlying store to be opened
  • open - successfully opened the store, available for use
  • closing - waiting for the store to be closed
  • closed - store has been successfully closed.

db.isOperational()

Returns true if the store accepts operations, which in the case of levelup means that status is either opening or open, because it opens itself and queues up operations until opened.

db.createReadStream([options])

Returns a Readable Stream of key-value pairs. A pair is an object with key and value properties. By default it will stream all entries in the underlying store from start to end. Use the options described below to control the range, direction and results.

db.createReadStream()
  .on('data', function (data) {
    console.log(data.key, '=', data.value)
  })
  .on('error', function (err) {
    console.log('Oh my!', err)
  })
  .on('close', function () {
    console.log('Stream closed')
  })
  .on('end', function () {
    console.log('Stream ended')
  })

You can supply an options object as the first parameter to createReadStream() with the following properties:

  • gt (greater than), gte (greater than or equal) define the lower bound of the range to be streamed. Only entries where the key is greater than (or equal to) this option will be included in the range. When reverse=true the order will be reversed, but the entries streamed will be the same.

  • lt (less than), lte (less than or equal) define the higher bound of the range to be streamed. Only entries where the key is less than (or equal to) this option will be included in the range. When reverse=true the order will be reversed, but the entries streamed will be the same.

  • reverse (boolean, default: false): stream entries in reverse order. Beware that due to the way that stores like LevelDB work, a reverse seek can be slower than a forward seek.

  • limit (number, default: -1): limit the number of entries collected by this stream. This number represents a maximum number of entries and may not be reached if you get to the end of the range first. A value of -1 means there is no limit. When reverse=true the entries with the highest keys will be returned instead of the lowest keys.

  • keys (boolean, default: true): whether the results should contain keys. If set to true and values set to false then results will simply be keys, rather than objects with a key property. Used internally by the createKeyStream() method.

  • values (boolean, default: true): whether the results should contain values. If set to true and keys set to false then results will simply be values, rather than objects with a value property. Used internally by the createValueStream() method.

db.createKeyStream([options])

Returns a Readable Stream of keys rather than key-value pairs. Use the same options as described for createReadStream() to control the range and direction.

You can also obtain this stream by passing an options object to createReadStream() with keys set to true and values set to false. The result is equivalent; both streams operate in object mode.

db.createKeyStream()
  .on('data', function (data) {
    console.log('key=', data)
  })

// same as:
db.createReadStream({ keys: true, values: false })
  .on('data', function (data) {
    console.log('key=', data)
  })

db.createValueStream([options])

Returns a Readable Stream of values rather than key-value pairs. Use the same options as described for createReadStream() to control the range and direction.

You can also obtain this stream by passing an options object to createReadStream() with values set to true and keys set to false. The result is equivalent; both streams operate in object mode.

db.createValueStream()
  .on('data', function (data) {
    console.log('value=', data)
  })

// same as:
db.createReadStream({ keys: false, values: true })
  .on('data', function (data) {
    console.log('value=', data)
  })

db.iterator([options])

Returns an abstract-leveldown iterator, which is what powers the readable streams above. Options are the same as the range options of createReadStream() and are passed to the underlying store.

These iterators support for await...of:

for await (const [key, value] of db.iterator()) {
  console.log(value)
}

db.clear([options][, callback])

Delete all entries or a range. Not guaranteed to be atomic. Accepts the following range options (with the same rules as on iterators):

  • gt (greater than), gte (greater than or equal) define the lower bound of the range to be deleted. Only entries where the key is greater than (or equal to) this option will be included in the range. When reverse=true the order will be reversed, but the entries deleted will be the same.
  • lt (less than), lte (less than or equal) define the higher bound of the range to be deleted. Only entries where the key is less than (or equal to) this option will be included in the range. When reverse=true the order will be reversed, but the entries deleted will be the same.
  • reverse (boolean, default: false): delete entries in reverse order. Only effective in combination with limit, to remove the last N records.
  • limit (number, default: -1): limit the number of entries to be deleted. This number represents a maximum number of entries and may not be reached if you get to the end of the range first. A value of -1 means there is no limit. When reverse=true the entries with the highest keys will be deleted instead of the lowest keys.

If no options are provided, all entries will be deleted. The callback function will be called with no arguments if the operation was successful or with an WriteError if it failed for any reason.

If no callback is passed, a promise is returned.

What happened to db.createWriteStream?

db.createWriteStream() has been removed in order to provide a smaller and more maintainable core. It primarily existed to create symmetry with db.createReadStream() but through much discussion, removing it was the best course of action.

The main driver for this was performance. While db.createReadStream() performs well under most use cases, db.createWriteStream() was highly dependent on the application keys and values. Thus we can't provide a standard implementation and encourage more write-stream implementations to be created to solve the broad spectrum of use cases.

Check out the implementations that the community has produced here.

Promise Support

Each function accepting a callback returns a promise if the callback is omitted. The only exception is the levelup constructor itself, which if no callback is passed will lazily open the underlying store in the background.

Example:

const db = levelup(leveldown('./my-db'))
await db.put('foo', 'bar')
console.log(await db.get('foo'))

Events

levelup is an EventEmitter and emits the following events.

Event Description Arguments
put Key has been updated key, value (any)
del Key has been deleted key (any)
batch Batch has executed operations (array)
clear Entries were deleted options (object)
opening Underlying store is opening -
open Store has opened -
ready Alias of open -
closing Store is closing -
closed Store has closed. -
error An error occurred error (Error)

For example you can do:

db.on('put', function (key, value) {
  console.log('inserted', { key, value })
})

Multi-process Access

Stores like LevelDB are thread-safe but they are not suitable for accessing with multiple processes. You should only ever have a store open from a single Node.js process. Node.js clusters are made up of multiple processes so a levelup instance cannot be shared between them either.

See Level/awesome for modules like multileveldown that may help if you require a single store to be shared across processes.

Contributing

Level/levelup is an OPEN Open Source Project. This means that:

Individuals making significant and valuable contributions are given commit-access to the project to contribute as they see fit. This project is more like an open wiki than a standard guarded open source project.

See the Contribution Guide for more details.

Big Thanks

Cross-browser Testing Platform and Open Source ♥ Provided by Sauce Labs.

Sauce Labs logo

Donate

Support us with a monthly donation on Open Collective and help us continue our work.

License

MIT

More Repositories

1

level

Universal abstract-level database for Node.js and browsers.
JavaScript
1,544
star
2

leveldown

Pure C++ Node.js LevelDB binding. An abstract-leveldown compliant store.
C++
773
star
3

level-js

An abstract-leveldown compliant store on top of IndexedDB.
JavaScript
542
star
4

awesome

An open list of awesome Level modules and resources.
JavaScript
287
star
5

memdown

In-memory abstract-leveldown store for Node.js and browsers.
JavaScript
285
star
6

rocksdb

Pure C++ Node.js RocksDB binding. An abstract-leveldown compliant store.
C++
229
star
7

abstract-leveldown

An abstract prototype matching the leveldown API.
JavaScript
146
star
8

party

Open a leveldb handle multiple times.
JavaScript
146
star
9

level-rocksdb

A convenience package bundling levelup and rocksdb.
JavaScript
146
star
10

abstract-level

Abstract class for a lexicographically sorted key-value database.
JavaScript
120
star
11

subleveldown

Split a levelup database into sublevels with their own keyspace, encoding and events.
JavaScript
119
star
12

electron-demo

Demo app loading LevelDB into an Electron context.
JavaScript
79
star
13

level-ttl

Add a 'ttl' (time-to-live) option to levelup for put() and batch().
JavaScript
69
star
14

level-browserify

No longer maintained: superseded by level v5.0.0.
JavaScript
63
star
15

classic-level

An abstract-level database backed by LevelDB.
JavaScript
58
star
16

multileveldown

multilevel implemented using leveldowns with reconnect support.
JavaScript
58
star
17

browser-level

An abstract-level database for browsers, backed by IndexedDB.
JavaScript
45
star
18

leveljs.org

Source code of leveljs.org
Pug
43
star
19

community

Discussion, support and common information for projects in the community.
42
star
20

encoding-down

An abstract-leveldown implementation that wraps another store to encode keys and values.
JavaScript
41
star
21

level-hyper

A convenience package bundling levelup and leveldown-hyper.
JavaScript
30
star
22

leveldown-mobile

This project has been abandoned.
C++
27
star
23

codec

Encode keys, values and range options, with built-in or custom encodings.
JavaScript
27
star
24

memory-level

In-memory abstract-level database for Node.js and browsers.
JavaScript
25
star
25

mem

A convenience package bundling levelup and memdown.
JavaScript
24
star
26

level-ws

A basic writable stream for abstract-level databases.
JavaScript
22
star
27

packager

A levelup package helper for distributing with an abstract-leveldown compatible back-end.
JavaScript
20
star
28

level-test

Inject temporary and isolated level stores (leveldown, level-js, memdown or custom) into your tests.
JavaScript
20
star
29

deferred-leveldown

An abstract-leveldown implementation that queues operations while a real abstract-leveldown instance is being opened.
JavaScript
19
star
30

many-level

Share an abstract-level database over the network or other kind of stream.
JavaScript
17
star
31

leveldown-hyper

Pure C++ Node.js HyperLevelDB binding. An abstract-leveldown compliant store.
C++
15
star
32

errors

Error types for levelup.
JavaScript
13
star
33

web-stream

Read from an abstract-level database using Web Streams.
JavaScript
12
star
34

iterator-stream

Turn an abstract-leveldown iterator into a readable stream.
JavaScript
12
star
35

level-lmdb

This project has been abandoned.
JavaScript
11
star
36

bench

Benchmark abstract-level databases.
JavaScript
8
star
37

level-mobile

[Currently not actively maintained] Fast & simple storage - a Node.js-style LevelDB wrapper (a convenience package bundling levelup & leveldown-mobile) [requires JXcore 0.3.0.3+]
JavaScript
8
star
38

read-stream

Read from an abstract-level database using Node.js streams.
JavaScript
6
star
39

rave-level

Use a LevelDB database from multiple processes with seamless failover.
JavaScript
6
star
40

level-basho

This project has been abandoned.
JavaScript
5
star
41

webpack-starter

Demonstrates bundling level for browsers using webpack.
JavaScript
5
star
42

concat-iterator

Concatenate entries from an iterator into an array.
JavaScript
5
star
43

compose

Compose a database factory from abstract-leveldown and levelup layers.
JavaScript
4
star
44

transcoder

Encode data with built-in or custom encodings.
JavaScript
4
star
45

supports

Create a manifest describing the abilities of an abstract-level database.
JavaScript
4
star
46

level-fstream

This project has been abandoned.
JavaScript
3
star
47

lazy-open

This project has been abandoned.
JavaScript
3
star
48

typings

Experimental typing development for level repositories. Deprecated.
TypeScript
3
star
49

database

2
star
50

browserify-starter

Demonstrates bundling level for browsers using browserify.
JavaScript
2
star
51

.github

Default community health files.
1
star