java-dirty

A fast file-based append-only object store, using memory mapped files.

Is java-dirty safe to use with multiple concurrent writers?

Absolutely not - but it's fast enough that putting it behind e.g. a Disruptor and consuming writes in a single thread should be fine.

Downloading from Maven

<dependency>
  <groupId>uk.co.probablyfine</groupId>
  <artifactId>java-dirty</artifactId>
  <version>1.6</version>
</dependency>

Usage

Creating a store.

Store<Foo> store = Store.of(Foo.class).from("/path/to/file");

Inserting an object

store.put(new Foo(1,2));

Iterating over all objects in the store

store.all().forEach(System.out::println);

Iterate over objects, most recent first

store.reverse().forEach(System.out::println);

Iterate over objects from a starting index

store.from(100).forEach(System.out::println);

Access an index directly

Optional<Foo> foo = store.get(1234);

Reset the entire store

store.reset(); // Reset position to 0, overwriting old entries

Close the store and its backing file

store.close();

Trying to read from/write to a closed store will throw a ClosedStoreException.

Observe each write

store.observeWrites((object, index) ->
  System.out.println("Stored "+object+" at "+index);
);

java-dirty does not support replacements, or deletions. Both .all() and .reverse() expose a Stream.

Examples

Look up most recent version of an object by index

Optional<StoredObject> first = store
    .reverse()
    .filter(x -> x.indexField == valueToFind)
    .findFirst();

Build an lookup index using write observers

Store<StoredObject> store = Store.of(StoredObject.class).from("/some/path");

Map<Integer, Integer> index = new HashMap<>();

store.observeWrites((object, location) -> {
  index.put(object.indexField, location);
});

store.put(new StoredObject(1234,5));

store.get(index.get(1234)); // Optional[StoredObject(1234,5)];

Supported Fields

java-dirty will only persist primitive fields on objects. All primitive types are currently supported.

Performance

See the README in java-dirty-benchmarks for the latest