• Stars
    star
    129
  • Rank 279,262 (Top 6 %)
  • Language
    Rust
  • License
    Apache License 2.0
  • Created over 5 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Distributed immutable data store with strong encryption and authentication

ATE

Navigation

What is ATE?

...is it a NoSQL database?
...is it a distributed redo log?
...is it a event BUS?
...is it a API framework?
...is it a distributed queue?
...is it a distributed cache?
...is it a secure encrypted vault?
...is it a quantum resistant communication framework?
...is it a WORM archive solution?

ATE is all these things and none of them; it is unique way to work with distributed data that can easily implement all of the above use cases - take a look at the examples for how you can achieve them.

Why the name?

The origin of the word "mutate" is the latin word '-ate':
https://www.dictionary.com/browse/mutate

Summary

ATE is a distributed immutable data store and built in memory based materialized view with strong encryption and authentication.

What does that mean?

This library is a way of working with data in modern distributed computing.

  • ...data is persisted to a distributed commit log.
  • ...partitions are divided into chains that shard data into physical domains.
  • ...streaming of data to the application occurs on demand as the app needs it.
  • ...each chain is a crypto-graph with unique asymmetric keys at differentiating nodes.
  • ...the root of the chain-of-trust validates the crypto-graph through various plugins.
  • ...strong authentication and authorized is by design built into the data model.
  • ...encryption is highly resistant to quantum attacks and uses fine-grained tenant keys.
  • ...all this is integrated into a shared-nothing highly portable executable.

Examples

Projects

Typical Deployment Pattern

     .-------------.          .- - - - - - -.
     |   Server    |              Server
     |             | .. .. .. |             | .. .. ..
     | >atedb solo |
     '------|----\-'          '- - - - - - -'
            |     \                 
        ws://yourserver.com/db
            |       \
     .------|------. \
     |Native Client|  .-----Browser-----.
     |             |  |.---------------.|
     | >program    |  || >wasm32-wasi  ||
     |  \ate.so    |  ||  \ate.wasm    ||
     '-------------'  |'---------------'|
                      '-----------------'

The easiest way to get up and running is to just build your app and point the
database URL at ws://wasmer.sh/db. You will need to register an account and verify
your identity however after this you can use the free databases and/or paid option.

Alternatively, if you wish to host your own ATE servers in infrastructure that you
manage and run then follow these high-level steps.

1. Server runs the 'atedb' process on some network reachable location
2. Create several records for each IP address under the same A-record in your DNS
3. Either create your own authentication server as well using the auth-server binary
   or just use the authentication servers hosted at Wasmer by pointing to
   ws://wasmer.sh/auth.

Quick Start

Cargo.toml

[dependencies]
tokio = { version = "*", features = ["full", "signal", "process"] }
serde = { version = "*", features = ["derive"] }
ate = { version = "*" }
wasmer-auth = { version = "*" }

main.rs

use serde::{Serialize, Deserialize};
use wasmer_auth::prelude::*;

#[derive(Debug, Serialize, Deserialize, Clone)]
struct MyData
{
    pi: String,
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>>
{
    let dio = DioBuilder::default()
        .with_session_prompt().await?
        .build("mychain")
        .await?;

    dio.store(MyData {
        pi: "3.14159265359".to_string(),
    })?;
    dio.commit().await?;

    Ok(())
}

Changelog

1.2.2  -= Code Refactoring =-
        + The crypto library has been split out from the main ATE library to reduce dependencies
          when using just cryptographic routines and to reduce build times.
1.2.1  -= Lazy Loading =-
        + Subscribing to chains can now load the data in the chain on demand as its needed
          which reduces the startup time considerably.
        + Temporal clients will default to lazy loading
        + Data loaded via the lazy loading mechanism will now be cached client side

1.1.1  -= Performance and Bug Fixes =-
        + Fixed an issue with the web sockets that caused sporadic disconnects
        + Improved the performance of web socket messages by reusing IV's
        + Reduced the message overhead with a new message encoding format

1.1.0  -= Comms Upgrade =-
        + Streaming websockets are now more stable as they use length headers to delimit messages.
        + Fixed a bug where disconnecting clients would drop neighbors on the same server.
        + Various changes to the interfaces for stability reasons
        (this upgrade is not backwards compatible with version 1.0.6)

1.0.6  -= Bug Fixes =-
        + Modified the interface slightly but most users should not be impacted
        + Fixed a bug around validators rejecting events during the subscribe
          process that re-reads them from disk - these validators should not be running
        + Added the ability to list all root objects
        + Added the ability to delete all root objects (and hence wipe a chain)
        + Fixed a serious deadlock situation when commiting transactions that was causing timeouts

1.0.2  -= WASM BUS =-
       + Integrated with the WASM bus (wasmer-bus) which allows for ATE to use
         the web sockets while running in a controlled sandbox.

1.0.0  -= Major Release =-
       + See [README.md](https://github.com/wasmerio/ate/blob/e0beedbbbd84f95cd6c7a9a45b8903058f65b6fd/README.md)

<=0.8.0 See commit history

High Level Design

.--[ atedb  ]---. .--[ atedb  ]---.      .-[auth-server]-.
|               | |               |      |               |
|>local redo-log| |>local redo-log|      |>local redo-log|
|.-------------.| |.-------------.|      |.-------------.|
|| Chain     1 || ||             ||      ||    user     ||
||             || || Chain     2 ||      ||   account   ||
|*-------------*| |*------|------*|      |*-----|-------*|
|               |       subscribe             login      
|                \________|_____________________|____
|                         |                     |    
|  >local redo-log                                   
|  >Crypto-Graph Materiaized View< (in memory)       
|  .----------------------------------.      session 
|  |             root(hash)           |   .-----------.
|  |              |                   |   |  -token   |
|  |      dao----dao(aes)             |---|  -claims  |
|  |              \                   |   |  -keys    |
|  |               dao                |   *-----------*
|  |                                  |

Feature Flags

  • 'client' - Client functionality that allows one to connect to ATE datachains and/or host them locally
  • 'server' - Server functionality required to create and run ATE in distributed mode with the data replicated on server nodes.
  • 'client_web' - Client functionality designed for running within a browser sandbox (--target=wasm32-wasi)

WebAssembly

When compiling for WASM use the following command:

cargo build --target wasm32-wasi --no-default-features --features client_web

Lower Level Example

Cargo.toml

[dependencies]
tokio = { version = "*", features = ["full", "signal", "process"] }
serde = { version = "*", features = ["derive"] }
ate = { version = "*" }

main.rs

use serde::{Serialize, Deserialize};
use ate::prelude::*;

#[derive(Clone, Serialize, Deserialize)]
struct World
{
    commandment: String
}

#[tokio::main]
async fn main() -> Result<(), AteError>
{
    // The default configuration will store the redo log locally in the temporary folder
    let conf = AteConfig::default();
    let builder = ChainBuilder::new(&conf).await.build();

    // We create a chain with a specific key (this is used for the file name it creates)
    let chain = builder.open(&ChainKey::from("universe")).await?;
    
    // We interact with the data stored in the chain-of-trust using a DIO
    let session = AteSession::default();
    let mut dio = chain.dio(&session).await;
    
    // In this example we store some data in the "World" object
    let key = dio.store(World {
        commandment: "Hello".to_string(),
    })?.key().clone();
    dio.commit().await?;
    
    // Now we retreive the data and print it to console
    println!("{} world!", dio.load::<World>(&key).await?.commandment);

    // All errors in ATE will convert into the AteError
    Ok(())
}

Contribution

If you would like to help setup a community to continue to develop this project then please contact me at [email protected]

More Repositories

1

wasmer

πŸš€ The leading Wasm Runtime supporting WASIX, WASI and Emscripten
Rust
16,915
star
2

wasmer-go

πŸΉπŸ•ΈοΈ WebAssembly runtime for Go
Go
2,679
star
3

wasmer-python

πŸπŸ•Έ WebAssembly runtime for Python
Rust
1,999
star
4

wasmer-php

πŸ˜πŸ•ΈοΈ WebAssembly runtime for PHP
PHP
964
star
5

winterjs

Winter is coming... ❄️
JavaScript
844
star
6

wasmer-js

Monorepo for Javascript WebAssembly packages by Wasmer
Rust
828
star
7

kernel-wasm

Sandboxed kernel mode WebAssembly runtime.
C
702
star
8

wasmer-java

β˜• WebAssembly runtime for Java
Java
544
star
9

awesome-wasi

😎 Curated list of awesome things regarding WebAssembly WASI ecosystem.
476
star
10

wasmer-ruby

πŸ’ŽπŸ•Έ WebAssembly runtime for Ruby
Rust
462
star
11

wasmer-postgres

πŸ’½πŸ•Έ Postgres library to run WebAssembly binaries.
Rust
392
star
12

wapm-cli

πŸ“¦ WebAssembly Package Manager (CLI)
Rust
368
star
13

webassembly.sh

Open-source and installable PWA terminal powered by WebAssembly, WAPM, and Wasmer-JS πŸ–₯
JavaScript
269
star
14

wasmer-rust-example

Example of WebAssembly embedding in Rust using Wasmer
Rust
163
star
15

vscode-wasm

WebAssembly extension for VSCode
Rust
122
star
16

wai

A language binding generator for `wai` (a precursor to WebAssembly interface types)
Rust
110
star
17

rusty_jsc

Rust bindings for the JavaScriptCore engine.
Rust
86
star
18

wasmer-c-api

Example of the C API to embed the Wasmer runtime
C
76
star
19

io-devices-lib

Library for interacting with the Wasmer Experimental IO Devices
WebAssembly
51
star
20

wasmer-pack

Rust
49
star
21

sonde-rs

A library to compile USDT probes into a Rust library
Rust
44
star
22

old-docs.wasmer.io

Wasmer Documentation (for standalone and embedded use cases)
WebAssembly
41
star
23

wasmer-install

Wasmer Binary Installer https://wasmer.io/
Shell
39
star
24

loupe

Profiling tool for Rust code.
Rust
35
star
25

c-wasm-simd128-example

Example C++ repo emitting Wasm SIMD 128 instructions
C++
30
star
26

wasmer-nginx-example

This is a simple example of Nginx running with wasmer
HTML
29
star
27

wasmer-ocaml

OCaml bindings for Wasmer
OCaml
28
star
28

cargo-wasmer

A cargo sub-command for publishing Rust crates to the WebAssembly Package Manager.
Rust
24
star
29

wasm-fuzz

Fuzzer for Wasm and Wasmer
JavaScript
22
star
30

wasmer.io

The Wasmer.io website
JavaScript
21
star
31

wasmer-bench

This is a repo for benchmarking Wasmer (compilation & runtime)
Rust
20
star
32

wcgi-wordpress-demo

PHP
15
star
33

wcgi-php-template

PHP
14
star
34

docs.wasmer.io

The Wasmer Docs Website (website deployed using Wasmer Edge)
MDX
14
star
35

setup-wasmer

GitHub action for setting up Wasmer
TypeScript
14
star
36

c-http-server

A very simple http server in c
JavaScript
13
star
37

rust-wasm-simd128-example

Example Rust repo emitting Wasm SIMD 128 instructions
Rust
10
star
38

wasm-debug

A runtime-independent crate for transforming Wasm-DWARF
Rust
9
star
39

wasmer-terminal-js

The WebAssembly terminal, revamped!
Rust
7
star
40

rust-cli-app-example

Example CLI app written in rust for Wasmer
Rust
7
star
41

wapm-publish

GitHub action for publishing to WAPM
TypeScript
7
star
42

wasmer-rust-customabi-example

An example repo to demonstrate how to create a module with a custom ABI to then use it from an embedder
Rust
6
star
43

wasm-interface-cli

A binary for verifying Wasm interfaces
Rust
6
star
44

interface-types

The `wasmer-interface-types` crate
Rust
5
star
45

ewasm-workshop

Ewasm workshow with Wasmer Metering
Rust
5
star
46

sgp4

Rust
5
star
47

llvm-custom-builds

Sandbox to produce custom LLVM builds for various platforms
Shell
5
star
48

sha2

A WebAssembly interface to Rust's sha2 crate
Rust
5
star
49

wasm-python-api

WebAssembly Python standard API prototype
Python
4
star
50

python-flask-example

Sample Python Flask server for Wasmer Edge
Python
4
star
51

shared-buffer

An abstraction over buffers backed by memory-mapped files or bytes in memory
Rust
3
star
52

wcgi-rust-template

Rust
3
star
53

wasmer-toml

Split out the wapm.toml parsing from wasmerio/wapm-cli
Rust
3
star
54

wasmer-nightly

Nightly releases of Wasmer
3
star
55

llvm-build

LLVM Distribution for being used with Wasmer
3
star
56

js-service-worker-example

Sample for a js service worker running on Wasmer Edge.
JavaScript
3
star
57

wasi-tests

Integration tests for WASI
Rust
2
star
58

.github

Github profile
2
star
59

wasmer-js-example

Wasmer-JS example
2
star
60

edge-react-starter

Wasmer Edge + Vite + React + TS starter template. Deploy it like it's hot πŸš€
CSS
1
star
61

rustfft

Rustfft for WASM, published on WAPM
Rust
1
star
62

inline-c

Rust
1
star
63

speed.wasmer.io

Speed tracker of Wasmer
CSS
1
star
64

windows-llvm-build

Build of LLVM in Windows
1
star
65

wasmer-edge-support

1
star
66

zig2wapm

CI script to clone zig/zig to wapm.io
Shell
1
star
67

wasm4-on-wapm

This is a demo repo to showcase how to upload your WASM-4 games to WAPM
1
star
68

wai-hashids

Hashids demonstration with WAI
Rust
1
star
69

astro-starter-a1d

Astro
1
star
70

flask-starter-324

Python
1
star
71

wasmerio-flask-template

Python
1
star
72

wasmer-integration-tests

Integration tests between Edge and the backend
Rust
1
star