• Stars
    star
    19
  • Rank 1,127,934 (Top 23 %)
  • Language
    Scala
  • License
    Apache License 2.0
  • Created over 4 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

akka-persistence-gcp-datastore is a journal and snapshot store plugin for akka-persistence using google cloud firestore in datastore mode.

GCP Datastore Akka Persistence Plugin

Test Download shields.io Codacy Badge Scala Steward badge

akka-persistence-gcp-datastore is a journal and snapshot store plugin for akka-persistence using Google Cloud Datastore. It uses the official Google Java Dependency to talk with the datastore.

Scala 2.12 & 2.13, Java 8 & Java 11, akka 2.6.X are supported.

The plugin supports the following functionality:

  • serialization of events and snapshots with play-json
  • peristence-query api
  • custom serialization

Related Blogposts

Usage & Setup

Versions: The table below lists the versions and their main dependencies

Version to use Scala 2.12 Scala 2.13 Scala 3 / Dotty Akka play-json google-cloud-datastore
1.0.2 ✓ ✓ ? 2.6.x 2.8.x 1.102.x

Dependency

You just need to add the following dependency to you sbt dependencies

libraryDependencies += "de.innfactory" %% "akka-persistence-gcp-datastore" % "X.Y.Z"

Configuration

Take a look at reference.conf under src/main/resources We forked the cqrs cassandra lightbend example with necessary changes for gcp-datastore (@ Demo Test Project available based on the CQRS Example from Lightbend https://github.com/innFactory/akka-persistence-gcp-datastore-example)

Add the following to your application.conf for a basic configuration:

akka {
  # use google cloud datastore as journal and snapshot store
  persistence {

    journal {
      plugin = "gcp-datastore-journal"
      auto-start-journals = ["gcp-datastore-journal"]
    }

    snapshot-store {
      plugin = "gcp-datastore-snapshot"
      auto-start-snapshot-stores = ["gcp-datastore-snapshot"]
    }
  }
}

Datastore Configuration

  1. Google Cloud Project with Datastore or FireStore in Datastore mode enabled

  2. Create a index.yml file with content bolow in the project that will use this plugin:

    indexes:
      - kind: journal
        properties:
          - name: persistenceId
          - name: sequenceNr
            direction: desc
    
      - kind: snapshot
        properties:
          - name: persistenceId
          - name: timestamp
            direction: desc
    
      - kind: snapshot
        properties:
          - name: persistenceId
          - name: timestamp
    
      - kind: journal
        properties:
        - name: tagsKey
        - name: timestamp
    
      - kind: journal
        properties:
        - name: persistenceId
        - name: sequenceNr
    

    index.yml

  3. Open terminal and execute

    gcloud app deploy index.yaml
    

    This is telling the GCP Datastore to build indexes for the plugin based on the yaml file

  4. Create a service account for read and write to datastore. Download the json and add it to the project

    src/main/resources/datastore.json
    

Persistence query API

The plugin supports the Persistence query APi, mostly used in CQRS applications to transform/migrate the events from the write side to the read side.

The ReadJournal is retrieved via the akka.persistence.datastore.journal.read.DatastoreScaladslReadJournal and akka.persistence.datastore.journal.read.DatastoreJavadslReadJournal. There is also a DatastoreReadJournalProvider.

import akka.persistence.datastore.journal.read.DatastoreScaladslReadJournal
import akka.persistence.query.{ EventEnvelope, PersistenceQuery }

val system = ??? //ActorSystem akka-classic or akka-typed then system.toClassic is needed. see the example.
val readJournal =
    PersistenceQuery(system).readJournalFor[DatastoreScaladslReadJournal]("gcp-datastore-query")

Supported Queries

All queries are live streams and they are not completed when they reaches the end of the currently stored events, but continue to push new events when new events are persisted.

eventsByTag

eventsByTags is used for retrieving events that were marked with a given tag.

eventsByPersistenceId

eventsByPersistenceId is used for retrieving events for a specific PersistentActor identified by its persistenceId

Testing

To test this plugin

(Source: https://cloud.google.com/datastore/docs/tools/datastore-emulator)

  1. gcloud components install cloud-datastore-emulator

  2. gcloud beta emulators datastore start --no-store-on-disk --consistency=1.0

  3. Set Env Variable DATASTORE_TESTHOST=http://<host>:<port> of datastore emulator

  4. Execute sbt run

  5. Before executing test reset datastore data: curl -X POST http://<host>:<port>/reset

There is a shell script under .circle ci which runs all of these tests. cqrs tests are outsourced in the example project.

Contribution policy

Contributions via GitHub pull requests are gladly accepted from their original author. Along with any pull requests, please state that the contribution is your original work and that you license the work to the project under the project's open source license. Whether or not you state this explicitly, by submitting any copyrighted material via pull request, email, or other means you agree to license the material under the project's open source license and warrant that you have the legal authority to do so.

Credits

innFactory GmbH is a lightbend partner from germany. We are experts for Apps, BigData & Cloud Computing. If you need help with your next project, feel free to ask for our support.

More Repositories

1

create-react-app-material-typescript-redux

A ready to use boilerplate for starting big react projects
TypeScript
288
star
2

react-native-dialogflow

A React-Native Bridge for the Google Dialogflow (API.AI) SDK
JavaScript
196
star
3

react-planet

A react lib for building circular menus in a very easy and handy way.
TypeScript
168
star
4

JFXC

Jonato JavaFX Controls - More Power for your JavaFX Gui
Java
43
star
5

react-native-quick-sample

A small and simple example app with navigation, data persistence, listview and animation!
JavaScript
34
star
6

bootstrap-akka-http.g8

bootstrap template for akka-http, slick, swagger, aws service
Scala
20
star
7

bootstrap-akka-graphql.g8

g8 bootstrap template for akka microservices with an graphql endpoint
Scala
18
star
8

akka-microservice-example

Kubernetes Deployment of two separate akka microservices which communicate via gRPC
Scala
17
star
9

react-native-aws-mobile-analytics

A react-native module for using Amazon's AWS Mobile Analytics with the aws-sdk
JavaScript
16
star
10

akka-jwt

Library for jwt authentication with akka
Scala
16
star
11

bootstrap-play2

A scala bootstrap template for the play 2 framework to build real world applications.
Scala
14
star
12

akka-lift-ml

akka http service for serving spark machine learning models
Scala
14
star
13

smithy4play

smithy4s Routing for Play Framework.
Scala
14
star
14

bootstrap-akka-http

bootstrap template for akka-http services
Scala
14
star
15

react-native-aws-mobile-analytics-demo

Demo React-Native app which uses react-native-aws-mobile-analytics
Objective-C
13
star
16

react-typesafe-routes

The last routing library you will ever need in your React projects. (At least if you're using react-router–dom but also why wouldn't you?)
TypeScript
13
star
17

material-ui-recoil-template

A ready to use boilerplate for starting [experimental] react projects
TypeScript
11
star
18

bootstrap-akka-graphql

bootstrap template for akka microservices with an graphql endpoint
Scala
11
star
19

aws-codepipeline-dashboard

A simple dashboard for monitoring your aws codepipelines.
TypeScript
10
star
20

aws-session-token-gui

Get a Amazon Web Service Token via desktop GUI
JavaScript
10
star
21

flutter-factory

CLI tool for creating a new Flutter project and other useful features.
TypeScript
9
star
22

akka-cluster-k8s

Deployment of an akka cluster on AWS/Google Kubernetes [EXAMPLE]
Scala
8
star
23

corsign-core

Scala
8
star
24

react-factory

TypeScript
7
star
25

react-starting-guide

This is an instruction to react and related technics. Please go through it and prepare yourself for programming react! Have fun!
7
star
26

scalafx-yaml-editor

This is a ScalaFX demo that contains an simple YAML editor with syntax highlighting and a search function.
Scala
7
star
27

docker

Usefull docker templates for our software projects
Dockerfile
6
star
28

bootstrap-aws-serverless

TypeScript
6
star
29

flutter-starting-guide

5
star
30

flutter_bloc_demo

A small app for demonstrating the usage of the BloC Pattern.
Dart
5
star
31

flutter-factory-templates

Dart
4
star
32

react-template

A ready to use boilerplate for starting big react projects
TypeScript
4
star
33

react-firebase-auth-example

A demo which shows how to implement the firebase authentication mechanism in react
TypeScript
3
star
34

react-mui-cookie-dialog

A cookie dialog in react based on material-ui.
TypeScript
2
star
35

scala-utils

Reusable scala utils for awesome innFactory projects.
Scala
2
star
36

react-mui-table-container

Enhancement of mui-virtualized-table with search and actions
TypeScript
2
star
37

plantuml-ifstyle

innFactory PlantUML Styling
2
star
38

flutter_lint_factory

Dart
2
star
39

flutter-todo-starter

Flutter Template for starting new projects. It includes a Flutter app with offline sync and a Scala backend.
Dart
1
star
40

akka-persistence-gcp-datastore-example

Datastore Tests for akka-persistence-gcp-datastore
Scala
1
star
41

smithy4play-example

Smithy Example based on Play2 with smithy4play
Scala
1
star
42

react-chuck-norris

Starting point for employee onboarding as well as new production projects
TypeScript
1
star
43

akka-typed-event-sourcing-example

Akka Typed Event Sourcing Example Applikation
Scala
1
star