• Stars
    star
    214
  • Rank 184,678 (Top 4 %)
  • Language
    Go
  • License
    MIT License
  • Created about 7 years ago
  • Updated 5 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

(AE|Cloud) Datastore Wrapper

Datastore Wrapper Go Documentation CI

(AppEngine | Cloud) Datastore wrapper for Go 👉

Simple. Happy. Respect standard library.

$ go get -u go.mercari.io/datastore/v2

see v2 doc at first.

Feature

DO

  • Wrap google.golang.org/appengine/datastore and cloud.google.com/go/datastore
    • keep key behavior
    • align to cloud.google.com/go/datastore first
  • Re-implement datastore package
  • Re-implement datastore.SaveStruct & LoadStruct
    • Ignore unmapped property
    • Add PropertyTranslator interface
      • Convert types like mytime.Unix to time.Time and reverse it
      • Rename property like CreatedAt to createdAt or created_at and reverse it
  • Re-implement PropertyLoadSaver
    • Pass context.Context to Save & Load method
  • Add retry feature to each RPC
    • e.g. Retry AllocateID when it failed
  • Add middleware layer
    • About...
      • Local Cache
      • AE Memcache
      • Logging
      • Retry
      • etc...
    • Easy to ON/OFF switching
  • Add some useful methods
    • aedatastore/TransactionContext

DON'T

  • have utility functions
  • support firestore

Restriction

  • aedatastore package
    • When using slice of struct, MUST specified datastore:",flatten" option.
      • original (ae & cloud) datastore.SaveStruct have different behaviors.
      • see aeprodtest/main.go /api/test3

Committers

Contribution

Please read the CLA below carefully before submitting your contribution.

https://www.mercari.com/cla/

Setup environment & Run tests

  • requirements
    • gcloud sdk
      • gcloud components install app-engine-go
      • gcloud components install beta cloud-datastore-emulator
  1. Testing in local
$ ./setup.sh # exec once
$ ./serve.sh # exec in background
$ ./test.sh

License

Copyright 2017 Mercari, Inc.

Licensed under the MIT License.

More Repositories

1

ml-system-design-pattern

System design patterns for machine learning
2,221
star
2

engineer-vocabulary-list

Engineer Vocabulary List in Japanese/English
1,770
star
3

gaurun

General push notification server in Go
Go
931
star
4

production-readiness-checklist

Production readiness checklist used for Mercari and Merpay microservices
844
star
5

tfnotify

A CLI command to parse Terraform execution result and notify it to GitHub
Go
619
star
6

Mew

The framework that support making MicroViewController.
Swift
485
star
7

tortoise

Tortoise: Shell-Shockingly-Good Kubernetes Autoscaling
Go
380
star
8

grpc-http-proxy

A reverse proxy server which translate JSON HTTP requests to gRPC calls based on protoreflect
Go
371
star
9

go-circuitbreaker

A context aware circuit breaker library in Go.
Go
358
star
10

mercari-microservices-example

Go
327
star
11

QRScanner

A simple QR Code scanner framework for iOS. Provides a similar scan effect to ios13+.
Swift
313
star
12

grpc-federation

gRPC Federation generates a gRPC server by writing a custom option in Protocol Buffers
Go
238
star
13

go-httpdoc

Golang package for generating API documentation from httptest. See example output
Go
228
star
14

mercari-slack-guidelines

Slack guidelines of Mercari.
202
star
15

ShimmerView

ShimmerView is a collection of APIs to construct Skelton View + Shimmering Effect type loading indicator on UIKit and SwiftUI.
Swift
195
star
16

mercari-engineering-ladder

Mercari's Expectations for Engineers in Various Stages of Their Career
163
star
17

mtc2018-app

The Official Conference App for Mercari Tech Conf 2018
Dart
134
star
18

go-grpc-interceptor

gRPC server insterceptor for golang
Go
124
star
19

go-dnscache

Go package for caching DNS lookup results in memory.
Go
121
star
20

certificate-expiry-monitor-controller

Certificate Expiry Monitor Controller monitors the expiration of TLS certificates used in Ingress.
Go
117
star
21

souzoh-recruitment

117
star
22

mtc2018-web

Mercari Tech Conf 2018
TypeScript
100
star
23

BottomHalfModal

A customizable bottom half modal used in merpay
Swift
95
star
24

kubetempura

Go
93
star
25

widebullet

Widebullet is an API gateway with JSON-RPC
Go
91
star
26

dietcube

The world super fly weight & flexible PHP framework.
PHP
79
star
27

docker-appengine-go

Projects has been moved
Dockerfile
74
star
28

spanner-autoscaler

Kubernetes Operator for Cloud Spanner autoscaling
Go
73
star
29

RxReduxK

Micro-framework for Redux implemented in Kotlin
Kotlin
67
star
30

testdeck

Testdeck is a framework for integration, end-to-end (E2E), and security testing of gRPC microservices written in Golang.
Go
65
star
31

DataflowTemplate

Mercari Dataflow Template
Java
64
star
32

go-emv-code

EMV® QR Code Encoder/Decoder for Go.
Go
64
star
33

hcledit

Go package to edit HCL configuration
Go
49
star
34

go-httpstats

Go package for reporting HTTP stats
Go
44
star
35

RemoteDataK

Algebraic data type (ADT) to represent the state of data that is loading from/to remote sources/destinations
Kotlin
44
star
36

fractal

Swift
36
star
37

go-bps

Go package to manage the basis point
Go
36
star
38

mtc2018-app-SwiftUI

Project to rewrite MTC2018 App by SwiftUI
Swift
33
star
39

siberi-android

A/B testing library for Android
Java
33
star
40

BalloonView

Makes balloon-like view with an arrow pointing to another view. Useful for onboarding tutorials
Swift
25
star
41

DataflowTemplates

Convenient Dataflow pipelines for transforming data between cloud data sources
Java
24
star
42

yasashii-wfh-comms

yasashii-WFH-Communication-Best-Practices
19
star
43

mercari-ml-merrec-pub-us

Python
18
star
44

swiftui-chart

Swift
17
star
45

honyakubot

Slack translation bot
Swift
14
star
46

github-app-token-generator

A simple github action written in go to retrieve an installation access token for an app installed into an organization.
Go
14
star
47

github-token-app

Github Token App is a package for generating short lived github tokens (expires in 1 hour) with minimum necessary permissions.
Python
13
star
48

terraform-provider-spinnaker

A Spinnaker provider for Terraform
Go
13
star
49

composer-diff-plugin

composer plugin to show library version diff at "composer update".
PHP
12
star
50

CFQIRBM

Collaborative Filtering Quantum Infinite Restricted Boltzmann Machine
Jupyter Notebook
12
star
51

Remi

Mascot
10
star
52

terraform-provider-openpgp

Terraform OpenPGP provider
Go
8
star
53

imageflux-cli

Go
8
star
54

eslint-config-mercari

JavaScript
7
star
55

universal-apk-plugin

Gradle Plugin allowing to create an Universal APK for debug purpose.
Kotlin
6
star
56

mkdocs-git-snippet

Python
6
star
57

commitver

Derive a semver version from the commit history in any repo.
Shell
6
star
58

kafka-connect-transform-kryptonite-gcp

Java
5
star
59

proto-to-type

A Node.js library that generates type definition for TypeScript from Protocol Buffer
TypeScript
4
star
60

stylelint-config-mercari

The shareable config for @stylelint by @mercari.
CSS
3
star
61

merlin

Merlin is an agent sends out alerts when kubernetes resources are misconfigured or not comply with custom rules.
Go
3
star
62

modserver

modserver is a simple Go module server.
Go
3
star
63

extract-primitives

extract-primitives extracts primitive values from TypeScript's declaration file(.d.ts).
TypeScript
2
star
64

Whitesource-Scan-Action

Shell
2
star
65

terraform-exfiltration-lab

HCL
2
star
66

merpay-netpayment-sdk-php

Merpay Online Payments SDK for PHP
PHP
1
star
67

pubsubloader

Archived. You can find a better way on https://github.com/GoogleCloudPlatform/DataflowTemplates/tree/master/v2/streaming-data-generator. Cloud Dataflow based Cloud Pub/Sub load generator and integration tester with custom data
Scala
1
star