• Stars
    star
    59
  • Rank 509,601 (Top 11 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created over 3 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Goven (go-oven) is a go library that allows you to have a drop-in query language for your database schema.

Go Reference Go Report Card codecov Mentioned in Awesome Go

Goven 🧑‍🍳

Goven (go-oven) is a go library that allows you to have a drop-in query language for your database schema.

  • Take any gorm database object and with a few lines of code you can make it searchable.
  • Safe query language not exposing SQL to your users.
  • Easily extensible to support more advanced queries for your own schema.
  • Basic grammar that allows for powerful queries.

Like a real life oven, it takes something raw (your database struct + a query input) and produces something specific (the schema specific parser + SQL output). We call the adaptors "recipes" that Goven can make. Currently Goven only supports a SQL adaptor, but the AST produced by the lexer/parser can easily be extended to other query languages.

Recipes

Basic Example

You can make a basic query using gorm against your database, something like this:

reflection := reflect.ValueOf(&User{})
queryAdaptor, err := sql_adaptor.NewDefaultAdaptorFromStruct(reflection)
if err != nil {
    return nil, err
}

dbQuery := db.WithContext(ctx)
parsedQuery, err := queryAdaptor.Parse("(name=james AND age > 11) OR [email protected]")
if err != nil {
	return nil, err
}
dbQuery = query.Model(User{}).Where(parsedQuery.Raw, sql_adaptor.StringSliceToInterfaceSlice(parsedQuery.Values)...)
err = dbQuery.Find(&users).Error

The values are interpolated to prevent injection attacks.

Extension Example

You can also extend the basic query language with regex matchers. An example would be having a Tag struct on your User schema.

type User struct {
	gorm.Model
	name string
	tags []Tag
}

type Tag struct {
	gorm.Model
	Key string
	Value string
}

You can make this searchable my defining a regex and a custom matcher.

e.g if we want tags[key]=value to work then we can add the following matcher when creating the adaptor.

KeyValueRegex = `(tags)\[(.+)\]`

// keyValueMatcher is a custom matcher for and tags[y].
func keyValueMatcher(ex *goven_parser.Expression) (*goven_sql.SqlResponse, error) {
	reg, err := regexp.Compile(KeyValueRegex)
	if err != nil {
		return nil, err
	}
	slice := reg.FindStringSubmatch(ex.Field)
	if slice == nil {
		return nil, errors.New("didn't match regex expression")
	}
	if len(slice) < 3 {
		return nil, errors.New("regex match slice is too short")
	}
	sq := goven_sql.SqlResponse{
		Raw:    fmt.Sprintf("id IN (SELECT user_id FROM %s WHERE key=? AND value%s?)", slice[1], ex.Comparator),
		Values: []string{slice[2], ex.Value},
	}
	return &sq, nil
}

Protecting Fields

Sometimes we may not want particular fields to be searchable by end users. You can protect them by removing them from the fields mapping when creating your adaptor.

defaultFields := goven_sql.FieldParseValidatorFromStruct(gorm)
delete(defaultFields, "fieldname")

Grammar

Goven has a simple syntax that allows for powerful queries.

Fields can be compared using the following operators:

=, !=, >=, <=, <, >, %

The % operator allows you to do partial string matching using LIKE.

Multiple queries can be combined using AND, OR.

Together this means you can build up a query like this:

model_name=iris AND version>=2.0

More advanced queries can be built up using bracketed expressions:

(model_name=iris AND version>=2.0) OR artifact_type=TENSORFLOW

More Repositories

1

seldon-core

An MLOps framework to package, deploy, monitor and manage thousands of production machine learning models
HTML
4,320
star
2

alibi

Algorithms for explaining machine learning models
Python
2,370
star
3

alibi-detect

Algorithms for outlier, adversarial and drift detection
Python
2,202
star
4

seldon-server

Machine Learning Platform and Recommendation Engine built on Kubernetes
Java
1,475
star
5

MLServer

An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
Python
675
star
6

tempo

MLOps Python Library
Python
112
star
7

seldonio.github.com

Seldon Documentation
HTML
31
star
8

k8s-local-docker-registry

Shell
29
star
9

seldon-spark

Seldon Spark Jobs
26
star
10

semantic-vectors-lucene-tools

Tools for building a Lucene index for Semantic Vectors
Java
21
star
11

mlgraph

Machine Learning Inference Graph Spec
21
star
12

seldon-ucl

Seldon UCL Project
JavaScript
17
star
13

seldon-vm

Seldon VM repo
JavaScript
16
star
14

seldon-deploy-sdk

SDK for Seldon Deploy
Mustache
13
star
15

sig-mlops-jenkins-classic

Jupyter Notebook
13
star
16

ml-prediction-schema

Generic schema structure for machine learning model predictions
13
star
17

seldon-operator

Seldon Core Operator for Kubernetes
Go
12
star
18

deploy-workshops

Jupyter Notebook
12
star
19

sig-mlops-seldon-jenkins-x

Jupyter Notebook
11
star
20

cassava-example

Example mlserver and seldon deployment for a cassava leaf classifier
Python
10
star
21

importer-movielens-10m

Create Seldon data import files from Movielens 10m source data
Python
10
star
22

trtis-k8s-scheduler

Custom Scheduler to deploy ML models to TRTIS for GPU Sharing
Go
10
star
23

movie-demo-frontend

js frontend for movie recommender demo
JavaScript
9
star
24

seldon-core-launcher

Seldon Core Cloud Launcher
Jupyter Notebook
9
star
25

seldon-core-examples

Python
8
star
26

eubot

Machine learning classifer for the EU Referendum
Python
7
star
27

seldon-js-lib

A Javascript library to interact with the Seldon Server
7
star
28

ansible-k8s-collection

Collection of Ansible roles and playbooks crafted for Seldon ecosystem
Jinja
7
star
29

seldon-importer-web

Web page importer
6
star
30

tensorflow-demo-notebooks

Building and deploying a TensorFlow MNIST digit classifier on Kubernetes with Seldon
Jupyter Notebook
6
star
31

seldon-java-client

Seldon Java REST Client
5
star
32

jenkins-x-seldon-core-sandbox

HTML
5
star
33

deep-mnist-webapp

A webapp that recognises characters you draw using Seldon and Tensorflow.
JavaScript
5
star
34

seldon-server-config-template

5
star
35

seldon-models

A repository of training, inference and packaging code for Seldon demo models
Jupyter Notebook
5
star
36

seldon-deploy-operator

Seldon Deploy installation
Makefile
4
star
37

i-am-spartakus

Go
4
star
38

cicd-demo-model-source-files

Makefile
4
star
39

triton-python-examples

Triton inference server python backend examples
Python
3
star
40

DistributedKernelShap

Python
3
star
41

JPMML-utils

Helper function to use JPMML with Seldon-Core
Java
3
star
42

seldon-server-config-vm

Seldon Server configuration required for the Docker VM
3
star
43

seldon-gcp-marketplace

Seldon Core GCP Marketplace
Makefile
2
star
44

environment-paladinrose-staging

Makefile
2
star
45

environment-seldon-core-test-ci-cluster-dev

Shell
2
star
46

alibi-testing

Repository for storing and loading model binaries for testing purposes
Python
2
star
47

environment-paladinrose-production

Makefile
2
star
48

seldon

Seldon Top Level Repo
2
star
49

helm-charts

Seldon Helm Charts
Mustache
2
star
50

cicd-demo-k8s-manifest-files

Shell
2
star
51

seldon-java-wrapper

Wrap java code for use with seldon-core
Java
2
star
52

test-ci-project

1
star
53

seldon-gitops

Example GitOps repository.
1
star
54

seldon-core-aws

Seldon Core AWS Marketplace Helm Charts
Smarty
1
star
55

seldon-deploy-demos-gitops-template

1
star
56

movie-demo-setup

movie-demo-setup
JavaScript
1
star
57

seldon-prometheus-exporter

Go
1
star
58

seldon-deploy-resources

Dockerfile
1
star
59

seldon-mlmd-tools

Python
1
star