• Stars
    star
    100
  • Rank 329,270 (Top 7 %)
  • Language
    Ruby
  • License
    Apache License 2.0
  • Created about 7 years ago
  • Updated 6 days ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

CockroachDB adapter for ActiveRecord.

ActiveRecord CockroachDB Adapter

CockroachDB adapter for ActiveRecord 5, 6, and 7. This is a lightweight extension of the PostgreSQL adapter that establishes compatibility with CockroachDB.

Installation

Add this line to your project's Gemfile:

gem 'activerecord-cockroachdb-adapter', '~> 7.0.0'

If you're using Rails 5.2, use the 5.2.x versions of this gem.

If you're using Rails 6.0, use the 6.0.x versions of this gem.

If you're using Rails 7.0, use the 7.0.x versions of this gem.

In database.yml, use the following adapter setting:

development:
  adapter: cockroachdb
  port: 26257
  host: <hostname>
  user: <username>

Configuration

In addition to the standard adapter settings, CockroachDB also supports the following:

  • use_follower_reads_for_type_introspection: Use follower reads on queries to the pg_type catalog when set to true. This helps to speed up initialization by reading historical data, but may not find recently created user-defined types.
  • disable_cockroachdb_telemetry: Determines if a telemetry call is made to the database when the connection pool is initialized. Setting this to true will prevent the call from being made.

Working with Spatial Data

The adapter uses RGeo and RGeo-ActiveRecord to represent geometric and geographic data as Ruby objects and easily interface them with the adapter. The following is a brief introduction to RGeo and tips to help setup your spatial application. More documentation about RGeo can be found in the YARD Docs and wiki.

Installing RGeo

RGeo can be installed with the following command:

gem install rgeo

The best way to use RGeo is with GEOS support. If you have a version of libgeos installed, you can check that it was properly linked with RGeo by running the following commands:

require 'rgeo'

RGeo::Geos.supported?
#=> true

If this is false, you may need to specify the GEOS directory while installing. Here's an example linking it to the CockroachDB GEOS binary.

gem install rgeo -- --with-geos-dir=/path/to/cockroach/lib/

Working with RGeo

RGeo uses factories to create geometry objects and define their properties. Different factories define their own implementations for standard methods. For instance, the RGeo::Geographic.spherical_factory accepts latitudes and longitues as its coordinates and does computations on a spherical surface, while RGeo::Cartesian.factory implements geometry objects on a plane.

The factory (or factories) you choose to use will depend on the requirements of your application and what you need to do with the geometries they produce. For example, if you are working with points or other simple geometries across long distances and need precise results, the spherical factory is a good choice. If you're working with polygons or multipolygons and analyzing complex relationships between them (intersects?, difference, etc.), then using a cartesian factory backed by GEOS is a much better option.

Once you've selected a factory, you need to create objects. RGeo supports geometry creation through standard constructors (point, line_string, polygon, etc.) or by WKT and WKB.

require 'rgeo'
factory = RGeo::Cartesian.factory(srid: 3857)

# Create a line_string from points
pt1 = factory.point(0,0)
pt2 = factory.point(1,1)
pt3 = factory.point(2,2)
line_string = factory.line_string([pt1,pt2,pt3])

p line_string.length
#=> 2.8284271247461903

# check line_string equality
line_string2 = factory.parse_wkt("LINESTRING (0 0, 1 1, 2 2)")
p line_string == line_string2
#=> true

# create polygon and test intersection with line_string
pt4 = factory.point(0,2)
outer_ring = factory.linear_ring([pt1,pt2,pt3,pt4,pt1])
poly = factory.polygon(outer_ring)

p line_string.intersects? poly
#=> true

Creating Spatial Tables

To store spatial data, you must create a column with a spatial type. PostGIS provides a variety of spatial types, including point, linestring, polygon, and different kinds of collections. These types are defined in a standard produced by the Open Geospatial Consortium. You can specify options indicating the coordinate system and number of coordinates for the values you are storing.

The adapter extends ActiveRecord's migration syntax to support these spatial types. The following example creates five spatial columns in a table:

create_table :my_spatial_table do |t|
  t.column :shape1, :geometry
  t.geometry :shape2
  t.line_string :path, srid: 3857
  t.st_point :lonlat, geographic: true
  t.st_point :lonlatheight, geographic: true, has_z: true
end

The first column, "shape1", is created with type "geometry". This is a general "base class" for spatial types; the column declares that it can contain values of any spatial type.

The second column, "shape2", uses a shorthand syntax for the same type as the shape1 column. You can create a column either by invoking column or invoking the name of the type directly.

The third column, "path", has a specific geometric type, line_string. It also specifies an SRID (spatial reference ID) that indicates which coordinate system it expects the data to be in. The column now has a "constraint" on it; it will accept only LineString data, and only data whose SRID is 3857.

The fourth column, "lonlat", has the st_point type, and accepts only Point data. Furthermore, it declares the column as "geographic", which means it accepts longitude/latitude data, and performs calculations such as distances using a spheroidal domain.

The fifth column, "lonlatheight", is a geographic (longitude/latitude) point that also includes a third "z" coordinate that can be used to store height information.

The following are the data types understood by PostGIS and exposed by the adapter:

  • :geometry -- Any geometric type
  • :st_point -- Point data
  • :line_string -- LineString data
  • :st_polygon -- Polygon data
  • :geometry_collection -- Any collection type
  • :multi_point -- A collection of Points
  • :multi_line_string -- A collection of LineStrings
  • :multi_polygon -- A collection of Polygons

Following are the options understood by the adapter:

  • :geographic -- If set to true, create a PostGIS geography column for longitude/latitude data over a spheroidal domain; otherwise create a geometry column in a flat coordinate system. Default is false. Also implies :srid set to 4326.
  • :srid -- Set a SRID constraint for the column. Default is 4326 for a geography column, or 0 for a geometry column. Note that PostGIS currently (as of version 2.0) requires geography columns to have SRID 4326, so this constraint is of limited use for geography columns.
  • :has_z -- Specify that objects in this column include a Z coordinate. Default is false.
  • :has_m -- Specify that objects in this column include an M coordinate. Default is false.

To create a PostGIS spatial index, add using: :gist to your index:

add_index :my_table, :lonlat, using: :gist

# or

change_table :my_table do |t|
  t.index :lonlat, using: :gist
end

Configuring ActiveRecord

ActiveRecord's usefulness stems from the way it automatically configures classes based on the database structure and schema. If a column in the database has an integer type, ActiveRecord automatically casts the data to a Ruby Integer. In the same way, the adapter automatically casts spatial data to a corresponding RGeo data type.

RGeo offers more flexibility in its type system than can be interpreted solely from analyzing the database column. For example, you can configure RGeo objects to exhibit certain behaviors related to their serialization, validation, coordinate system, or computation. These settings are embodied in the RGeo factory associated with the object.

You can configure the adapter to use a particular factory (i.e. a particular combination of settings) for data associated with each type in the database.

Here's an example using a Geos default factory:

RGeo::ActiveRecord::SpatialFactoryStore.instance.tap do |config|
  # By default, use the GEOS implementation for spatial columns.
  config.default = RGeo::Geos.factory_generator

  # But use a geographic implementation for point columns.
  config.register(RGeo::Geographic.spherical_factory(srid: 4326), geo_type: "point")
end

The default spatial factory for geographic columns is RGeo::Geographic.spherical_factory. The default spatial factory for cartesian columns is RGeo::Cartesian.preferred_factory. You do not need to configure the SpatialFactoryStore if these defaults are ok.

More information about configuration options for the SpatialFactoryStore can be found in the rgeo-activerecord docs.

Reading and Writing Spatial Columns

When you access a spatial attribute on your ActiveRecord model, it is given to you as an RGeo geometry object (or nil, for attributes that allow null values). You can then call the RGeo api on the object. For example, consider the MySpatialTable class we worked with above:

record = MySpatialTable.find(1)
point = record.lonlat                  # Returns an RGeo::Feature::Point
p point.x                              # displays the x coordinate
p point.geometry_type.type_name        # displays "Point"

The RGeo factory for the value is determined by how you configured the ActiveRecord class, as described above. In this case, we explicitly set a spherical factory for the :lonlat column:

factory = point.factory                # returns a spherical factory

You can set a spatial attribute by providing an RGeo geometry object, or by providing the WKT string representation of the geometry. If a string is provided, the adapter will attempt to parse it as WKT and set the value accordingly.

record.lonlat = 'POINT(-122 47)'  # sets the value to the given point

If the WKT parsing fails, the value currently will be silently set to nil. In the future, however, this will raise an exception.

record.lonlat = 'POINT(x)'         # sets the value to nil

If you set the value to an RGeo object, the factory needs to match the factory for the attribute. If the factories do not match, the adapter will attempt to cast the value to the correct factory.

p2 = factory.point(-122, 47)       # p2 is a point in a spherical factory
record.lonlat = p2                 # sets the value to the given point
record.shape1 = p2                 # shape1 uses a flat geos factory, so it
                                   # will cast p2 into that coordinate system
                                   # before setting the value
record.save

If you attempt to set the value to the wrong type, such as setting a linestring attribute to a point value, you will get an exception from the database when you attempt to save the record.

record.path = p2      # This will appear to work, but...
record.save           # This will raise an exception from the database

Spatial Queries

You can create simple queries based on representational equality in the same way you would on a scalar column:

record2 = MySpatialTable.where(:lonlat => factory.point(-122, 47)).first

You can also use WKT:

record3 = MySpatialTable.where(:lonlat => 'POINT(-122 47)').first

Note that these queries use representational equality, meaning they return records where the lonlat value matches the given value exactly. A 0.00001 degree difference would not match, nor would a different representation of the same geometry (like a multi_point with a single element). Equality queries aren't generally all that useful in real world applications. Typically, if you want to perform a spatial query, you'll look for, say, all the points within a given area. For those queries, you'll need to use the standard spatial SQL functions provided by PostGIS.

To perform more advanced spatial queries, you can use the extended Arel interface included in the adapter. The functions accept WKT strings or RGeo features.

point = RGeo::Geos.factory(srid: 0).point(1,1)

# Example Building model where geom is a column of polygons.
buildings = Building.arel_table
containing_buiildings = Building.where(buildings[:geom].st_contains(point))

See the rgeo-activerecord YARD Docs for a list of available PostGIS functions.

Validation Issues

If you see an RGeo::Error::InvalidGeometry (LinearRing failed ring test) message while loading data or creating geometries, this means that the geometry you are trying to instantiate is not topologically valid. This is usually due to self-intersections in the geometry. The default behavior of RGeo factories is to raise this error when an invalid geometry is being instansiated, but this can be ignored by setting the uses_lenient_assertions flag to true when creating your factory.

regular_fac = RGeo::Geographic.spherical_factory
modified_fac = RGeo::Geographic.spherical_factory(uses_lenient_assertions: true)

wkt = "POLYGON (0 0, 1 1, 0 1, 1 0, 0 0)" # closed ring with self intersection

regular_fac.parse_wkt(wkt)
#=> RGeo::Error::InvalidGeometry (LinearRing failed ring test)

p modified_fac.parse_wkt(wkt)
#=>  #<RGeo::Geographic::SphericalPolygonImpl>

Be careful when performing calculations on potentially invalid geometries, as the results might be nonsensical. For example, the area returned of an hourglass made of 2 equivalent triangles with a self-intersection in the middle is 0.

Note that when using the spherical_factory, there is a chance that valid geometries will be interpreted as invalid due to floating point issues with small geometries.

Modifying the adapter?

See CONTRIBUTING.md for more details on setting up the environment and making modifications.

More Repositories

1

cockroach

CockroachDB - the open source, cloud-native distributed SQL database.
Go
29,128
star
2

pebble

RocksDB/LevelDB inspired key-value database in Go
Go
4,455
star
3

errors

Go error library with error portability over the network
Go
1,869
star
4

copyist

Mocking your SQL database in Go tests has never been easier.
Go
833
star
5

apd

Arbitrary-precision decimals for Go
Go
540
star
6

cockroach-operator

k8s operator for CRDB
Go
264
star
7

docs

CockroachDB user documentation
HTML
177
star
8

django-cockroachdb

CockroachDB Backend for Django
Python
155
star
9

sqlalchemy-cockroachdb

SQLAlchemy adapter for CockroachDB
Python
135
star
10

rpc-bench

Benchmarking various RPC implementations
Go
87
star
11

examples-orms

Sample uses of CockroachDB with popular ORMs
Go
82
star
12

helm-charts

Helm charts for cockroachdb
Go
81
star
13

examples-go

Sample uses of cockroachDB.
Go
75
star
14

terraform-provider-cockroach

Terraform provider for CockroachDB Cloud
Go
53
star
15

cdc-sink

cdc-sink is a toolkit for ingesting logical replication feeds into a CockroachDB cluster
Go
52
star
16

sequelize-cockroachdb

Use Sequelize and CockroachDB together
JavaScript
52
star
17

cockroachdb-todo-apps

CockroachDB To-Do Apps
Python
51
star
18

c-rocksdb

🚫 DEPRECATED
C++
46
star
19

datadriven

Data-Driven Testing for Go
Go
38
star
20

logtags

key/value annotations for Go contexts
Go
35
star
21

cockroach-gen

CockroachDB with pre-generated Go code
Go
33
star
22

movr

A fictional ride sharing company.
Python
32
star
23

loadgen

CockroachDB load generators
Go
30
star
24

k8s

Images and utilities to run cockroach on kubernetes
Go
26
star
25

redact

Utilities to redact Go strings for confidentiality
Go
25
star
26

c-snappy

C++
21
star
27

ui

Published assets for Cockroach Labs user interfaces
TypeScript
19
star
28

cockroachdb-parser

Apache licensed CockroachDB parser and dependencies.
Go
19
star
29

roachprod

Internal CockroachDB production testing tool
Go
15
star
30

cockroachdb-typescript

A modern cloud-native web app using TypeScript, React, Prisma, Netlify serverless functions, and CockroachDB
TypeScript
14
star
31

crlfmt

Formatter for CockroachDB's additions to the Go style guide.
Go
14
star
32

homebrew-tap

`brew install cockroachdb/tap/cockroach`
Ruby
14
star
33

c-protobuf

🚫 DEPRECATED: go-gettable version of google/protobuf
C++
13
star
34

watermill-crdb

CockroachDB Pub/Sub for the Watermill project.
Go
13
star
35

circuitbreaker

Circuit Breakers in Go
Go
13
star
36

cockroachdb-cloudformation

Quickly setup dev/test CockroachDB clusters using AWS CloudFormation and Kubernetes
Shell
12
star
37

molt

Migrate Off Legacy Things - open-source tooling to assist migrations to CockroachDB.
Go
12
star
38

vendored

CockroachDB's vendored Go dependencies
Go
11
star
39

c-jemalloc

🚫 DEPRECATED
C
11
star
40

yacc

Fork of go yacc tool with increased hardcoded constants to handle larger grammars.
Go
11
star
41

examples-python

Sample uses of CockroachDB
Python
10
star
42

c-lz4

C
10
star
43

build-cache

Cache the installed output of go builds
Go
9
star
44

definitive_guide_sample_code

Sample code for The Definitive Guide to CockroachDB
JavaScript
8
star
45

quickstart-code-samples

Java
8
star
46

pcf-crdb-service-broker

Pivotal Cloud Foundry service broker for CockroachDB.
Go
8
star
47

dcos-cockroachdb-service

Framework for running CockroachDB on Mesosphere DC/OS
Python
8
star
48

backport

automatically backport pull requests
Go
7
star
49

cockroach-proto

Protocol Buffer
7
star
50

python-heroku-cockroachdb

A modern cloud-native web app using Python, Flask, Heroku, and CockroachDB
Python
7
star
51

walkabout

Walkabout generates visitor-pattern accessors for your existing go structs.
Go
6
star
52

sample-apps

6
star
53

nextjs-react-vercel-crdb-app

A CockroachDB sample app for coordinating social events using Next.js with react-bootstrap deployed on vercel
JavaScript
6
star
54

rksql

Fork of scaledata/rksql
Go
5
star
55

ddshop

TodoMVC app for distributed database workshop
JavaScript
5
star
56

postgres-test

Dockerfile
5
star
57

benchviz

A tool used for visualizing results from benchmark tests over time.
Go
5
star
58

stress

A fork of Golang's stress function.
Go
5
star
59

cockroach-cloud-sdk-go

Go client SDK for CockroachDB Cloud
Mustache
5
star
60

example-app-rust-postgres

Rust
4
star
61

protoc-gen-gogoroach

protoc (protobuf compiler) plugin for CockroachDB protobuf files
Go
4
star
62

generated-diagrams

HTML
4
star
63

roachspeed

codespeed instance
Python
4
star
64

hibernate-savepoint-fix

Java
4
star
65

scripts

Utility and helper scripts for CockroachDB contributors -
Vim Script
4
star
66

go-lab

Scripts and test programs to reveal what happens under the hood in Go.
Go
4
star
67

yarn-vendored

CockroachDB's vendored JavaScript dependencies
3
star
68

cockroachdb.github.io

Supporting static content for CockroachDB
JavaScript
3
star
69

cgo_static_boom

cgo static binary boom
Go
3
star
70

docs-docker

Docker image for the docs build
3
star
71

libedit

CockroachDB fork of the libedit terminal line-editing library
3
star
72

gostdlib

Vendor-friendly Go standard library packages
Go
3
star
73

dev

A utility for performing common CockroachDB development tasks
3
star
74

example-app-python-psycopg3

Simple CRUD application in Python using the psycopg3 driver.
Python
3
star
75

scp-interview

Shell
3
star
76

efcore.pg.cockroach

C#
3
star
77

university-event-driven-architecture-for-java-developers-app-exercises

Exercise code for Cockroach University - Event Driven Architecture for Java Developers
Java
3
star
78

bincheck

Verify CockroachDB binaries
Shell
3
star
79

eddie

Big Eddie, the golang contract enforcer
Go
3
star
80

roachperf-og

🚫 Deprecated CockroachDB performance tool β€” use roachprod instead!
Go
3
star
81

university-getting-started-with-sql-app-exercises

Exercise code for the Cockroach University - Getting Started with SQL for Application Developers course
Shell
3
star
82

jsonb-spec

Acceptance tests/spec for CockroachDB JSONB support, complement to scoping RFC
JavaScript
3
star
83

social-events-app

JavaScript
3
star
84

ttycolor

Conditionally expose ANSI color codes for use on a terminal
Go
2
star
85

admin-ui-components

Shared frontend components between the CRDB Admin UI and other apps
TypeScript
2
star
86

rails-crdb-app

A CockroachDB leaderboard sample application using Rails deployed on Heroku
Ruby
2
star
87

fullstack-node-cockroachdb-app

Fullstack sample application in Node.js using Sequelize
JavaScript
2
star
88

university-multi-region-course-app-exercises

exercise code for the cockroach university multi-region course
Java
2
star
89

go-plus

API library for CRL's custom Go extensions
Go
2
star
90

tokenbucket

Token bucket implementation in Go
Go
2
star
91

homebrew-go

🚫 DEPRECATED 🚫 β€” use `brew install go` instead
Ruby
2
star
92

build-utils

🚫 DEPRECATED: Build utilities for CockroachDB
Go
2
star
93

psycopg2-cockroachdb

Testing psycopg2 against cockroachdb
Shell
2
star
94

katacoda

In-browser, interactive tutorials on CockroachDB
Shell
2
star
95

cockroachdb-hibernate

Package to improve compatibility between CockroachDB and Hibernate
2
star
96

pulumi-poc

Demo of simple control plane built on top of Pulumi.
Go
2
star
97

avrogen

Avro generator for import testing
Go
1
star
98

redcarpet-extender

Extends the Redcarpet Markdown parser to handle CockroachDB documentation
1
star
99

s3checker

Check s3 support for bulk operations
Go
1
star
100

university-getting-started-with-node-postgres-app-exercises

JavaScript
1
star