• Stars
    star
    167
  • Rank 226,592 (Top 5 %)
  • Language
    C
  • License
    Other
  • Created over 9 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Community version of the PostgreSQL multi-purpose Ruby Foreign Data Wrapper

Holycorn: PostgreSQL multi-purpose Ruby data wrapper Travis

(m)Ruby + PostgreSQL = <3

Holycorn makes it easy to implement a Foreign Data Wrapper using Ruby.

It is based on top of mruby, that provides sandboxing capabilities the regular Ruby VM "MRI/CRuby" does not implement.

Join the chat at https://gitter.im/franckverrot/holycorn

Built-in Wrappers

Holycorn is the combination of the mruby VM, some supporting gems (some basic libraries and some more advanced ones), and custom code that implement the actual foreign data wrappers.

All the following wrappers are currently linked against Holycorn:

  • Redis, using the mruby-redis gem

INSTALLATION

Prerequisites

  • PostgreSQL 9.4+

Setup

Simply run

rake

to vendor and build mruby.

Now that mruby is built, building holycorn requires to run

make

and installing it only requires to run

make install

Now connect to PostgreSQL and install the extension:

位 psql
psql (9.5.14)
Type "help" for help.

DROP EXTENSION holycorn CASCADE;
CREATE EXTENSION holycorn;

Using Builtin Foreign Data Wrappers

Manual Creation of Foreign Tables

A set of builtin FDW are distributed with Holycorn for an easy setup. All one needs to provide are the options that will allow the FDW to be configured:

位 psql
psql (9.5.14)
Type "help" for help.

CREATE SERVER holycorn_server FOREIGN DATA WRAPPER holycorn;
CREATE FOREIGN TABLE redis_table (key text, value text)
  SERVER holycorn_server
  OPTIONS ( wrapper_class 'HolycornRedis'
          , host '127.0.0.1'
          , port '6379'
          , db '0');

Automatic Import using IMPORT FOREIGN SCHEMA

Alternatively, Holycorn supports IMPORT FOREIGN SCHEMA and the same can be accomplished by using that statement instead:

位 psql
psql (9.5.14)

CREATE SERVER holycorn_server FOREIGN DATA WRAPPER holycorn;
IMPORT FOREIGN SCHEMA holycorn_schema
FROM SERVER holycorn_server
INTO holycorn_tables
OPTIONS ( wrapper_class 'HolycornRedis'
        , host '127.0.0.1'
        , port '6379'
        , db '0'
        , prefix 'holycorn_'
        );

Please note that custom data wrappers have to use the holycorn_schema schema as it would otherwise result in an error at runtime.

Note: IMPORT FOREIGN SCHEMA requires us to use a custom target schema and Holycorn encourages users to use a prefix to help avoiding name collisions.

Access Foreign Tables

As Holycorn doesn't support INSERTs yet, let's create some manually:

redis-cli
127.0.0.1:6379> select 0
OK
127.0.0.1:6379> keys *
(empty list or set)
127.0.0.1:6379> set foo 1
OK
127.0.0.1:6379> set bar 2
OK
127.0.0.1:6379> set baz 3
OK
127.0.0.1:6379> keys *
1) "bar"
2) "foo"
3) "baz"

Now that the table has been created and we have some data in Redis, we can select data from the foreign table.

SELECT * from redis_table;

 key | value
-----+-------
 bar | 2
 foo | 1
 baz | 3
(3 rows)

Using custom scripts

Alternatively, custom scripts can be used as the source for a Foreign Data Wrapper:

DROP EXTENSION holycorn CASCADE;
CREATE EXTENSION holycorn;
CREATE SERVER holycorn_server FOREIGN DATA WRAPPER holycorn;
CREATE FOREIGN TABLE holytable (some_date timestamptz) \
  SERVER holycorn_server
  OPTIONS (wrapper_path '/tmp/source.rb');

(IMPORT FOREIGN SCHEMA is also supported here.)

And the source file of the wrapper:

# /tmp/source.rb
class Producer
  def initialize(env = {}) # env contains informations provided by Holycorn
  end

  def each
    @enum ||= Enumerator.new do |y|
      10.times do |t|
        y.yield [ Time.now ]
      end
    end
    @enum.next
  end

  def import_schema(args = {})
    # Keys are:
    #   * local_schema: the target schema
    #   * server_name: name of the foreign data server in-use
    #   * wrapper_class: name of the current class
    #   * any other OPTIONS passed to IMPORT FOREIGN SCHEMA
  end

  self
end

For more details about #import_schema, please take a look at the examples located in in the builtin_wrappers directory.

Now you can select data out of the wrapper:

位 psql
psql (9.5.14)
Type "help" for help.

franck=# SELECT * FROM holytable;
      some_date
---------------------
 2015-06-21 22:39:24
 2015-06-21 22:39:24
 2015-06-21 22:39:24
 2015-06-21 22:39:24
 2015-06-21 22:39:24
 2015-06-21 22:39:24
 2015-06-21 22:39:24
 2015-06-21 22:39:24
 2015-06-21 22:39:24
 2015-06-21 22:39:24
(10 rows)

Pretty neat.

SUPPORTED SCRIPTS

General rules

Any type of Ruby object can act as a FDW. The only requirements are that it can receive .new (with arity = 1) and return an object that can receive each (arity = 0).

It doesn't have to be a Class, and there's currently no will to provide a superclass to be inherited from.

In future versions, there will be many more callbacks to interact with PG's FDW infrastructure through Holycorn.

Also, the script can only be a single word - like MyClass - as long as MyClass has been defined and exists within your compilation of mruby.

Environment

A hash is passed by Holycorn to the Ruby script. Its current keys are:

  • PG_VERSION
  • PG_VERSION_NUM
  • PACKAGE_STRING
  • PACKAGE_VERSION
  • MRUBY_RUBY_VERSION
  • WRAPPER_PATH

SUPPORTED TYPES (Ruby => PG)

Builtin types

  • MRB_TT_FREE => null
  • MRB_TT_FALSE => Boolean
  • MRB_TT_TRUE => Boolean
  • MRB_TT_FIXNUM => Int64
  • MRB_TT_SYMBOL => Text
  • MRB_TT_UNDEF => Unsupported
  • MRB_TT_FLOAT => Float8
  • MRB_TT_CPTR => Unsupported
  • MRB_TT_OBJECT => Text (to_s is called)
  • MRB_TT_CLASS => Text (class.to_s is called)
  • MRB_TT_MODULE => Text (to_s is called)
  • MRB_TT_ICLASS => Unsupported
  • MRB_TT_SCLASS => Unsupported
  • MRB_TT_PROC => Text (inspect is called)
  • MRB_TT_ARRAY => Text (inspect is called)
  • MRB_TT_HASH => Text (inspect is called)
  • MRB_TT_STRING => Text
  • MRB_TT_RANGE => Text (inspect is called)
  • MRB_TT_EXCEPTION => Unsupported
  • MRB_TT_FILE => Unsupported
  • MRB_TT_ENV => Unsupported
  • MRB_TT_DATA => See "Arbitraty Ruby objects" section
  • MRB_TT_FIBER => Text (inspect is called)
  • MRB_TT_MAXDEFINE => Unsupported

Arbitraty Ruby objects

  • Time (Ruby) => timestamptz

CONFIGURATION

Server

None (yet).

Foreign Table

Either wrapper_class or wrapper_path can be used to defined whether an external script or a built-in wrapper will manage the foreign table.

  • wrapper_class: Name of the built-in wrapper class
  • wrapper_path: Path of a custom script

In both case, any other option will be pushed down to the wrapper class via the constructor.

TODO

  • Array type
  • JSON type
  • Range type
  • Support PG 9.5's IMPORT FOREIGN SCHEMA for easy setup

Note on Patches/Pull Requests

  • Fork the project.
  • Make your feature addition or bug fix.
  • Add tests for it. This is important so I don't break it in a future version unintentionally.
  • Commit, do not mess with version or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
  • Send me a pull request. Bonus points for topic branches.

LICENSE

Copyright (c) 2015-2018 Franck Verrot

Holycorn is an Open Source project licensed under the terms of the LGPLv3 license. Please see http://www.gnu.org/licenses/lgpl-3.0.html for license text.

AUTHOR

Franck Verrot, @franckverrot

More Repositories

1

activevalidators

Collection of ActiveModel/ActiveRecord validators
Ruby
306
star
2

terraform-provider-stripe

A Terraform Provider for Stripe
Go
242
star
3

EmulationResources

Collection of resources for emulator developers
HTML
91
star
4

no_querying_views

No more querying views in your Rails apps
Ruby
60
star
5

git_fdw

PostgreSQL Git Foreign Data Wrapper
C
47
star
6

clamav-client

ClamAV::Client connects to a Clam Anti-Virus clam daemon and send commands
Ruby
40
star
7

trek

Trek is a CLI/ncurses explorer for HashiCorp Nomad clusters.
Go
33
star
8

blake2

BLAKE2 - fast secure hashing - for Ruby
Ruby
28
star
9

mruby-r

mruby-r: Use (m)Ruby for returning data to R
Ruby
26
star
10

fruby

A French version of Ruby (based on mruby)
C
19
star
11

rb_import

rb_import: ES6 `import`/Node's `require`, for Ruby.
Ruby
19
star
12

codes-bic-france

Code BIC des Banques Fran莽aises
13
star
13

arnoldc.rb

The ugliest implementation of the Arnold Schwarzenegger based programming language, in Ruby.
Ruby
10
star
14

terraform-provider-homebrew

Terraform provider for Homebrew
Go
8
star
15

ers

ers - ERb-like templating engine for Rust
Rust
8
star
16

yamlsh

Interactive REPL for authoring YAML files
Ruby
8
star
17

git-mine-commit

Mine your git commit hash (SHA1) because you know... YOLO
Shell
7
star
18

tictactoe-elm

Tic Tac Toe with Elm
JavaScript
7
star
19

paybox

Paybox's payment gateway -- UNMAINTAINED
Ruby
7
star
20

awesome-rfcs

Curated list of RFCs / Community Processes
6
star
21

isolate

Prevent processes from accessing unauthorized files
C
6
star
22

gendarme

Gendarme checks for preconditions and postrelations on the methods you want it to monitor.
Ruby
6
star
23

wtf

Secret project, sssshhhh
Ruby
5
star
24

namos

馃殌 Naming and Synchronization in a Decentralized Computer System 馃殌 (Author: David P. Reed)
CSS
5
star
25

tweetnacl

TweetNaCl Ruby C-extension
C
5
star
26

OpenNICSwitcher

OSX app that makes using an OpenNIC DNS server a no-brainer (under development)
Ruby
4
star
27

adversarial-ml

Adversarial Machine Learning
3
star
28

bmmlexporter

Simple exporter from Bmml files
Ruby
3
star
29

Reader.elm

RSS front-end a-la Google Reader created with Elm (elm-lang.org)
Elm
3
star
30

postgresql-performance-subquery-vs-join

R
2
star
31

game-of-life-in-elm

Game of Life in Elm
HTML
2
star
32

raven

Consul-based Control Plane for Envoy Proxy
Go
2
star
33

dependency_revealer

Extract dependencies info from a Gemfile (and a Gemfile.lock)
Ruby
2
star
34

gem-license-check

License reporter and checker for your Ruby projects
Ruby
2
star
35

TCL

Des donn茅es, des transports en communs lyonnais.
JavaScript
1
star
36

golo-hs

Playground, parsing Golo with Haskell's attoparsec
Haskell
1
star
37

TaskRunner

Experimental Task Runner in Haskell
Haskell
1
star
38

fr-lang

Functional Ruby Programming Language Experiment
Ruby
1
star
39

dalek

dalek is an experimental (and unfinished) campfire bot that can self-upgrade, watch the video for more information
Ruby
1
star
40

ar2json

ActiveRecord extension that uses the database to build a record's JSON representation
Ruby
1
star
41

live-note

Live-Note
Python
1
star
42

Reader.ex

REST API for Reader.elm
Elixir
1
star
43

dotfiles

Vim Script
1
star
44

CodeBeautifier

This Rails plugin modifies the content that goes out your Rails application and format it nicely.
Ruby
1
star
45

gemtalker

Talk to Gemcutter's API using XMPP/Google Talk [Invite [email protected] to chat in GTalk]
Python
1
star
46

exportable

exportable will help you export arrays of objects and reuse the export definitions across your classes
Ruby
1
star
47

chrust

Changes the current Rust
Shell
1
star
48

hudson-git-flow

discover new github topic/feature branches and add as Hudson jobs
Ruby
1
star
49

heroku-cedar-examples

Ruby, NodeJS and Clojure app examples to run on Heroku's Cedar stack
Shell
1
star
50

pivotbot

A Pivotal Tracker Command Line Tool
Haskell
1
star
51

lmftfy

Let Me Fork That For You, the best way to get your patch applied to your favorite project
Ruby
1
star
52

autostopper

Simple script that stops your running AWS EC2 instances.
Ruby
1
star
53

jruby-experiments

Some personal JRuby experiments
Ruby
1
star
54

Blink-FPGA-LED

VHDL
1
star