• Stars
    star
    54
  • Rank 526,738 (Top 11 %)
  • Language
    Crystal
  • License
    MIT License
  • Created about 6 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

gcf.cr provides serverless execution and deployment of crystal language code in Google Cloud Functions

gcf.cr

CircleCI

GCF provides managed execution of crystal language code within Google Cloud Functions. GCF compiles your crystal code statically using the crystallang/crystal docker image or optionally using your local crystal installation (if it is capable of static compilation) via the --local option. It then bundles your compiled crystal code in a thin node.js wrapper function and deploys it to GCP using the options you specify. An API is also provided for writing to the console, throwing errors, and returning the final response.

Installation

  1. set up a Google Cloud Platform account if you don't have one already and create an initial project
  2. install the gcloud sdk if you haven't already
  3. log in to gcloud locally via gcloud init if you haven't already
  4. install docker (if you haven't already)
  5. set up docker to not require sudo
  6. start the docker daemon (e.g. sudo systemctl start docker) if it isn't already running
  7. clone the repo git clone [email protected]:sam0x17/gcf.cr.git
  8. run ./setup. This will compile and install a gcf binary in /usr/bin.

If you plan to use docker-based static compilation (default option), you don't need to install crystal on your system as long as you have a statically compiled gcf binary. You can use the build_static script included in the repo to build a static binary for gcf using docker. That said, having crystal locally installed will make it easier to write tests.

Getting Started

All cloud functions should consist of a crystal app project (created via crystal init app) where the main project file (e.g src/my_project.cr) meets the requirements outlined below.

Add the following to your shard.yml file and run shards install:

# shard.yml
...
dependencies:
  gcf:
    github: sam0x17/gcf.cr
    branch: master

Create a class that inherits from GCF::CloudFunction and defines a run method that accepts an argument of type JSON::Any, typically named params:

# src/example.cr
require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    # your code here
  end
end

Once you have a setup like this, you can use the various API functions from within your run method, which makes up the body of your cloud function. The available API functions are listed below. Note that methods like send, send_file, and redirect stop execution when they are run, meaning any code after these methods will not run unless it was already defined in an at_exit block. If you do not call one of these methods in your function, your function will run until it times out.

Once you are done writing your function, you can deploy it using gcf --deploy.

Crystal API

A crystal-based API is provided for communicating with the Google Cloud Function host process so you can do things like log to the console redirect the browser, or send textual or file-based data to the browser. The API is a thin layer on top of the underlying ExpressJS API used by Google Cloud Functions, and uses a combination of inter-process communication and files to send data to/from the host process.

The most basic requirement, as outlined in the getting started section, is that you create a class that inherits from GCF::CloudFunction and provide a definition for the run(params : JSON::Any) method.

params

The run method of your crystal function will provide the HTTP params object in the form of a JSON::Any object named params. You can access values in the params object as you would a hash:

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    console.log params
    send "color: #{params["color"]?}"
  end
end

If you send the http parameter color with the value of red, the function will return "color: red" with a 200 status code and you will get the following console output: console image

console.log(msg)

Logs whatever you pass it to the GCF console with info priority. Equivalent to using console.log in JavaScript. msg is interpolated so non-strings may be passed in.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    console.log "some info here"
  end
end

console.warn(msg)

Logs whatever you pass it to the GCF console with warn priority. Equivalent to using console.warn in JavaScript. msg is interpolated so non-strings may be passed in.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    console.warn "woah, warning"
  end
end

console.error(msg)

Logs whatever you pass it to the GCF console with error priority. Equivalent to using console.error in JavaScript. msg is interpolated so non-strings may be passed in.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    console.error "GASP! an error"
  end
end

send(content)

An alias for send(200, content), where 200 is the HTTP OK/ready status code.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    send "OK, done executing"
  end
end

send(status : Int, content)

Sends the interpolated version of content as output to the browser with an HTTP status code of status, and stops execution of the cloud function. This is forwarded to req.send in ExpressJS.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    send 200, "<h1>YO</h1>"
  end
end

redirect(url : String)

An alias for redirect false, url, since temporary redirects are usually preferable when used with cloud functions.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    redirect "https://google.com"
  end
end

redirect(permanent : Bool, url : String)

Redirects the browser to the specified url. If permanent is true, it will do a 301 redirect. If permanent is false, it will do a 302 redirect.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    redirect true, "https://google.com"
  end
end

send_file(path : String)

An alias for send_file 200, path, since typically you will only want to send file content with a status code of 200.

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    send_file "van_gogh.jpg"
  end
end

send_file(status : Int, path : String)

Sends the file at the specified path to the browser with an HTTP status code of status (to write files from crystal in a cloud function, you need to write to something in /tmp).

require "gcf"

class Example < GCF::CloudFunction
  def run(params : JSON::Any)
    send_file 200, "van_gogh.jpg"
  end
end

Note on puts

If you call puts directly from within a cloud function's run method, this gets mapped to console.log. This does not apply to puts calls that are made indirectly (e.g. calling code outside of this class), so the contents of these puts calls will not be handled correctly and lead to undefined behavior.

Note on exceptions

Right now exceptions work locally but not in a deployed function. We are working on this but please feel free to take a look at #1 and help out if you have any ideas. From what we can tell, all execution stops the second an exception is thrown, even from within a try-catch block, when a function is executing on GCP. For now we are logging a generic message stating that an error occurred, however we are unable to retrieve the error stacktrace or name (locally we are able to do this).

Tests / Specs

You can test your cloud function with specs the same way you would any conventional crystal-based app or library. See gcf_spec.cr for examples of how to test for particular functions outputs, redirects, etc. Also make sure to set GCF.test_mode = true in your spec_helper.cr file before any specs are loaded or the development server will attempt to run before your specs can run.

Example spec_helper.cr file:

# spec/spec_helper.cr
GCF.test_mode = true

require "spec"
require "../src/my_project.cr"

Development Mode / Test Server

A built-in test server is provided (based on Kemal) that allows you to simulate requests to your (HTTP-triggered) cloud functions on your local machine. Simply compile and run your cloud function e.g. crystal run src/*.cr and Kemal will automatically start the test server on port 8080. You can access it in a web browser by going to http://localhost:8080/, which will trigger your cloud function and load the result as if you just visited the HTTP trigger URL for the function. Note that while send_file/send/redirect methods normally stop execution of your function once they execute, this is not the case in the test server, though this is something we are trying to find a workaround for.

GET and POST params that are sent to the test server are automatically loaded into the params of your cloud function when it is invoked.

gcf_test$ crystal run src/gcf_test.cr
[development] Kemal is ready to lead at http://0.0.0.0:8080
2018-07-27 04:04:06 -04:00 200 GET / 375.0ยตs
{"color" => "red"}
2018-07-27 04:04:13 -04:00 200 GET /?color=red 1.41ms

Deploying

Note that GCF expects your crystal function to follow the directory structure imposed by crystal init app, in that all of your crystal code should reside in project_name/src/. During compilation, GCF uses the src/*.cr glob to compile all crystal files in the src directory.

Note also that GCF will automatically consult gcloud to discover the current GCP project id if one isn't specified.

Below you can find some basic usage examples fo rcommon use cases. For full usage information, please see the output of gcf --help.

Compile the current directory using the docker image and deploy as a function named after the current directory (default):

gcf --deploy

Specifying the source directory, static compilation using the local crystal installation, the function name, the memory capacity of the deployed function, and the google project ID respectively.

gcf --deploy --source /home/sam/proj --local --name hello-world --memory 2GB --project cool-project

Or using shorthand:

gcf -d -s /home/sam/proj -l -n hello-world -m 2GB -p cool-project

TODO

  1. attribute API so we don't need command line params
  2. fix exceptions logging bug (#1)
  3. more testing

More Repositories

1

nginx-android-app

android app version of nginx
C
55
star
2

mongo_orm

Mongo ORM: A simple ORM for using MongoDB with the crystal programming language, designed for use with Amber. Based loosely on Granite ORM. Supports Rails-esque models, associations and embedded documents.
Crystal
32
star
3

DSeg

Invariant Superpixel Features for Object Detection
C++
18
star
4

macro_state

A simple set of Rust proc macros for reading and writing global compile-time state between macro calls
Rust
18
star
5

routes_classic

Routes addon but compatible with WoW Classic
Lua
17
star
6

macro_magic

A collection of Rust proc macros that allow the exporting and importing of TokenStream2s of items in foreign contexts and files
Rust
15
star
7

supertrait

A Rust crate that uses macro hackery to enable const fn trait items and default associated types on traits in stable Rust.
Rust
14
star
8

crystal-alpine

Dockerfile for alpine linux with crystal that will allow you to compile static crystal binaries on any system
Shell
12
star
9

html-minifier

A zero-dependency HTML/CSS/Javascript minifier for the Crystal language
Crystal
10
star
10

assert.cr

Provides C++-like assert functionality that you can sprinkle throughout your crystal program. Assertions are disabled via macro when the `--release` compile-time flag is specified, allowing performant release builds.
Crystal
7
star
11

docify

Allows for dynamic compile-time embedding of existing tests and examples in your Rust doc comments and markdown files
Rust
7
star
12

tempdir

Simple creation and automatic deletion of temporary directories in an easy to use crystal shard
Crystal
5
star
13

browser_piano

A simple 100% pure javascript browser-based piano demo featuring multiple sound channels. Can be played with computer keyboard.
JavaScript
4
star
14

conduit

A batteries-included vanilla js frontend framework with client-side routing for creating cloud storage hosted, search engine friendly SPAs and web apps that interact with an external API server. The ultimate serverless frontend framework.
Crystal
4
star
15

crystal-gcf-concept

Proof of concept of crystal running in a Google Cloud Function
JavaScript
4
star
16

bettercast

Cast from anything, to anything, no strings attached
Java
3
star
17

magnetfilter

a python utility for adding the best current trackers to a magnet link
Python
3
star
18

css-minifier

Embeds the venerable clean-css from npm within a crystal shard via Duktape
Crystal
3
star
19

video-manager

a tool for automatically optimizing files in your video library using ffmpeg
Crystal
3
star
20

js-minifier

A javascript minifier for crystal, using embedded uglifier-js and duktape.cr
JavaScript
2
star
21

truthy

Adds intelligent to_b (to_boolen) to all objects in crystal
Crystal
2
star
22

s3cmd-shim

Drop-in docker-based CLI replacement for s3cmd for systems (like macos) where s3cmd doesn't work properly
Shell
2
star
23

hierarch_old

Hierarch: A new, blazingly fast, in-memory proof-of-concept data structure and indexing algorithm for querying on dynamic attribute/tag/type-based hierarchical data.
C++
2
star
24

es6-minifier

A self-contained crystal shard that can minify ES6+ JavaScript.
Shell
2
star
25

bolts

A vaguely rails-like fast web framework for Rust
Rust
2
star
26

chiron

Replace your entire webpack HTML/CSS/JavaScript/LESS compilation, minification, and deployment pipeline with a single binary
Crystal
2
star
27

quoth

A developer-friendly parsing library for implementing domain specific languages and syntax parsers in Rust
1
star
28

macro_magic_example

Rust
1
star
29

derive_parse2

Intended as a spiritual successor to derive_syn_parse, supports all of the original features plus some extra features. Fully compatible with syn 2.x, and implemented entirely in thoroughly tested proc macros
Rust
1
star
30

raylib.cr

Crystal
1
star
31

sin

Sin aims to be an alternative to the proc-macro2/syn ecosystem geared towards developer UX and custom syntax parsing.
Rust
1
star
32

pub-fields

A simple proc macro that makes all fields of a struct public
Rust
1
star
33

lake

A simple connection pooling shard for the crystal language suitable for use with Redis and probably other things
Crystal
1
star
34

domain-db

crystal langauge shard containing a dynamically updated database of public domain suffixes and top level domain extensions
Crystal
1
star
35

secrets.cr

safe environment-specific secret storing and loading for crystal language apps and libraries
Crystal
1
star
36

aes.cr

crystal wrapper for OpenSSL AES CBC mode encryption routines
Crystal
1
star
37

ROSE-NodeFinder

A library for ROSE that can index an AST and provide O(1) access to all nodes of specified types that are the descendants of specified nodes without excessive memory overhead. In other words, NodeFinder answers queries of the form "given node A, return all descendants of A that are of type T" in constant time without wasting memory.
C++
1
star