• Stars
    star
    42
  • Rank 656,625 (Top 13 %)
  • Language
    Crystal
  • License
    MIT License
  • Created almost 6 years ago
  • Updated almost 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Create and deploy AWS lambdas written in Crystal

crystal-aws-lambda

A small library to simplify running lambdas written in crystallang. Not production ready, just an evening hack!

Installation

You can include this as a dependency in your project in shards.yml file

dependencies:
  lambda_builder:
    github: spinscale/crystal-aws-lambda
    branch: main

Now run the the shards command to download the dependency. You can now create your own lambda handlers like this

require "lambda_builder"

runtime = Lambda::Builder::Runtime.new

runtime.register_handler("httpevent") do |input|
  req = Lambda::Builder::HTTPRequest.new(input)
  user = req.query_params.fetch("hello", "World")
  response = Lambda::Builder::HTTPResponse.new(200, "Hello #{user} from Crystal")
  # not super efficient, serializing to JSON string and then parsing, simplify this
  JSON.parse response.to_json
end

runtime.register_handler("scheduledevent") do |input|
  runtime.logger.debug("Hello from scheduled event, input: #{input}")
  JSON.parse "{}"
end

runtime.register_handler("snsevent") do |input|
  runtime.logger.info("SNSEvent input: #{input}")
  JSON.parse "{}"
end

runtime.run

The input variable is of type JSON::Any and represents the JSON handed over by the lambda event.

There is a helper class to create a HTTP request from the input, no need to do that manually.

Deployment

Make sure the serverless framework is set up properly. The next step is to create a proper serverless configuration file like this

service: crystal-hello-world

provider:
  name: aws
  runtime: provided

package:
  artifact: ./bootstrap.zip

functions:
  httpevent:
    handler: httpevent
    events:
      - http:
          memorySize: 128
          path: hello
          method: get

  snsevent:
    handler: snsevent
    memorySize: 128
    events:
      - sns: my-sns-topic

  scheduledevent:
    handler: scheduledevent
    memorySize: 128
    events:
      - schedule:
          rate: rate(10 minutes)
          input:
            hello: world

If you are using osx, make sure you are building your app using docker, as an AWS lambda runtime environment is based on Linux. You can create a linux binary using docker like this

docker run --rm -it -v $PWD:/app -w /app crystallang/crystal:latest crystal build src/bootstrap.cr -o bin/bootstrap --release --static --no-debug

Now package the zip file required for deployment and deploy

zip -j bootstrap.zip bin/bootstrap
sls deploy

In order to monitor executions you can check the corresponding function logs like this

sls logs -f httpevent -t
sls logs -f snsevent -t
sls logs -f scheduledevent -t

you can also get some very simple metrics per functions (this might require additional permissions)

sls metrics -f httpevent
sls metrics -f snsevent
sls metrics -f scheduledevent

Compiling with shared libraries

While often it's easy to compile with just static linking on a host system like Alpine, sometimes it's more convenient to deploy a dynamically linked binary. To do this, all the required linked dependencies have to be copied into the zip file. The binary itself needs to be modified using patchelf to find them too.

The required libraries can be obtained automatically using ldd bin/bootstrap. An example build and packaging process looks like this:

shards build --release bootstrap
mkdir -p build/lib

# copy the main binary
cp bin/bootstrap build

# copy the libraries
cp $(ldd bin/bootstrap | awk '/ => / {print $3}') build/lib
# copy the loader
cp /lib64/ld-linux-x86-64.so.2 build/lib
# update the library lookup paths to use ./lib rather than the system copies
patchelf --set-interpreter ./lib/ld-linux-x86-64.so.2 --set-rpath ./lib --force-rpath build/bootstrap

# zip it all up for deployment
(cd build && zip -r ../bootstrap.zip *)

Example

If you want to get up and running with an example, run the following commands

git clone https://github.com/spinscale/crystal-aws-lambda
cd crystal-aws-lambda/example
# download dependencies
shards
# built binary (using docker under osx) and creates the zip file
make
# deploy to AWS, requires the serverless tool to be properly set up
sls deploy

This will start a sample runtime, that includes a HTTP endpoint, a scheduled event and an SQS listening event.

Contributing

  1. Fork it (https://github.com/spinscale/crystal-aws-lambda/fork)
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature'), also run bin/ameba
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request
  6. Don't forget to add proper tests, if possible

Contributors

More Repositories

1

elasticsearch-ingest-opennlp

An Elasticsearch ingest processor to do named entity extraction using Apache OpenNLP
Java
269
star
2

elasticsearch-suggest-plugin

Plugin for elasticsearch which uses the lucene FSTSuggester
Java
202
star
3

elasticsearch-opennlp-plugin

Additional opennlp mapping type for elasticsearch in order to perform named entity recognition
Java
136
star
4

dropwizard-blog-sample

A sample dropwizard application using elasticsearch as storage backend
JavaScript
84
star
5

elasticsearch-ecommerce-search-app

A small Micronaut based demo environment to show Elasticsearch as a product search engine
Java
77
star
6

elasticsearch-ingest-langdetect

Ingest processor doing language detection for fields
Java
71
star
7

elasticsearch-graphite-plugin

[UNMAINTAINED] Elasticsearch plugin which pushes data to a graphite server
Java
66
star
8

elasticsearch-facetgrapher

Small hack to draw date histogram facets as graph using nvd3.js
JavaScript
54
star
9

play-cookbook

Source code for most of the recipes featured in the play framework cookbook
Java
53
star
10

elasticsearch-rest-client-samples

Elasticsearch REST client samples using Testcontainers
Java
50
star
11

cookiecutter-elasticsearch-ingest-processor

A cookiecutter template for an elasticsearch ingest processor plugin
Java
47
star
12

elasticsearch-river-streaming-json

A sample elasticsearch river able to stream json data in
Java
38
star
13

link-rating

A sample Spring Boot application to demo the Elastic Stack
Java
20
star
14

alfred-workflow-elastic-docs

An alfred workflow to easily search the elastic documentation
JavaScript
15
star
15

serverless-owntracks-kotlin

A set of serverless AWS lambdas written in Kotlin to store location tracking data from Owntracks
Kotlin
12
star
16

play-ofbiz

Play module to support the OFBiz entity engine, a persistence layer
Java
11
star
17

serverless-reverse-geocoder

Demo using Apache Lucene has a reverse geocoder, running as a CLI app via Graal, AWS Lambda or Google Cloud Run
Java
11
star
18

elasticsearch-facet-georegion

A sample elasticsearch facet implementation which allows to group by geographical region
Java
10
star
19

spark-groovy

Groovy syntactic sugar on top of the spark web framework
Groovy
9
star
20

play-solr

Solr plugin for the playframework
Java
8
star
21

spring-boot-app-search

Spring Boot App Search Demo
Java
7
star
22

maxcube-java

A CLI client for maxcube eq3 devices and a java dead simple libary
Java
6
star
23

seccomp-samples

A few seccomp samples, that can be run in a vagrant VM
Crystal
5
star
24

kibana2json

CLI tool to convert JSON copied from the kibana console back to valid JSON
Crystal
5
star
25

alfred-workflow-elastic-docs.cr

An alfred workflow to easily search the elastic documentation
Crystal
5
star
26

spring-boot-reactive-observability-demo

Spring Boot sample app to demo Elastic Observability
Java
4
star
27

javalin-custom-tailwindcss-example

Create a custom Tailwind CSS build with Gradle in your Java project
HTML
3
star
28

grok.cr

Crystal implementation of a grok
Crystal
3
star
29

play-bidding-sample

A sample bidding application using web sockets
Java
3
star
30

observability-java-samples

Elastic APM Java examples
Java
2
star
31

git-log-to-elasticsearch

A CLI tool to index git repository metadata into Elasticsearch
Crystal
2
star
32

social-network-search-sample

Svelte based SPA example of using Elasticsearch for a social network search
Svelte
1
star
33

elasticsearch-plugin-testcontainers-sample

Using Testcontainers To Test Elasticsearch Plugins
Java
1
star
34

quarkus-logging-ecs

Quarkus Logging extension to output log message in JSON ECS format
Java
1
star
35

javalin-cookie-session-store

A cookie based session store for Javalin
Java
1
star
36

katamari

Polyglot Asynchronous Middleware Framework
Java
1
star
37

jrebel-spark-plugin

JRebel plugin for the spark java webframework
Java
1
star
38

spinscale.github.com

Pages Repository
HTML
1
star
39

javalin-elasticsearch-sample-app

A sample javalin app using the built-in Java HTTP client to connect to Elasticsearch
Java
1
star
40

lead

lead - little elasticsearch alerting deployer
Crystal
1
star
41

talk-elasticsearch-security-manager-and-seccomp

Corresponding demos to my talk about Elasticsearch, its use of the Java Security Manager & seccomp
Java
1
star
42

elastic-stack-meetup-stream

Using the Elastic Stack to visualize the meetup.com reservation stream
Shell
1
star