• Stars
    star
    127
  • Rank 282,790 (Top 6 %)
  • Language
    R
  • License
    Other
  • Created about 3 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Run R containers on AWS Lambda

lambdr

R-CMD-check CRAN status CRAN downloads Last commit Codecov test coverage license

This package provides an R runtime for the AWS Lambda serverless compute service. It is intended to be used to create containers that can run on AWS Lambda. lambdr provides the necessary functionality for handling the various endpoints required for accepting new input and sending responses.

This package is unofficial. Its creators are not affiliated with Amazon Web Services, nor is its content endorsed by Amazon Web Services. Lambda, API Gateway, EventBridge, CloudWatch, and SNS are services of Amazon Web Services.

The default behaviour is to convert the body of the received event from JSON into arguments for the handler function using the jsonlite package. For example, a raw event body of {"number": 9} will be converted to list(number = 9). The handler function will then receive the arguments directly after unlisting, eg. number = 9. This works for direct invocations, as well as situations where the user wishes to implement behaviour specific to a trigger.

Some invocation types have their own logic for converting the event body into an R object. This is useful for say, using an R function in a Lambda behind an API Gateway, so that the R function does not need to deal with the HTML elements of the invocation. The below invocation types have custom logic implemented. Refer to the vignettes or the package website for more information.

Alternatively, user-defined functions can be provided for parsing event content and serialising results. The user can also use the identity function as a deserialiser to pass the raw event content --- as a string --- to the handler function. Refer to ?lambda_config for more information.

invocation type implementation stage
direct
API Gateway (REST)
API Gateway (HTML)
EventBridge
SNS

Installation

When the package is made available on CRAN it can be installed with:

install.packages("lambdr")

The development version is available with:

remotes::install_github("mdneuzerling/lambdr")

Running

In a runtime.R file, source all functions needed and then run:

lambdr::start_lambda()

This runtime.R file should be executed by the Docker image containing your Lambda code.

The lambdr::start_lambda() function relies on environment variables configured by AWS. It will fail if run locally. In particular, the handler as configured by the user through AWS will determine which function handles the Lambda events. For debugging and testing, values can be provided to the function in the absence of environment variables. See ?lambdr::lambda_config for details.

Example

Consider the following runtime.R file:

parity <- function(number) {
  list(parity = if (as.integer(number) %% 2 == 0) "even" else "odd")
}

lambdr::start_lambda()

The parity function accepts a number argument and returns its parity as a named list, for example:

parity(5)
# $parity
# [1] "odd"


parity(8)
# $parity
# [1] "even"

This function can then be placed into a Docker image. An example is provided below, but the key components are:

  • Start from the public.ecr.aws/lambda/provided parent image, which provides the basic components necessary to serve a Lambda
  • Install R and dependencies, both system dependencies and R packages, including the lambdr package
  • Copy across runtime.R and any other necessary files
  • Generate a simple bootstrap which runs runtime.R with R
  • Set the handler as the CMD. The lambdr package interprets the handler as the name of the function to use, in this case, "parity". The CMD can also be set (or overriden) when setting up the Lambda in AWS.
FROM public.ecr.aws/lambda/provided

ENV R_VERSION=4.0.3

RUN yum -y install wget git tar

RUN yum -y install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm \
  && wget https://cdn.rstudio.com/r/centos-7/pkgs/R-${R_VERSION}-1-1.x86_64.rpm \
  && yum -y install R-${R_VERSION}-1-1.x86_64.rpm \
  && rm R-${R_VERSION}-1-1.x86_64.rpm

ENV PATH="${PATH}:/opt/R/${R_VERSION}/bin/"

# System requirements for R packages
RUN yum -y install openssl-devel

RUN Rscript -e "install.packages(c('httr', 'jsonlite', 'logger', 'remotes'), repos = 'https://packagemanager.rstudio.com/all/__linux__/centos7/latest')"
RUN Rscript -e "remotes::install_github('mdneuzerling/lambdr')"

RUN mkdir /lambda
COPY runtime.R /lambda
RUN chmod 755 -R /lambda

RUN printf '#!/bin/sh\ncd /lambda\nRscript runtime.R' > /var/runtime/bootstrap \
  && chmod +x /var/runtime/bootstrap

CMD ["parity"]

The image is built and uploaded to AWS Elastic Container Registry (ECR). First, a repository is created:

aws ecr create-repository --repository-name parity-lambda --image-scanning-configuration scanOnPush=true

This provides a URI, the resource identifier of the created repository. The image can now be pushed:

docker tag mdneuzerling/r-on-lambda:latest {URI}/parity-lambda:latest
aws ecr get-login-password | docker login --username AWS --password-stdin {URI}
docker push {URI}/parity-lambda:latest

In either the AWS console or the command line, a Lambda can be created from this image. Call the Lambda "parity" to match the function name. Tests can be executed within the console. Alternatively the Lambda can be invoked from the CLI:

aws lambda invoke --function-name parity \
  --invocation-type RequestResponse --payload '{"number": 8}' \
  /tmp/response.json --cli-binary-format raw-in-base64-out

The output is now available in the generated file:

cat /tmp/response.json            
{"parity":"even"}

Hex logo by Phizz Leeder

More Repositories

1

getsysreqs

Determine system requirements from R packages using the RStudio Package Manager. This is a weekend project, not a real package, so please think twice before using it for anything serious.
R
45
star
2

r-on-lambda

An attempt to get an R runtime and function working on AWS Lambda using a container.
R
32
star
3

ptvapi

Unofficial R wrapper for the Public Transport Victoria (PTV) timetable API
R
17
star
4

DrakeModelling

Combining drake workflows with R package development to train and execute a machine learning model
R
14
star
5

RDCOMOutlook

Wrapper for programmatic access to Microsoft Outlook on Windows through RDCOMClient
R
14
star
6

NLPRMetaflow

Using the R Metaflow bindings to make NLP model tuning a little less painful
R
13
star
7

ReviewSentiment

An end-to-end data science pipeline using just R and GitHub Actions
R
11
star
8

exemplar

Generate validation functions to make sure one object looks like another (its exemplar)
R
11
star
9

plumber-on-k8s

Run a `plumber` API on Kubernetes
Dockerfile
6
star
10

starfleet

Configuration files for my personal Kubernetes (k3s) cluster
Python
6
star
11

diamonds-via-lambda

Serverless on-demand R markdown reports in the browser
Dockerfile
5
star
12

advent_of_code

My personal solutions to the Advent of Code puzzles
Julia
3
star
13

usedocker

This is an abandoned package that I intended to use for simplifying the creation of Dockerfiles for R projects. Please don't use this package. Use dockyard instead.
R
3
star
14

monstr

Dungeons and Dragons Tools and Data for R
R
3
star
15

mdneuzerling.com

Personal website and blog, created with R markdown, hugodown, and hugo.
HTML
3
star
16

shiny_pub_time

My friends and I couldn't decide when to go to the pub. https://shiny.mdneuzerling.com/pub_time
R
2
star
17

OutlookWithR

R script that sends an email with Outlook with a plot as an inline image
R
2
star
18

aphhansard

R
2
star
19

praiseme

A package to deliver praise because sometimes that's what we need
R
2
star
20

melburn-wine

Presentation given to the Melbourne Users of R Network on 2019-04-10
HTML
2
star
21

shiny_wine

Simple shiny app for producing a scatterplot, with variable selection from drop-down menus. Data from UCI Machine Learning Repository (https://archive.ics.uci.edu/ml/datasets/Wine)
R
2
star
22

neuzr

Personal Package of Miscellaneous R Functions
R
1
star
23

AtomicAlgebra

For exploring nonassociative algebras and relation algebras, including the generation of all nonassociative algebras on four atoms.
Python
1
star
24

RAreducts

Generates the signatures that are reducts of the signature of Tarski's relation algebras
Python
1
star
25

useR2018_recap

Slides for a presentation I gave to some data scientists on 2018-08-17
HTML
1
star
26

test-get-pins

Basic Lambda for returning content of a pin in a hardcoded S3 bucket
Dockerfile
1
star
27

ModelAsAPackage

Concept for creating an R model as a package, with execution separate from training
R
1
star