Retinal

Serverless Framework-based AWS Lambda function triggered by S3 events to resize images with the excellent Sharp module. By using the Sharp module (which uses the libvips library), image processing can be 3x-5x faster than using ImageMagick, thus reducing the time your function spends running, which can potentially dramatically decrease your lambda function's cost. The function's behaviour can be controlled entirely with configuration.

Contents

1. What is it?
2. Installation
3. Setup
4. Testing
5. Deployment
6. Configuration
7. Building
8. Troubleshooting
9. Change log

What is it?

A tool to take images uploaded to an S3 bucket and produce one or more images of varying sizes, optimizations and other operations all controlled from a simple configuration file. It does this by creating an AWS Lambda function with the help of the Serverless Framework.

Installation

Please note, currently the master branch is broken, please use v0.11.0 instead. See comment.

Installation can be achieved with the following commands

git clone https://github.com/adieuadieu/serverless-sharp-image
cd serverless-sharp-image
yarn install

(It is possible to exchange yarn for npm if yarn is too hipster for your taste. No problem.)

Or, if you have serverless installed globally:

serverless install -u https://github.com/adieuadieu/serverless-sharp-image

Then, modify the config.js and event.json files, adapting them to your needs. More on configuration below.

Setup

Credentials

You must configure your AWS credentials either by defining AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environmental variables, or using an AWS profile. You can read more about this on the Serverless Credentials Guide. It's a bit of a pain in the ass if you have many projects/credentials.

In short, either:

export AWS_PROFILE=<your-profile-name>

or

export AWS_ACCESS_KEY_ID=<your-key-here>
export AWS_SECRET_ACCESS_KEY=<your-secret-key-here>

Testing

Make sure the bucket in config.js exists.

Then:

yarn test

You can also try out the service by invoking it. First deploy it with yarn run deploy and then you can invoke your function with yarn run invoke. This will invoke the function with the test event in event.json. You may need to tweak this file to match your setup.

Deployment

serverless deploy -v

This package bundles a lambda-execution-environment-ready version of the Sharp library which allows you to deploy the lambda function from any OS.

Configuration

The lambda service is designed to be controlled by configuration. From the configuration you can setup how one or more images will be manipulated, with direct access to the underlying methods of Sharp for full control.

module.exports = {
  name: 'serverless-sharp-image',
  provider: {
    profile: 'CH-CH-CH-CHANGEME',
    stage: 'dev',
    region: 'us-east-1',
  },  
  sourceBucket: 'my-sweet-unicorn-media',
  sourcePrefix: 'originals/',
  destinationBucket: 'my-sweet-unicorn-media',
  destinationPrefix: 'web-ready/',
  all: [['rotate'], ['toFormat', 'jpeg', { quality: 80 }]],
  outputs: [
    {
      key: '%(filename)s-200x200.jpg',
      params: {
        ACL: 'public-read',
      },
      operations: [['resize', 200, 200], ['max'], ['withoutEnlargement']],
    },
    {
      key: '%(filename)s-100x100.jpg',
      operations: [['resize', 100, 100], ['max'], ['withoutEnlargement']],
    },
  ],
}

TODO: document configuration better/more detail

all - applied to the image before creating all the outputs

outputs - define the files you wish to generate from the source

• key: uses sprintf internally
• params: set some specific S3 options for the image when uploaded to the destination S3 bucket. See more about the param options on the AWS S3's upload method documentation
• operations: Lists of Sharp's methods you want performed on your image. For example if you want to perform the Sharp method sharp(image).resize(200, 300) you would define this in your configuration as ["resize", 200, 300] Note that method's are performed in order they appear in the configuration, and differing order can produce different results.

Available placeholders for use in the output S3 object's key

• key - The full object key with which the service was invoked

  Example:

  • Given object key: unicorns/and/pixie/sticks/omg.jpg
%(key)s - "unicorns/and/pixie/sticks/omg.jpg"

• type - The Content-Type of the object, as returned by S3

  Example:

  • Given Content-Type: image/jpeg
%(type)s - "image/jpeg"

• crumbs - The crumbs of the S3 object as an array (e.g. the object key split by "/", not including the filename)

  Example:

  • Given object key: unicorns/and/pixie/sticks/omg.jpg
%(crumbs[0])s - "unicorns"
    %(crumbs[2])s - "pixie"

• directory - The "directory" of the S3 object

  Example:

  • Given object key: unicorns/and/pixie/sticks/omg.jpg
%(directory)s - "unicorns/and/pixie/sticks"

• filename - The file name (minus the last extension)

  Example:

  • Given object key: unicorns/and/pixie/sticks/omg.jpg
%(filename)s - "omg"

• extension - The file's extension determined by the Content-Type returned by S3

  Example:

  • Given Content-Type: image/png
%(extension)s - "png"

Building

Although not necessary (it's pre-packaged/included), if you'd like, you can build the sharp module native binaries for Lambda yourself with:

yarn build:sharp

This requires that you have Docker installed and running. More info here.

Troubleshooting

export AWS_CLIENT_TIMEOUT=3000000

Change log

See the CHANGELOG