Contentful AWS S3 and lambda static website generator
Library to generate static websites hosted on Amazon S3 using the Contentful sync API as source of the content and AWS Lambda to generate the static pages. This library also uses internally AWS Kinesis to distribute the page rendering workload.
Synopsis
Contentful is a content management platform for web applications, mobile apps and connected devices. It allows you to create, edit & manage content in the cloud and publish it anywhere via powerful API. Contentful offers tools for managing editorial teams and enabling cooperation.
Amazon S3 is a object storage accessible using the HTTP protocol, it's designed for durability of 99.999999999% of objects.
AWS Lambda is a compute service that runs code in response to events managing automatically compute resources necessary to run the code.
AWS Kinesis is a cloud-based service for real-time processing of large, distributed data streams.
Disclaimer
This project is at proof of concept stage and is built on a best effort kind of approach without any guarantee of correctness, support or reliability.
Architecture
This project uses 2 lambda functions connected via an AWS Kinesis stream in order to render static pages from the entities fetched from the Contentful Delivery sync API.
The first lambda function manages the communication with the Contentful sync api, and writes every single entity to the AWS kinesis stream.
The second lambda function reads the entities from the kinesis stream and per each entity renders a Dot.js and saves the resulting HTML on Amazon S3.
+-----------------+
| |
| lambda |
| contentful sync |
| |
+--------+--------+
|
|
v
+--------+--------+
| |
| kinesis stream |
| |
+-----------------+
|
|
v
+-----------------+
| lambda |
| HTML template |
| rendering |
+-----------------+
|
|
v
+-----------------+
| |
| S3 storage |
| |
+-----------------+
Prerequisites
To use this library is necessary to have an Amazon AWS account and configure the following elements:
- An S3 bucket that will be used to store the static pages and the Synchronization status.
- A Kinesis stream that will be used to distribute the rendering workload.
- An IAM role that will be used by the lambda functions. This role should have the following permissions
- getObject, putObject, putObjectACL on the S3 bucket
- PutRecords on the kinesis stream
- logging permission as described in the AWS HOWTO
- A working installation of the AWS CLI
Configuration
Create a JSON configuration file as follows with the data of the
Contentful space that you want to generate static pages for and the
AWS entities details.
Please note that this library assumes that the source space is the
cfexampleapi
Contentful example space, to work with other spaces is
necessary to adapt the templates in the templates
directory.
{
"aws": {
"region": "us-west-2",
"s3": {
"bucket": "lambda-bucket-test"
},
"kinesis":{
"streamName": "lambda-test-kinesis-stream"
}
},
"contentful": {
"accessToken": "b4c0n73n7fu1",
"space": "cfexampleapi"
}
}
Save the file with the name config.json
in the root directory of this
project.
Create another file to configure the shell wrappers filling in the appropriate AWS details
# the aws region
AWS_REGION=us-west-2
# the aws account number (used to prefix some AWS ARNs)
AWS_ACCOUNT_NUMBER=111111111111
# the name of the role that the lambda function will adopt
LAMBDA_IAM_ROLE=lambda-executor
# the name of the lambda function controlling the Contentful sync API
LAMBDA_CONTENTFUL_SYNC_NAME=contentful-sync-test
# the name of the lambda static rendering function
LAMBDA_KINESIS_RENDERING_NAME=contentful-consumer-test
Save the file with the name config.sh
in the root directory of this
project.
Installation
Install the npm dependencies.
npm install
Create the lambda function to handle the Contentful sync API:
./bin/upload-sync.sh
This will create a zip archive of the project, upload it on AWS Lambda
and it will configure the LAMBDA_CONTENTFUL_SYNC_NAME
function.
Create the lambda function to handle the Kinesis to S3 rendering:
./bin/upload-consumer.sh
This will create a zip archive of the project, upload it on AWS Lambda
and it will configure the LAMBDA_KINESIS_RENDERING_NAME
function.
Bind the LAMBDA_KINESIS_RENDERING_NAME
function to the AWS Kinesis
stream:
./bin/add_source.sh
This will instruct the lambda function to consume the Kinesis stream.
Usage
Trigger execution of the AWS Lambda Contentful sync function
./bin/exec.sh
This should generate in the configured AWS S3 bucket one HTML file per each entry or asset in the original Contentful space. The lambda functions will log in the log section of AWS CloudWatch.
License
Copyright (c) 2015 Contentful GmbH - Paolo Negri. See LICENSE.txt for further details.