[NOTE]: This is a highly experimental (and proof of concept) library so do not expect all python packages to work flawlessly. Also, cloud functions are now (Summer 2018) rolling out native support for python3 in EAP so that also might be an option, check out the #functions channel on googlecloud-community.slack.com where the product managers hang around and open to help you out!
cloud-functions-python
py-cloud-fn is a CLI tool that allows you to write and deploy Google cloud functions in pure python, supporting python 2.7 and 3.5
(thanks to @MitalAshok for helping on the code compatibility).
No javascript allowed!
The goal of this library is to be able to let developers write light weight functions
in idiomatic python without needing to worry about node.js. It works OOTB with pip,
just include a file named requirements.txt that is structured like this:
pycloudfn==0.1.206
jsonpickle==0.9.4
as you normally would when building any python application. When building (for production), the library will pick up this file and make sure to install the dependencies. It will do so while caching all dependencies in a virtual environment, to speed up subsequent builds.
TLDR, look at the examples
Run pip install pycloudfn to get it.
You need to have Google cloud SDK installed, as well as
the Cloud functions emulator and npm if you want to
test your function locally.
You also need Docker installed and running as well as the gcloud CLI. Docker is needed to build for the production environment, regardless of you local development environment.
Currently, http, pubsub and bucket events are supported (no firebase).
Usage
CLI
usage: py-cloud-fn [-h] [-p] [-f FILE_NAME] [--python_version {2.7,3.5,3.6}]
function_name {http,pubsub,bucket}
Build a GCP Cloud Function in python.
positional arguments:
function_name the name of your cloud function
{http,pubsub,bucket} the trigger type of your cloud function
optional arguments:
-h, --help show this help message and exit
-p, --production Build function for production environment
-i, --production_image
Docker image to use for building production environment
-f FILE_NAME, --file_name FILE_NAME
The file name of the file you wish to build
--python_version {2.7,3.5}
The python version you are targeting, only applies
when building for production
Usage is meant to be pretty idiomatic:
Run py-cloud-fn <function_name> <trigger_type> to build your finished function.
Run with -h to get some guidance on options. The library will assume that you have a file named main.py if not specified.
The library will create a cloudfn folder wherever it is used, which can safely be put in .gitignore. It contains build files and cache for python packages.
$DJANGO_SETTINGS_MODULE=mysite.settings py-cloud-fn my-function http -f function.py --python_version 3.5
_____ _ _ __
| __ \ | | | | / _|
| |__) | _ ______ ___| | ___ _ _ __| |______| |_ _ __
| ___/ | | |______/ __| |/ _ \| | | |/ _` |______| _| '_ \
| | | |_| | | (__| | (_) | |_| | (_| | | | | | | |
|_| \__, | \___|_|\___/ \__,_|\__,_| |_| |_| |_|
__/ |
|___/
Function: my-function
File: function.py
Trigger: http
Python version: 3.5
Production: False
β ΄ Building, go grab a coffee...
β Generating javascript...
β Ό Cleaning up...
Elapsed time: 37.6s
Output: ./cloudfn/target/index.js
Dependencies
This library works with pip OOTB. Just add your requirements.txt file in the root
of the repo and you are golden. It obviously needs pycloudfn to be present.
Authentication
Since this is not really supported by google, there is one thing that needs to be done to make this work smoothly: You can't use the default clients directly. It's solvable though, just do
from cloudfn.google_account import get_credentials
biquery_client = bigquery.Client(credentials=get_credentials())And everything is taken care off for you!! no more actions need be done.
Handling a http request
Look at the Request object for the structure
from cloudfn.http import handle_http_event, Response
def handle_http(req):
return Response(
status_code=200,
body={'key': 2},
headers={'content-type': 'application/json'},
)
handle_http_event(handle_http)If you don't return anything, or return something different than a cloudfn.http.Response object, the function will return a 200 OK with an empty body. The body can be either a string, list or dictionary, other values will be forced to a string.
Handling http with Flask
Flask is a great framework for building microservices. The library supports flask OOTB. If you need to have some routing / parsing and verification logic in place, flask might be a good fit! Have a look at the example to see how easy it is!
from cloudfn.flask_handler import handle_http_event
from cloudfn.google_account import get_credentials
from flask import Flask, request
from flask.json import jsonify
from google.cloud import bigquery
app = Flask('the-function')
biquery_client = bigquery.Client(credentials=get_credentials())
@app.route('/', methods=['POST', 'GET'])
def hello():
print request.headers
return jsonify(message='Hello world!', json=request.get_json()), 201
@app.route('/lol')
def helloLol():
return 'Hello lol!'
@app.route('/bigquery-datasets', methods=['POST', 'GET'])
def bigquery():
datasets = []
for dataset in biquery_client.list_datasets():
datasets.append(dataset.name)
return jsonify(message='Hello world!', datasets={
'datasets': datasets
}), 201
handle_http_event(app)Handling http with Django
Django is a great framework for building microservices. The library supports django OOTB. Assuming you have setup your django application in a normal fashion, this should be what you need. You need to setup a pretty minimal django application (no database etc) to get it working. It might be a little overkill to squeeze django into a cloud function, but there are some pretty nice features for doing request verification and routing in django using for intance django rest framework.
See the example for how you can handle a http request using django.
from cloudfn.django_handler import handle_http_event
from mysite.wsgi import application
handle_http_event(application)Handling a bucket event
look at the Object for the structure, it follows the convention in the Storage API
from cloudfn.storage import handle_bucket_event
import jsonpickle
def bucket_handler(obj):
print jsonpickle.encode(obj)
handle_bucket_event(bucket_handler)Handling a pubsub message
Look at the Message for the structure, it follows the convention in the Pubsub API
from cloudfn.pubsub import handle_pubsub_event
import jsonpickle
def pubsub_handler(message):
print jsonpickle.encode(message)
handle_pubsub_event(pubsub_handler)Deploying a function
I have previously built go-cloud-fn, in which there is a complete CLI available for you to deploy a function. I did not want to go there now, but rather be concerned about building the function and be super light weight. Deploying a function can be done like this:
(If you have the emulator installed,
just swap gcloud beta functions with npm install && functions and you are golden!).
HTTP
py-cloud-fn my-function http --production && \
cd cloudfn/target && gcloud beta functions deploy my-function \
--trigger-http --stage-bucket <bucket> && cd ../..Storage
py-cloud-fn my-bucket-function bucket -p && cd cloudfn/target && \
gcloud beta functions deploy my-bucket-function --trigger-bucket \
<trigger-bucket> --stage-bucket <stage-bucket> && cd ../..Pubsub
py-cloud-fn my-topic-function bucket -p && cd cloudfn/target && \
gcloud beta functions deploy my-topic-function --trigger-topic <topic> \
--stage-bucket <bucket> && cd ../..Adding support for packages that do not work
- Look at the build output for what might be wrong.
- Look for what modules might be missing.
- Add a line-delimited file for hidden imports and a folder called cloudfn-hooks in the root of your repo, see more at Pyinstaller for how it works. Check out this for how to add hooks.
Troubleshooting
When things blow up, the first thing to try is to delete the cloudfn cache
folder. Things might go a bit haywire when builds are interrupted or other
circumstances. It just might save the day! Please get in touch at twitter if
you bump into anything: @MartinSahlen
License
Copyright Β© 2017 Martin Sahlen
Distributed under the MIT License