• This repository has been archived on 05/May/2021
  • Stars
    star
    559
  • Rank 79,109 (Top 2 %)
  • Language
    Python
  • License
    ISC License
  • Created about 11 years ago
  • Updated over 3 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

πŸ’Œ Add webhook subscriptions to your Django app.

Travis CI Build PyPI Download PyPI Status

What are Django REST Hooks?

REST Hooks are fancier versions of webhooks. Traditional webhooks are usually managed manually by the user, but REST Hooks are not! They encourage RESTful access to the hooks (or subscriptions) themselves. Add one, two or 15 hooks for any combination of event and URLs, then get notificatied in real-time by our bundled threaded callback mechanism.

The best part is: by reusing Django's great signals framework, this library is dead simple. Here's how to get started:

  1. Add 'rest_hooks' to installed apps in settings.py.
  2. Define your HOOK_EVENTS in settings.py.
  3. Start sending hooks!

Using our built-in actions, zero work is required to support any basic created, updated, and deleted actions across any Django model. We also allow for custom actions (IE: beyond CRUD) to be simply defined and triggered for any model, as well as truly custom events that let you send arbitrary payloads.

By default, this library will just POST Django's JSON serialization of a model, but you can alternatively provide a serialize_hook method to customize payloads.

Please note: this package does not implement any UI/API code, it only provides a handy framework or reference implementation for which to build upon. If you want to make a Django form or API resource, you'll need to do that yourself (though we've provided some example bits of code below).

Changelog

Version 1.6.0:

Improvements:

  • Default handler of raw_hook_event uses the same logic as other handlers (see "Backwards incompatible changes" for details).

  • Lookup of event_name by model+action_name now has a complexity of O(1) instead of O(len(settings.HOOK_EVENTS))

  • HOOK_CUSTOM_MODEL is now similar to AUTH_USER_MODEL: must be of the form app_label.model_name (for django 1.7+). If old value is of the form app_label.models.model_name then it's automatically adapted.

  • rest_hooks.models.Hook is now really "swappable", so table creation is skipped if you have different settings.HOOK_CUSTOM_MODEL

  • rest_hooks.models.AbstractHook.deliver_hook now accepts a callable as payload_override argument (must accept 2 arguments: hook, instance). This was added to support old behavior of raw_custom_event.

Fixes:

  • HookAdmin.form now honors settings.HOOK_CUSTOM_MODEL

  • event_name determined from action+model is now consistent between runs (see "Backwards incompatible changes")

Backwards incompatible changes:

  • Dropped support for django 1.4
  • Custom HOOK_FINDER-s should accept and handle new argument payload_override. Built-in finder rest_hooks.utls.find_and_fire_hook already does this.
  • If several event names in settings.HOOK_EVENTS share the same 'app_label.model.action' (including 'app_label.model.action+') then django.core.exceptions.ImproperlyConfigured is raised
  • Receiver of raw_hook_event now uses the same logic as receivers of other signals: checks event_name against settings.HOOK_EVENTS, verifies model (if instance is passed), uses HOOK_FINDER. Old behaviour can be achieved by using trust_event_name=True, or instance=None to fire a signal.
  • If you have settings.HOOK_CUSTOM_MODEL of the form different than app_label.models.model_name or app_label.model_name, then it must be changed to app_label.model_name.

Development

Running the tests for Django REST Hooks is very easy, just:

git clone https://github.com/zapier/django-rest-hooks && cd django-rest-hooks

Next, you'll want to make a virtual environment (we recommend using virtualenvwrapper but you could skip this we suppose) and then install dependencies:

mkvirtualenv django-rest-hooks
pip install -r devrequirements.txt

Now you can run the tests!

python runtests.py

Requirements

  • Python 2 or 3 (tested on 2.7, 3.3, 3.4, 3.6)
  • Django 1.5+ (tested on 1.5, 1.6, 1.7, 1.8, 1.9, 1.10, 1.11, 2.0)

Installing & Configuring

We recommend pip to install Django REST Hooks:

pip install django-rest-hooks

Next, you'll need to add rest_hooks to INSTALLED_APPS and configure your HOOK_EVENTS setting:

### settings.py ###

INSTALLED_APPS = (
    # other apps here...
    'rest_hooks',
)

HOOK_EVENTS = {
    # 'any.event.name': 'App.Model.Action' (created/updated/deleted)
    'book.added':       'bookstore.Book.created',
    'book.changed':     'bookstore.Book.updated+',
    'book.removed':     'bookstore.Book.deleted',
    # and custom events, no extra meta data needed
    'book.read':         'bookstore.Book.read',
    'user.logged_in':    None
}

### bookstore/models.py ###

class Book(models.Model):
    # NOTE: it is important to have a user property
    # as we use it to help find and trigger each Hook
    # which is specific to users. If you want a Hook to
    # be triggered for all users, add '+' to built-in Hooks
    # or pass user_override=False for custom_hook events
    user = models.ForeignKey('auth.User', on_delete=models.CASCADE)
    # maybe user is off a related object, so try...
    # user = property(lambda self: self.intermediary.user)

    title = models.CharField(max_length=128)
    pages = models.PositiveIntegerField()
    fiction = models.BooleanField()

    # ... other fields here ...

    def serialize_hook(self, hook):
        # optional, there are serialization defaults
        # we recommend always sending the Hook
        # metadata along for the ride as well
        return {
            'hook': hook.dict(),
            'data': {
                'id': self.id,
                'title': self.title,
                'pages': self.pages,
                'fiction': self.fiction,
                # ... other fields here ...
            }
        }

    def mark_as_read(self):
        # models can also have custom defined events
        from rest_hooks.signals import hook_event
        hook_event.send(
            sender=self.__class__,
            action='read',
            instance=self # the Book object
        )

For the simplest experience, you'll just piggyback off the standard ORM which will handle the basic created, updated and deleted signals & events:

>>> from django.contrib.auth.models import User
>>> from rest_hooks.models import Hook
>>> jrrtolkien = User.objects.create(username='jrrtolkien')
>>> hook = Hook(user=jrrtolkien,
                event='book.added',
                target='http://example.com/target.php')
>>> hook.save()     # creates the hook and stores it for later...
>>> from bookstore.models import Book
>>> book = Book(user=jrrtolkien,
                title='The Two Towers',
                pages=327,
                fiction=True)
>>> book.save()     # fires off 'bookstore.Book.created' hook automatically
...

NOTE: If you try to register an invalid event hook (not listed on HOOK_EVENTS in settings.py) you will get a ValidationError.

Now that the book has been created, http://example.com/target.php will get:

POST http://example.com/target.php \
    -H Content-Type: application/json \
    -d '{"hook": {
           "id":      123,
           "event":   "book.added",
           "target":  "http://example.com/target.php"},
         "data": {
           "title":   "The Two Towers",
           "pages":   327,
           "fiction": true}}'

You can continue the example, triggering two more hooks in a similar method. However, since we have no hooks set up for 'book.changed' or 'book.removed', they wouldn't get triggered anyways.

...
>>> book.title += ': Deluxe Edition'
>>> book.pages = 352
>>> book.save()     # would fire off 'bookstore.Book.updated' hook automatically
>>> book.delete()   # would fire off 'bookstore.Book.deleted' hook automatically

You can also fire custom events with an arbitrary payload:

from rest_hooks.signals import raw_hook_event

user = User.objects.get(id=123)
raw_hook_event.send(
    sender=None,
    event_name='user.logged_in',
    payload={
        'username': user.username,
        'email': user.email,
        'when': datetime.datetime.now().isoformat()
    },
    user=user # required: used to filter Hooks
)

How does it work?

Django has a stellar signals framework, all REST Hooks does is register to receive all post_save (created/updated) and post_delete (deleted) signals. It then filters them down by:

  1. Which App.Model.Action actually have an event registered in settings.HOOK_EVENTS.
  2. After it verifies that a matching event exists, it searches for matching Hooks via the ORM.
  3. Any Hooks that are found for the User/event combination get sent a payload via POST.

How would you interact with it in the real world?

Let's imagine for a second that you've plugged REST Hooks into your API. One could definitely provide a user interface to create hooks themselves via a standard browser & HTML based CRUD interface, but the real magic is when the Hook resource is part of an API.

The basic target functionality is:

POST http://your-app.com/api/hooks?username=me&api_key=abcdef \
    -H Content-Type: application/json \
    -d '{"target":    "http://example.com/target.php",
         "event":     "book.added"}'

Now, whenever a Book is created (either via an ORM, a Django form, admin, etc...), http://example.com/target.php will get:

POST http://example.com/target.php \
    -H Content-Type: application/json \
    -d '{"hook": {
           "id":      123,
           "event":   "book.added",
           "target":  "http://example.com/target.php"},
         "data": {
           "title":   "Structure and Interpretation of Computer Programs",
           "pages":   657,
           "fiction": false}}'

It is important to note that REST Hooks will handle all of this hook callback logic for you automatically.

But you can stop it anytime you like with a simple:

DELETE http://your-app.com/api/hooks/123?username=me&api_key=abcdef

If you already have a REST API, this should be relatively straightforward, but if not, Tastypie is a great choice.

Some reference Tastypie or Django REST framework: + REST Hook code is below.

Tastypie

### resources.py ###

from tastypie.resources import ModelResource
from tastypie.authentication import ApiKeyAuthentication
from tastypie.authorization import Authorization
from rest_hooks.models import Hook

class HookResource(ModelResource):
    def obj_create(self, bundle, request=None, **kwargs):
        return super(HookResource, self).obj_create(bundle,
                                                    request,
                                                    user=request.user)

    def apply_authorization_limits(self, request, object_list):
        return object_list.filter(user=request.user)

    class Meta:
        resource_name = 'hooks'
        queryset = Hook.objects.all()
        authentication = ApiKeyAuthentication()
        authorization = Authorization()
        allowed_methods = ['get', 'post', 'delete']
        fields = ['event', 'target']

### urls.py ###

from tastypie.api import Api

v1_api = Api(api_name='v1')
v1_api.register(HookResource())

urlpatterns = patterns('',
    (r'^api/', include(v1_api.urls)),
)

Django REST framework (3.+)

### serializers.py ###

from django.conf import settings
from rest_framework import serializers, exceptions

from rest_hooks.models import Hook


class HookSerializer(serializers.ModelSerializer):
    def validate_event(self, event):
        if event not in settings.HOOK_EVENTS:
            err_msg = "Unexpected event {}".format(event)
            raise exceptions.ValidationError(detail=err_msg, code=400)
        return event    
    
    class Meta:
        model = Hook
        fields = '__all__'
        read_only_fields = ('user',)

### views.py ###

from rest_framework import viewsets

from rest_hooks.models import Hook

from .serializers import HookSerializer


class HookViewSet(viewsets.ModelViewSet):
    """
    Retrieve, create, update or destroy webhooks.
    """
    queryset = Hook.objects.all()
    model = Hook
    serializer_class = HookSerializer

    def perform_create(self, serializer):
        serializer.save(user=self.request.user)

### urls.py ###

from rest_framework import routers

from . import views

router = routers.SimpleRouter(trailing_slash=False)
router.register(r'webhooks', views.HookViewSet, 'webhook')

urlpatterns = router.urls

Some gotchas:

Instead of doing blocking HTTP requests inside of signals, we've opted for a simple Threading pool that should handle the majority of use cases.

However, if you use Celery, we'd really recommend using a simple task to handle this instead of threads. A quick example:

### settings.py ###

HOOK_DELIVERER = 'path.to.tasks.deliver_hook_wrapper'


### tasks.py ###

from celery.task import Task

import json
import requests


class DeliverHook(Task):
    max_retries = 5

    def run(self, target, payload, instance_id=None, hook_id=None, **kwargs):
        """
        target:     the url to receive the payload.
        payload:    a python primitive data structure
        instance_id:   a possibly None "trigger" instance ID
        hook_id:       the ID of defining Hook object
        """
        try:
            response = requests.post(
                url=target,
                data=json.dumps(payload),
                headers={'Content-Type': 'application/json'}
            )
            if response.status_code >= 500:
                response.raise_for_status()
        except requests.ConnectionError:
            delay_in_seconds = 2 ** self.request.retries
            self.retry(countdown=delay_in_seconds)


def deliver_hook_wrapper(target, payload, instance, hook):
    # instance is None if using custom event, not built-in
    if instance is not None:
        instance_id = instance.id
    else:
        instance_id = None
    # pass ID's not objects because using pickle for objects is a bad thing
    kwargs = dict(target=target, payload=payload,
                  instance_id=instance_id, hook_id=hook.id)
    DeliverHook.apply_async(kwargs=kwargs)

We also don't handle retries or cleanup. Generally, if you get a 410 or a bunch of 4xx or 5xx, you should delete the Hook and let the user know.

Extend the Hook model:

The default Hook model fields can be extended using the AbstractHook model. For example, to add a is_active field on your hooks:

### settings.py ###

HOOK_CUSTOM_MODEL = 'path.to.models.CustomHook'

### models.py ###

from django.db import models
from rest_hooks.models import AbstractHook

class CustomHook(AbstractHook):
    is_active = models.BooleanField(default=True)

The extended CustomHook model can be combined with a the HOOK_FINDER setting for advanced QuerySet filtering.

### settings.py ###

HOOK_FINDER = 'path.to.find_and_fire_hook'

### utils.py ###

from .models import CustomHook

def find_and_fire_hook(event_name, instance, **kwargs):
    filters = {
        'event': event_name,
        'is_active': True,
    }

    hooks = CustomHook.objects.filter(**filters)
    for hook in hooks:
        hook.deliver_hook(instance)

More Repositories

1

django-drip

πŸ’§ Use Django admin to manage drip campaign emails using querysets on Django's User model.
Python
636
star
2

resthooks

A lightweight subscription notification layer on top of your existing REST API
HTML
551
star
3

django-knowledge

Add a help desk or knowledge base to your Django project with only a few lines of boilerplate code.
Python
490
star
4

email-reply-parser

πŸ“§ Email reply parser library for Python
Python
462
star
5

zapier-platform

The SDK for you to build an integration on Zapier
JavaScript
338
star
6

zapier-platform-cli

πŸ’» Build Zapier integrations and test locally using the JavaScript tools you already know.
JavaScript
259
star
7

kubechecks

Check your Kubernetes changes before they hit the cluster
Go
150
star
8

apollo-server-integration-testing

Test helper for writing apollo-server integration tests
TypeScript
134
star
9

Zapier-for-Alfred

An Alfred workflow to trigger Zaps
122
star
10

stripeboard

A simple Django app that collects and displays Stripe data.
JavaScript
119
star
11

prom-aggregation-gateway

An aggregating push gateway for Prometheus
Go
112
star
12

google-yolo-inline

A demo of how to include Google One-tap sign up anywhere on your own site
HTML
90
star
13

node-resthooksdemo

A simple node.js RESTHooks demo built upon the Sails Web Framework
JavaScript
68
star
14

intl-dateformat

Format a date using Intl.DateTimeFormat goodness.
TypeScript
62
star
15

redux-router-kit

Routing tools for React+Redux
JavaScript
51
star
16

visual-builder

Learn how to use Zapier Visual Builder to create new Zapier integrations
HTML
49
star
17

django-rest-framework-jsonmask

Exposes Google Partial Response syntax in Django REST Framework
Python
46
star
18

react-element-portal

Blend React into your exising site by rendering elements inline, targeting an element by id.
JavaScript
34
star
19

zapier-platform-example-app-github

An example GitHub app for the Zapier platform.
JavaScript
27
star
20

tfbuddy

Terraform Cloud pull request alternate workflow
Go
26
star
21

preoomkiller-controller

Preoomkiller Controller evicts pods gracefully before they get OOMKilled by Kubernetes
Go
24
star
22

zapier-platform-core

πŸ”Œ The core Zapier platform library / SDK.
JavaScript
24
star
23

conspiracysanta.com

No longer maintained: A better secret santa for teams.
JavaScript
19
star
24

django-stalefields

No longer updated: Automatic .update() instead of .save() for models.
Python
19
star
25

litdoc

πŸ“ A simple Markdown-based 3-column documentation builder.
HTML
18
star
26

django-birdcage

Utilities for maintaining forwards compatibility with Django releases.
Python
15
star
27

zapier-platform-schema

πŸ“– The core Zapier Platform schema.
JavaScript
15
star
28

jsonmask

Implements Google Partial Response dictionary pruning in Python
Python
14
star
29

resthookdemo

A quick and easy demo application for rest hooks.
CSS
13
star
30

profiling-python-like-a-boss

Sample code for a Zapier engineering blog post
Python
13
star
31

zapier-platform-example-app-onedrive

An example Zapier Platform App that demonstrates a complete App using the OneDrive API
JavaScript
12
star
32

zapier-platform-example-app-rest-hooks

An example Zapier Platform App that demonstrates REST Hook Triggers
JavaScript
10
star
33

saasr

A SaaS subscription simulator - a tool to learn about the statistical mechanics of subscriptions
R
10
star
34

langchain-nla-util

Python
7
star
35

awsjavasdk

Boilerplate rJava Access to the AWS Java SDK
R
6
star
36

zapier-sugarcrm-box

No longer updated: A vagrant box to boot up an instance of sugarcrm for local testing
PHP
6
star
37

docker-brubeck

A simple Dockerfile for GitHub's Brubeck (StatsD-compatible metrics aggregator).
Shell
5
star
38

type-system-benchmarks

Benchmark several type systems and ways to build them (right now Typescript and Flow)
TypeScript
4
star
39

zapier-platform-example-app-files

An example Zapier Platform App that demonstrates file handling (stashFile and dehydrate).
JavaScript
4
star
40

redis-statsd

A simple script which pipes Redis statistics into StatsD
Python
4
star
41

eslint-plugin-zapier

🚿 A shareable version of the .eslintrc file used internally at Zapier.
JavaScript
4
star
42

zapier-platform-example-app-minimal

Zapier CLI Platform Minimal App
JavaScript
4
star
43

zapier-platform-example-app-session-auth

An example Zapier Platform App that demonstrates session authentication (username/password exchange for session key)
JavaScript
4
star
44

zapier-platform-example-app-custom-auth

An example Zapier Platform App that demonstrates custom authentication (API keys)
JavaScript
4
star
45

zapier-platform-example-app-oauth2

An example Zapier Platform App that demonstrates OAuth2
JavaScript
4
star
46

zapier-platform-example-app-trigger

An example Zapier Platform App that demonstrates Triggers
JavaScript
3
star
47

kairos

Python
3
star
48

zapier-platform-example-app-dynamic-dropdown

An example Zapier Platform App that demonstrates Dynamic Dropdowns
JavaScript
3
star
49

zapier-zapier-zapier-zapier-coffee-script

No longer updated: Zapier's very own CoffeeScript version which tweaks for our internal tools.
CoffeeScript
3
star
50

docker-proxysql

ProxySQL Docker images
Shell
2
star
51

zapier-platform-example-app-resource

An example Zapier Platform App that demonstrates Resources
JavaScript
2
star
52

nano-flux

No longer maintained: Tiny, simple flux lib.
JavaScript
2
star
53

zapier-platform-example-app-create

An example Zapier Platform App that demonstrates Creates
JavaScript
2
star
54

is-it-online

Wait for a URL to return 2XX, then show an OS notification and exit.
JavaScript
2
star
55

zapier-platform-example-app-microsoft-exchange

An example Zapier CLI app for MS Exchange
JavaScript
2
star
56

django-linttest

Lint Test is a simple Django app to perform flake8 linting on your project
Python
2
star
57

release-notes

an npm module for pulling merged PRs and generating release notes from them
TypeScript
2
star
58

babel-preset-zapier

🏯 A babel preset for Zapier
JavaScript
1
star
59

parquetr

Read/Write Parquet from R
R
1
star
60

docker-statsd

Run StatsD in a Docker container.
JavaScript
1
star
61

proxysql-benchmark

Benchmark proxysql
Python
1
star
62

docker-graphite

A simple Dockerfile that runs Graphite/Carbon (and nothing else!)
Shell
1
star
63

sentinel-graylog

A simple script which pipes Redis Sentinel messages into Graylog
Python
1
star
64

ecr-catalog-refresh

Simple sidecar to refresh a catalogfile from ECR
Python
1
star
65

zapier-platform-example-app-basic-auth

An example Zapier Platform App that demonstrates Basic Auth
JavaScript
1
star
66

diff-match-patch-cython

A quick hack that speeds up dmp by about ~3x.
Python
1
star
67

terraform-provider-opslevel

Terraform provider for OpsLevel.com
Go
1
star
68

zapier-platform-example-app-search

An example Zapier Platform App that demonstrates Searches
JavaScript
1
star