• Stars
    star
    282
  • Rank 146,549 (Top 3 %)
  • Language
    Erlang
  • License
    BSD 3-Clause "New...
  • Created over 10 years ago
  • Updated almost 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

RabbitMQ Exchange that publishes messages received from PostgreSQL Notifications.

PostgreSQL LISTEN Exchange

A RabbitMQ exchange type that translates PostgreSQL NOTIFY messages to AMQP messages and publishes them to bound queues. The PostgreSQL NOTIFY message channel is used as the routing key for the message using direct exchange style routing mechanics.

Build Status

Example

To publish Postgres notifications as AMQP messages into queues bound to a x-pgsql-listen exchange with a binding key of test, run the following command in psql:

postgres=# NOTIFY test, 'This is a test';

Installation

Extract the contents of the release zip file into your RabbitMQ plugins directory. Once extracted, run rabbitmq-plugins enable pgsql_listen_exchange.

Configuration

Argument Based Configuration

To connect to PostgreSQL using something other than the default pgsql://postgres@localhost:5432/postgres connection, you can add arguments when declaring the exchange:

Setting Description Data Type
x-host The PostgreSQL server hostname String
x-port The port to connect on Number
x-dbname The database name to connect to String
x-user The user to connect as String
x-password The password to use when connecting String

Policy Based Configuration

To apply configuration via a policy, the following settings are available:

Setting Description Data Type
pgsql-listen-host The PostgreSQL server hostname String
pgsql-listen-port The port to connect on Number
pgsql-listen-dbname The database name to connect to String
pgsql-listen-user The user to connect as String
pgsql-listen-password The password to use when connecting String

Configuration in rabbitmq.config

You can also change the default connection values in the rabbitmq.config file:

Setting Description Data Type Default Value
host The PostgreSQL server hostname list "localhost"
port The port to connect on integer 5432
dbname The database name to connect to list "postgres"
user The user to connect as list "postgres"
password The password to use when connecting list ""

Example rabbitmq.config

[{pgsql_listen_exchange,
  [
    {host, "localhost"},
    {port, 5432},
    {dbname, "postgres"},
    {user, "postgres"},
    {password, ""}
  ]}
].

Message Properties

The exchange will automatically add the following properties to messages:

Property Value
app_id pgsql-listen-exchange
headers See "Headers Properties Values" table below
timestamp The UNIX epoch timestamp of the publishing server

Headers Property Values

The following table details the values of the headers property that is set on each message.

Key Value
pgsql-channel The PostgreSQL notification channel
pgsql-server The host and port of the PostgreSQL server
source-exchange The pgsql-listen-exchange that the notification was received by

Specifying Other Properties

In addition to the automatically set message properties, the exchange can set configured message properties. To set one of the supported message properties, specify the property name and value when binding to the exchange. For example, to set the content_type property, specify content_type and the value it should be set to when binding a queue to the exchange. The following message properties are supported:

Property Data Type
content_encoding String
content_type String
delivery_mode Number
priority Number
reply_to String
type String

Building

Steps to custom build a version of the pgsql-listen-exchange plugin follow the development environment instructions at https://www.rabbitmq.com/build-server.html#prerequisites and then run the following:

make tests
make dist
zip -r pgsql-listen-exchange.zip plugins/epgsql-* plugins/pgsql-listen-exchange-*

Unzip that file into the plugins directory for RabbitMQ and enable as you would any other plugin.

More Repositories

1

consulate

Python client for the Consul HTTP API
Python
342
star
2

queries

PostgreSQL database access simplified
Python
256
star
3

rabbitpy

A pure python, thread-safe, minimalistic and pythonic RabbitMQ client library
Python
242
star
4

tinman

Tinman is a Tornado support package including an application wrapper/runner and a set of handy decorators.
Python
187
star
5

RabbitMQ-in-Depth

Examples and materials for RabbitMQ in Depth
Python
133
star
6

flatdict

Python module for interacting with nested dicts as a single level dict with delimited keys.
Python
111
star
7

env-aws-params

Inject AWS SSM Parameters as Environment Variables
Go
86
star
8

rejected

rejected is a consumer framework for RabbitMQ
Python
58
star
9

pamqp

Low level AMQP frame encoding and decoding library
Python
51
star
10

rabbitmq-flume-plugin

A Flume plugin that provides a RabbitMQ Source and Sink
Java
48
star
11

pycon2013-logging

Examples from my "Become a Logging Expert in 30 Minutes" talk at PyCon 2013
Python
48
star
12

glyphicons-sprite-generator

The GLYPHICONS Sprite Generator allows you to change the size and colors of the Bootstap icons using the Free or Pro versions of GLYPHICONS.
Python
47
star
13

rabbitmq-pulse

rabbitmq-pulse is a RabbitMQ plugin that publishes node and queue information, pushing stats instead of polling the Management API
Erlang
45
star
14

tornado-elasticsearch

Extends the official Elasticsearch Python API adding Tornado AsyncHTTPClient support
Python
38
star
15

influxdb-storage-exchange

Post JSON structured event messages to InfluxDB
Erlang
33
star
16

alpine-rabbitmq-autocluster

RabbitMQ image with the autocluster plugin
32
star
17

email-normalize

Return a normalized email-address stripping ISP specific behaviors
Python
30
star
18

pgdumplib

Python3 library for reading and writing pg_dump files using the custom format
Python
30
star
19

On-Rabbits-and-Elephants

Code for my lightning talk at pgCon 2011
Python
29
star
20

helper

Development library for quickly writing configurable applications and daemons.
Python
26
star
21

statelessd

statelessd is a stateless HTTP to AMQP publishing gateway prototype in Python
Python
25
star
22

kvpbench

Key-Value Store Benchmark Application
Python
24
star
23

srvlookup

A small wrapper for dnspython to return SRV records for a given host, protocol, and domain name as a list of namedtuples
Python
23
star
24

passport

Passport generates templated configuration files using Consul's Service Discovery and Key/Value database tools
Python
22
star
25

aiorabbit

An AsyncIO RabbitMQ client for Python 3
Python
21
star
26

tredis

An asynchronous Redis client for Tornado
Python
20
star
27

hockeyapp

Python Client for the HockeyApp.net API
Python
19
star
28

mredis

MRedis is a multi-server wrapper for the excellent Python Redis client.
Python
19
star
29

progrock

A multi-progressbar implementation to complement multiprocessing.Process.
Python
18
star
30

pg-statsd

pg_statsd is a set of PostgreSQL user-defined functions that provide and interface to statsd.
C
15
star
31

strftimerl

Erlang implementation of strftime
Erlang
13
star
32

urilib

A RFC-3986 URI Library for parsing and building URIs
Erlang
13
star
33

infoblox

A python library for interfacing with Infoblox NIOS.
Python
12
star
34

mikkoo

Mikkoo is a PgQ to RabbitMQ Relay
Python
12
star
35

huesos-de-vaquero

A skeleton project for web development using Cowboy (Erlang) for back-end development and Google Web-Starter-Kit for front-end development
Erlang
12
star
36

tornado-aws

A low-level Amazon Web Services API client for Tornado
Python
12
star
37

houston

Easy docker stack deployment to CoreOS clusters using Fleet and Consul
Python
11
star
38

framewerk

Framewerk is a PHP5 OO Framework
PHP
10
star
39

alpine-pgbouncer

Minimalistic Alpine Linux pgBouncer container
Shell
8
star
40

httpbl

Python client library for the project-honeypot.org Http:BL API
Python
8
star
41

cardiff

Cardiff is an extendible statsd clone written in Python that supports all statsd metric types and a variety of configurable stats destinations.
Python
8
star
42

fluent-plugin-json-in-json

Fluentd parser plugin that parses JSON attributes with JSON strings in them
Ruby
7
star
43

erlln

Example Erlang OTP application that listens for PostgreSQL NOTIFY messages
Erlang
7
star
44

pgpasslib

Library for getting passwords from a PostgreSQL Password file
Python
7
star
45

conserl

Erlang client for Consul
Erlang
6
star
46

mgmt-stress

A RabbitMQ stress-testing tool
Erlang
6
star
47

privatepaste

v2 of PrivatePaste.com
Erlang
5
star
48

python-tornado.tmbundle

A TextMate 2 Bundle for Tornado
5
star
49

ossicons

An icon font for popular Open Source project logos
HTML
5
star
50

alpine-postgres

gavinmroy/alpine-postgres is a Docker image with a small disk footprint for PostgreSQL
Shell
5
star
51

dynamodb-backup

CLI utility for backing up DynamoDB to Avro containers
Python
4
star
52

pgsql-utilities

A collection of PostgreSQL utilities
PHP
4
star
53

logging-config

A wrapper class for the Python standard logging module
Python
4
star
54

fleetpy

An opinionated fleet API client for Python
Python
4
star
55

rabbitmq-sns-plugin

RabbitMQ plugin for AWS SNS interoperability
Makefile
4
star
56

golconde

Golconde (gŏl-kŏn'də) is a queue based data-distribution solution for PostgreSQL written in Python 2.6.
Python
4
star
57

github-issue-autoresponder

Check GitHub projects periodically for new issues and automatically comment or label them based upon string matching rules
Python
4
star
58

rabbitstew

A command-line tool for publishing messages to RabbitMQ
Python
4
star
59

couchclient

A light-weight, read-only CouchDB client
Python
4
star
60

pglifecycle

A PostgreSQL schema management tool
Python
3
star
61

rmq-node-report

A small set of scripts to gather information from a local RabbitMQ node
Erlang
3
star
62

pgparse

Python wrapper for libpg_query
Python
3
star
63

alpine-rabbitmq

RabbitMQ image with a small disk footprint
3
star
64

PyCon-2012-Tornado-IOLoop-Talk-Examples

Examples from my 2012 PyCon talk: "More than just a pretty web framework, the Tornado IOLoop"
Python
3
star
65

menwith

Menwith is a memcache text protocol command utilization analysis tool.
Python
3
star
66

elmer

Elmer allows granular monitoring of RabbitMQ queues with Nagios and other monitoring tools using custom queue arguments.
Erlang
3
star
67

http-exchange

RabbitMQ exchange for stateless publishing over HTTP
Erlang
3
star
68

alpine-statsd

A statsd container with a small disk footprint
3
star
69

boxcar

An experimental OS/X app for interfacing with Vagrant
Objective-C
3
star
70

rmq-cluster-rebalance

CLI application for rebalancing queues in a RabbitMQ cluster
Python
3
star
71

rabbitmq-in-depth-cookbook

Cookbook used by Vagrant when setting up the RabbitMQ in Depth VM
Python
2
star
72

httpc-aws

A light-weight, relatively unopinionated AWS API client for Erlang
Erlang
2
star
73

cowboy-bones-gutenberg-generator

An #Erlang Cowboy web app Gutenberg generator with i18n, templating, and more
Erlang
2
star
74

tdns

An asynchronous Tornado pycares DNS client wrapper
Python
2
star
75

rabbitmq-management-ui-prototype

A ReactJS RabbitMQ Management UI Prototype
TypeScript
2
star
76

pgdumpreport

Create reports from pg_dump archives
JavaScript
2
star
77

hashicorp-packaging

Makefile and templates for creating deb packages for HashiCorp (and ecosystem) projects.
Makefile
2
star
78

formulary

Easy management of AWS Cloud Formation stacks
Python
2
star
79

alpine-redis

Redis image with a small disk footprint
Dockerfile
2
star
80

ecs-pipeline-deploy

An opinionated deployment application for ECS services
Python
2
star
81

rabbitmq-pagerduty

A RabbitMQ monitoring plugin for triggering PagerDuty alerts
Erlang
2
star
82

pg-graph-dump

Dependency graph based dump and reload of PostgreSQL DDL
Erlang
2
star
83

ppp-loan-data

PPP FOIA Data
2
star
84

mdnserl

Erlang library for providing RFC-6762 multicast-DNS announcement and discovery
Erlang
1
star
85

rmq-definitions

Deterministically sort and format RabbitMQ definition backups
Python
1
star
86

telegraf-pgbouncer

pgBouncer Stats Collector for Telegraf
Python
1
star
87

ssm-ps-template

Command line application to render templates with data from SSM Parameter Store
Python
1
star
88

s3-config-overlay

Fetch config from amazon S3 for other containers
Shell
1
star
89

estuary

Accumulates Avro datum received via RabbitMQ messages and stores them in Avro container files
Erlang
1
star
90

remy

Remy is a tool for managing Chef repositories and automating the addition and management of cookbooks with a chef server using Jenkins and GitHub.
Python
1
star
91

topological

Python3 library of topological sorting algorithms
Python
1
star
92

google-apps-for-domains-utilities

Small python utilities for managing Google Apps for Domains accounts
Python
1
star
93

apiary

JavaScript
1
star
94

tornado-template-gen

This application uses the Tornado Template module to generate static HTML files while still making available a subset of the Tornado template methods like static_url.
Python
1
star
95

rabbitmq-aws

A RabbitMQ Plugin fork of httpc-aws - Now @ rabbitmq/rabbitmq-aws
Erlang
1
star
96

tornado-dynamodb

Deprecated for sprockets.clients.dynamodb
Python
1
star