• Stars
    star
    200
  • Rank 195,325 (Top 4 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created about 2 years ago
  • Updated 10 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Small golang server to push PgSQL listen/notify events into websockets

pg_eventserv

A PostgreSQL-only event server in Go. Does one thing and one thing only: take events generated by the PostgreSQL NOTIFY command and passes the payload along to waiting WebSockets clients.

Setup and Installation

Download

Builds of the latest code:

Source

Download the source code and build:

make build

Basic Operation

The executable will read user/connection information from the DATABASE_URL environment variable and connect to the database, allowing any client with HTTP access to the server to connect and set up a WebSocket listening to any allowed channel on the server.

Linux/MacOS

export DATABASE_URL=postgresql://username:password@host/dbname
./pg_eventserv

Windows

SET DATABASE_URL=postgresql://username:password@host/dbname
pg_eventserv.exe

Client Side

Once the service is running, you need a client to attach a web socket to it. You can use the built-in viewer at http://localhost:7700/ for testing, but you will eventually need to build one into your client application.

Here is a very simple Javascript client, for example.

<!DOCTYPE html>
<html lang="en">
  <body>
    <p><textarea id="display" rows="20" cols="60"></textarea></p>
    <p id="status"></p>
    <script>
      window.onload = function() {
        // events on channel 'people'
        var url = "ws://localhost:7700/listen/people";
        var status = document.getElementById("status");
        var display = document.getElementById("display");
        // open socket and connect event handlers
        var ws = new WebSocket(url);
        ws.onopen = function() {
            status.innerHTML = "Socket open.";
        };
        ws.onerror = function(error) {
            status.innerHTML = "Socket error.";
        };
        ws.onmessage = function (e) {
          // First try to parse message as JSON.
          // Catch failures and return.
          try {
            var payload = JSON.parse(e.data);
            display.innerHTML += JSON.stringify(payload, null, 2) + "\n";
          }
          catch (err) {
            display.innerHTML += e.data + "\n";
          }
          display.scrollTop = display.scrollHeight;
        };
        ws.onclose = function(event) {
            status.innerHTML = "Socket closed.";
        }
      }
    </script>
  </body>
</html>

You can also test the service by pointing your browser at the service test page, http://localhost:7700/, and entering one or more channel names.

Raising a Notification

To send a message from the database to the web socket client, connect to your database. Then run the NOTIFY command:

NOTIFY channelname, 'message to send';

You can also raise a notification by running the pg_notify() function.

SELECT pg_notify('channelname', 'message to send');

Trouble-shooting

To get more information about what is going on behind the scenes, run with the --debug commandline parameter on, or turn on debugging in the configuration file:

./pg_eventserv --debug

Configuration Using Environment Variables

Any parameter in the configuration file can be over-ridden at run-time in the environment. Prepend the upper-cased parameter name with ES_ to set the value. For example, to change the HTTP port using the environment:

export ES_HTTPPORT=7777

Operation

The purpose of pg_eventserv is to take events that are generated in the database and make them accessible to web clients. The general idea is to instrument the database model to capture the events that are of interest to your application, so you don't need to build that orchestration somewhere else. Databases have lots of logic and filtering smarts, and generating the events of interests inside the database can simplify development of event-driven applications.

Listening to a Channel

You can listen to any channel allowed by the service (the default is to open all channels, but that can be limited with the Channels configuration option) by opening a WebSocket connection with the following URL pattern.

ws://{host}:7700/listen/{channel}

Once the channel is open all NOTIFY commands in the database that reference the channel will cause a WebSockets message to be sent to all clients listening to the channel.

Simple Data Notification

The most basic form of event is a change of data. An insert or an update, for example. Here's an example that generates a new event for every insert and update on the table.

CREATE TABLE people (
    pk serial primary key,
    ts timestamptz DEFAULT now(),
    name text,
    age integer,
    height real
);

CREATE OR REPLACE FUNCTION data_change() RETURNS trigger AS
$$
    DECLARE
        js jsonb;
    BEGIN
        SELECT to_jsonb(NEW.*) INTO js;
        js := jsonb_set(js, '{dml_action}', to_jsonb(TG_OP));
        PERFORM (
            SELECT pg_notify('people', js::text)
        );
        RETURN NEW;
    END;
$$ LANGUAGE 'plpgsql';

DROP TRIGGER IF EXISTS data_change_trigger
  ON people;
CREATE TRIGGER data_change_trigger
    BEFORE INSERT OR UPDATE ON people
    FOR EACH ROW
        EXECUTE FUNCTION data_change();

Install these functions, turn on a web socket for the people channel (use the service front page, for example) then run some data modifications.

INSERT INTO people (name, age, height) VALUES ('Paul', 51, 1.9);
INSERT INTO people (name, age, height) VALUES ('Colin', 65, 1.5);

Filtered Data Notification

Sending data modification events is less interesting than sending events when some kind of condition exists. For example, we might only want to raise events when the new data indicates a height greater than 2.0.

CREATE OR REPLACE FUNCTION data_change() RETURNS trigger AS
$$
    DECLARE
        js jsonb;
    BEGIN
        IF NEW.height >= 2.0
        THEN
            SELECT to_jsonb(NEW.*) INTO js;
            PERFORM (
                SELECT pg_notify('people', js::text)
            );
        END IF;
        RETURN NEW;
    END;
$$ LANGUAGE 'plpgsql';

Then send in some updates that pass the filter and others that don't.

UPDATE people SET name = 'Shorty', height = 1.5 WHERE age = 51;
UPDATE people SET name = 'Bozo', height = 2.1 WHERE age = 51;
INSERT INTO people (name, age, height) VALUES ('Stretch', 33, 2.8);

More Repositories

1

postgres-operator

Production PostgreSQL for Kubernetes, from high availability Postgres clusters to full-scale database-as-a-service.
Go
3,916
star
2

crunchy-containers

Containers for Managing PostgreSQL on Kubernetes by Crunchy Data
Shell
1,011
star
3

pg_tileserv

A very thin PostGIS-only tile server in Go. Takes in HTTP tile requests, executes SQL, returns MVT tiles.
Go
858
star
4

pgmonitor

PostgreSQL Monitoring, Metrics Collection and Alerting Resources from Crunchy Data
PLpgSQL
568
star
5

pg_featureserv

Lightweight RESTful Geospatial Feature Server for PostGIS in Go
Go
451
star
6

crunchy-proxy

PostgreSQL Connection Proxy by Crunchy Data (beta)
Go
414
star
7

pg_parquet

Copy to/from Parquet in S3 from within PostgreSQL
Rust
248
star
8

postgres-operator-examples

Examples for deploying applications with PGO, the Postgres Operator from Crunchy Data
Handlebars
184
star
9

pgCompare

pgCompare – a straightforward utility crafted to simplify the data comparison process, providing a robust solution for comparing data across various database platforms.
Java
79
star
10

postgresql-prometheus-adapter

Remote storage adapter enabling Prometheus to use PostgreSQL as a long-term store for time-series metrics.
Go
75
star
11

Postgres-AI-Tutorial

Python
58
star
12

pgnodemx

A PostgreSQL extension that provides SQL functions to allow capture of node OS metrics via SQL queries.
C
47
star
13

pgstigcheck-inspec

PostgreSQL STIG Compliance Testing Solution using InSpec
Ruby
41
star
14

crunchy-postgresql-manager

PostgreSQL-as-a-Service Management Utility. Superseded by Crunchy Container Suite: https://github.com/CrunchyData/crunchy-containers
HTML
36
star
15

pgbouncer_fdw

PostgreSQL Foreign Data Wrapper to Connect to pgbouncer
PLpgSQL
32
star
16

crunchy_check_access

Functions and views to facilitate PostgreSQL object access inspection
PLpgSQL
28
star
17

bridge-cli

CLI for Crunchy Bridge
Crystal
23
star
18

postgres-operator-client

Go
14
star
19

pgo-osb

Open Service Broker Implementation Based on the Crunchy PostgreSQL Operator
Go
13
star
20

crunchy-demo-data

Demo Data that is used in Crunchy Data Demos and Workshop that can be freely redistributed
HTML
13
star
21

crunchy-katacoda

Shell
10
star
22

openshift-postgres-cartridge

OpenShift V2 PG cartridge
Shell
8
star
23

backup-scripts

High Performance PostgreSQL Tools
Shell
8
star
24

crunchy-watch

A PostgreSQL Automated Failover Container
Go
8
star
25

pgSimload

pgSimload, a versatile CLI tool to create activity on PostgreSQL server(s) and/or test HA in Crunchy Postgres or Crunchy Postgres for Kubernetes
Go
7
star
26

terraform-provider-crunchybridge

Go
6
star
27

exporter2perfdata

Go
5
star
28

pgseccomp

C
5
star
29

crunchy-bridge-for-analytics-examples

Crunchy Bridge for Analytics Examples repository
PLpgSQL
4
star
30

pg_goggles

pg_goggles provides better scaled and summarized views into PostgreSQL's cryptic internal counters.
4
star
31

openshift-postgres-rls-cartridge

Shell
3
star
32

nixpkgs

Nix
3
star
33

crunchy-bridge-operator

Go
3
star
34

crunchy-hugo-theme

Theme used for Crunchy Data generated documentation
JavaScript
3
star
35

skybridge2

Go
2
star
36

postgres-ci-cd-demo

Python
2
star
37

os-pg-testing

Shell
1
star
38

postgres-realtime-demo

Python
1
star
39

pgmonitor-extension

PLpgSQL
1
star
40

spatial-suite-rest-demo

Python
1
star
41

crystal-keyring

Keyring implementation for Crystal
Crystal
1
star
42

crunchy-docs

TypeScript
1
star
43

doc-content-guide

1
star
44

crunchy-postgresql-manager-openshift

CPM Openshift version
HTML
1
star
45

homebrew-brew

homebrew tap
Ruby
1
star
46

spatial-suite-demo-webapp

JavaScript
1
star