• Stars
    star
    107
  • Rank 323,587 (Top 7 %)
  • Language
    Go
  • License
    GNU Affero Genera...
  • Created over 10 years ago
  • Updated 11 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

πŸ„ High-performance Server-Sent Events endpoint for Go

sseserver πŸ„β€β™‚οΈ

PkgGoDev Build Status CodeFactor Go Report Card

A high performance and thread-safe Server-Sent Events server for Go with hierarchical namespacing support.

This library has powered the streaming API endpoint for πŸ’« Emojitracker in production since 2014, where it routinely handles dispatching hundreds of messages per second to thousands of simultaneous clients, on a single Heroku dyno.

Introduction

Hierarchical Namespaced Channels*

A client can subscribe to channels reflecting the content they are interested in. For example, say you are broadcasting events to the namespaces /pets/cats and /pets/dogs. A client could subscribe to the parent channel /pets in the hierarchy and receive all messages for either.

In sseserver, channels have infinite depth and are automatically created on the fly with zero setup -- just broadcast and it works.

(*There's probably a more accurate term for this. If you know it, let me know.)

Performance

Designed for high throughput as primary performance consideration. In my preliminary benchmarking (on v1.0, circa 2014) this can handle ~100K/sec messages broadcast across ~1000 open HTTP connections on a 3.4GHz Intel Core i7 (using a single core, e.g. with GOMAXPROCS=1). There still remains quite a bit of optimization to be done so it should be able to get faster if needed.

SSE vs Websockets

SSE is the oft-overlooked uni-directional cousin of websockets. Being "just HTTP" it has some benefits:

  • Trvially easier to understand, plaintext format. Can be debugged by a human with curl.
  • Supported in most major browsers for a long time now. Everything except IE/Edge, but an easy polyfill!
  • Built-in standard support for automatic reconnection, event binding, etc.
  • Works with HTTP/2.

See also Smashing Magazine: "Using SSE Instead Of WebSockets For Unidirectional Data Flow Over HTTP/2".

Documentation

GoDoc

Namespaces are URLs

For clients, no need to think about protocol. To subscribe to one of the above namespaces from the previous example, just connect to http://$SERVER/pets/dogs. Done.

Example Usage

A simple Go program utilizing this package:

package main

import (
    "time"
    "github.com/mroth/sseserver"
)

func main() {
    s := sseserver.NewServer() // create a new server instance

    // broadcast the time every second to the "/time" namespace
    go func() {
        ticker := time.Tick(time.Duration(1 * time.Second))
        for {
            // wait for the ticker to fire
            t := <-ticker
            // create the message payload, can be any []byte value
            data := []byte(t.Format("3:04:05 pm (MST)"))
            // send a message without an event on the "/time" namespace
            s.Broadcast <- sseserver.SSEMessage{"", data, "/time"}
        }
    }()

    // simulate sending some scoped events on the "/pets" namespace
    go func() {
        time.Sleep(5 * time.Second)
        s.Broadcast <- sseserver.SSEMessage{"new-dog", []byte("Corgi"), "/pets/dogs"}
        s.Broadcast <- sseserver.SSEMessage{"new-cat", []byte("Persian"), "/pets/cats"}
        time.Sleep(1 * time.Second)
        s.Broadcast <- sseserver.SSEMessage{"new-dog", []byte("Terrier"), "/pets/dogs"}
        s.Broadcast <- sseserver.SSEMessage{"new-dog", []byte("Dauchsand"), "/pets/cats"}
        time.Sleep(2 * time.Second)
        s.Broadcast <- sseserver.SSEMessage{"new-cat", []byte("LOLcat"), "/pets/cats"}
    }()

    s.Serve(":8001") // bind to port and begin serving connections
}

All these event namespaces are exposed via HTTP endpoint in the /subscribe/:namespace route.

On the client, we can easily connect to those endpoints using built-in functions in JS:

// connect to an event source endpoint and print results
var es1 = new EventSource("http://localhost:8001/subscribe/time");
es1.onmessage = function(event) {
    console.log("TICK! The time is currently: " + event.data);
};

// connect to a different event source endpoint and register event handlers
// note that by subscribing to the "parent" namespace, we get all events
// contained within the pets hierarchy.
var es2 = new EventSource("http://localhost:8001/subscribe/pets")
es2.addEventListener("new-dog", function(event) {
    console.log("WOOF! Hello " + event.data);
}, false);

es2.addEventListener("new-cat", function(event) {
    console.log("MEOW! Hello " + event.data);
}, false);

Which when connecting to the server would yield results:

TICK! The time is currently: 6:07:17 pm (EDT)
TICK! The time is currently: 6:07:18 pm (EDT)
TICK! The time is currently: 6:07:19 pm (EDT)
TICK! The time is currently: 6:07:20 pm (EDT)
WOOF! Hello Corgi
MEOW! Hello Persian
TICK! The time is currently: 6:07:21 pm (EDT)
WOOF! Hello Terrier
WOOF! Hello Dauchsand
TICK! The time is currently: 6:07:22 pm (EDT)
TICK! The time is currently: 6:07:23 pm (EDT)
MEOW! Hello LOLcat
TICK! The time is currently: 6:07:24 pm (EDT)

Of course you could easily send JSON objects in the data payload instead, and most likely will be doing this often.

Another advantage of the SSE protocol is that the wire-format is so simple. Unlike WebSockets, we can connect with curl to an endpoint directly and just read what's going on:

$ curl http://localhost:8001/subscribe/pets
event:new-dog
data:Corgi

event:new-cat
data:Persian

event:new-dog
data:Terrier

event:new-dog
data:Dauchsand

event:new-cat
data:LOLcat

Yep, it's that simple.

Keep-Alives

All connections will send periodic :keepalive messages as recommended in the WHATWG spec (by default, every 15 seconds). Any library adhering to the EventSource standard should already automatically ignore and filter out these messages for you.

Admin Page

By default, an admin status page is available for easy monitoring.

screenshot

It's powered by a simple JSON API endpoint, which you can also use to build your own reporting. These endpoints can be disabled in the settings (see Server.Options).

HTTP Middleware

sseserver.Server implements the standard Go http.Handler interface, so you can easily integrate it with most existing HTTP middleware, and easily plug it into whatever you are using for your current routing, etc.

License

AGPL-3.0. Dual commercial licensing available upon request.

More Repositories

1

unindexed

πŸ”Žβ” website that irrevocably deletes itself once indexed
JavaScript
1,315
star
2

emojitracker

πŸ’« track ALL the emoji
743
star
3

phoenix-showdown

πŸ‡ benchmark Sinatra-like web frameworks
Elixir
704
star
4

weightedrand

βš–οΈ Fast weighted random selection for Go
Go
378
star
5

scmpuff

πŸ”’ Numeric file shortcuts for common git commands
Go
373
star
6

slacknimate

πŸ‘― Realtime text animation for Slack chatops
Go
286
star
7

evalcache

🐣 zsh plugin to cache eval loads to improve shell startup time
Shell
173
star
8

bootslap

β˜• bootstraps macOS to be usable
Shell
120
star
9

emojistatic

πŸ’© CDN static asset generation for emoji
CSS
98
star
10

exmoji

😎 Emoji encoding swiss army knife for Elixir/Erlang
Elixir
95
star
11

emoji_data.rb

😎 Emoji encoding swiss army knife for Ruby
Ruby
87
star
12

emoji-data-js

😎 Emoji encoding swiss army knife for NodeJS
CoffeeScript
64
star
13

ramdisk

🐏 Convenience wrapper for managing RAM disks
Go
47
star
14

benchwarmer

βŒ› Elixir micro-benchmarking library
Elixir
47
star
15

bump

🌻 CLI tool to draft a GitHub Release for the next semantic version
Go
46
star
16

git-muzak

🎢 Background music for your git commits
Shell
32
star
17

cameraform

πŸ“Ή Simple Flash+JS library for webcam capture and submission.
ActionScript
24
star
18

recyclebin

♻️ measures usage of a particular term on twitter
Go
21
star
19

grayratio

πŸ’¬ gray:blue ratio for iMessage conversations
Shell
16
star
20

twitter-streaming-showdown

🚣 benchmark Twitter Streaming API libraries
Ruby
15
star
21

cssquirt

πŸ’§ Embeds images (or directories of images) directly into CSS via the Data URI scheme.
Ruby
14
star
22

tinygeoip

πŸ‰ tiny geoip microservice
Go
13
star
23

momocode

πŸ‘ Visual fingerprinting for 20-byte Ethereum addresses via emoji
Solidity
12
star
24

xkcdpass

πŸ” xkcd style password generator for iOS
Swift
12
star
25

pullcrusher

🎩 optimizes all images in a GitHub repo & sends a pull request with the changes.
Ruby
10
star
26

emojidoll

🎎 twitter bot to generate Emoji dolls
Ruby
10
star
27

pigstream

🐷🐀 Twitter bot that insta-taunts frustrated people who can't beat a level on Angry Birds, using the streaming API.
Ruby
10
star
28

consider

πŸ€” quickly check code comments for subtle (racism|sexism|ableism)
Shell
9
star
29

golang-challenge-1

Solution for Go Challenge #1 (in Go and Elixir)
Go
9
star
30

deepclean

πŸ—‘οΈ scan and remove junk files from your source code directories
Go
9
star
31

scalafmt-native

Statically-linked GraalVM "native image" binaries of scalafmt
Dockerfile
8
star
32

my-boxen

πŸ‘” MacOSX system provisioning via Boxen [DEPRECATED, see mroth/bootslap]
Ruby
8
star
33

git-prompt-useremail

πŸ’‚β€β™‚οΈ zsh plugin adds prompt reminders for git user.email
Shell
8
star
34

hurricanecamp

πŸŒ€ Hurricane Dev Camp is a stay-at-home hackathon to coincide with Hurricane Irene.
8
star
35

subtleist

🎏 Anonymously remind of subtle-isms and other Recurse Center social rules in Slack
Go
8
star
36

nanogeoip

πŸ‰ tiny and blazing fast experimental geoip microservice
Rust
8
star
37

stardotws

🌟 source for a fun EMOJI DOMAIN
HTML
7
star
38

upcoming-cloud-warrior

☁️ Quick script/instructions to to get the Upcoming.org Archive Team Project running on multiple Heroku instances in the cloud.
Shell
7
star
39

hubhumans

πŸ‘ͺ Automatically create a `humans.txt` file based upon public members of a GitHub organization.
Ruby
6
star
40

bogan-martin-award

🐲 old-skool Flickr staff award website
HTML
6
star
41

goodvsevil

🐱🐢 A quick hack to compare keyword counts in the Twitter Streaming API, using puppies and kittens.
Ruby
5
star
42

portfolio

🎨 my portfolio website
HTML
5
star
43

personalappeals

🏩 Hot or Not of the Wikipedia contributors.
JavaScript
5
star
44

semverdesc

🎯 git describe with semantic version compatible names
Go
4
star
45

dotfiles

β­• My dotfiles repository. There are many like it, but this one is mine.
Shell
4
star
46

howami

πŸ’‰ command line tool to show a summary of Fitbit health data
Ruby
4
star
47

pdftotextcloud

πŸ“š pdftotext as a web service
JavaScript
4
star
48

pybaztag

🐰 Python convenience wrapper for the Nabaztag API.
Python
4
star
49

readtime

⌚ estimate how long it will take to read text
Go
4
star
50

shopmon

πŸͺ Monitor for in-stock products from Shopify powered stores
Go
4
star
51

forafriendbot

πŸ™‹ twitter bot that poses questions.... for a friend.
CoffeeScript
4
star
52

flickr-nearby-webos

πŸ“± Quick and dirty location services test for WebOS, source code to support blog post.
JavaScript
3
star
53

github-cli-xref

♻️ Tool to make it easy to cross-reference GitHub issues.
Ruby
3
star
54

fashionhack

πŸ‘  fashion hackday project with @kellan
Ruby
3
star
55

flickr-wholovesyou

πŸ’ž see which Flickr members most frequently favorite photos of you (or someone else you know!)
Ruby
3
star
56

tamanegi

🌰 generates tor .onion hashes
Go
3
star
57

hubfavor

πŸ™‡ determine who in a GitHub org is most likely to do you a favor
Ruby
3
star
58

meatballtracker

🍝 monitor and alert for delicious meatballs at a local restaurant.
Ruby
3
star
59

isgregdead

πŸ’€ Check whether a coworker (who occasionally keeps odd hours) is alive.
Ruby
3
star
60

foursquare-token-echo

4οΈβƒ£πŸ’­ simple webpage to receive and echo the client_token for a Foursquare OAuth2 request.
2
star
61

flickr-socialvenn

πŸ‘¬ Generates a weighted contact intersection venn diagram for a Flickr member.
2
star
62

poidh

πŸ‘€ pics or it didn't happen!
Ruby
2
star
63

sse-bench

πŸ“Š benchmarks Server-Sent Events endpoints
CoffeeScript
2
star
64

dees-colors

🌈 preview text as seen by a friend with grapheme-color synesthesia.
2
star
65

jitter

πŸ‘― Go timers with random jitter
Go
2
star
66

emojicompare

πŸ˜„β‰οΈπŸ˜„ A comparison of the Apple and Twitter emoji glyphs
2
star
67

go2go-docker

🐳 Docker image for the dev.go2go experimental Go branch
Dockerfile
2
star
68

loremfile

πŸ“œ Generate lorem ipsum text of a specific size
Go
1
star
69

.github

😈 Default community health files
1
star
70

dinnermint

🍬 automated metadata processing of personal photos on Flickr.
Ruby
1
star
71

tordesc

Parser for Tor network data descriptors
Rust
1
star
72

scalafmt-docker

Simple scalafmt packaging as a Docker Hub automated build
Dockerfile
1
star
73

rando-slackrisian

🎲 super quick CLI hack to get a random member of your Slack Team
JavaScript
1
star
74

base100-go

πŸ’― Go implementation of Base100 emoji encoding
Go
1
star
75

mta2json

πŸš‹ Proxies reqs to the MTA for Realtime Transit Feed data, converts results to JSON.
JavaScript
1
star
76

howlong

Quick self reference to answer a question I frequently get on calls.
HTML
1
star
77

sunnyinphilly

β›… is it sunny in Philadelphia?
1
star
78

timeduration

πŸ• simple CLI tool to convert humanized time durations
Go
1
star
79

xsort

80% faster versions of Go sort.Search* wrappers
Go
1
star
80

lolcapture

πŸ“Ή experimental capture tool for lolcommits
Swift
1
star
81

fitdump

πŸ“‰ Parse Fitbit data export files
Go
1
star
82

khan-bootstrap

πŸŽ“ A minimal setup for hacking the Khan Academy codebase
Makefile
1
star