• Stars
    star
    417
  • Rank 103,829 (Top 3 %)
  • Language
    JavaScript
  • License
    MIT License
  • Created over 8 years ago
  • Updated almost 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A comparison of websocket servers in multiple languages and frameworks

websocket-shootout

This project is designed to compare websocket servers in multiple languages and frameworks and has a companion blog post. The servers all implement an extremely simple protocol with only two messages: echo and broadcast. An echo is returned to the sending client. A broadcast is sent to all connected clients. Both messages take a payload value that should be delivered to the appropriate destination.

Example broadcast message:

{"type":"broadcast","payload":{"foo": "bar"}}

For the platforms with low level websocket implementations the above message would work directly. For platforms with higher level abstractions such as Phoenix and Rails the message must be encoded to be compatible with their message standards.

Platforms

The following platforms currently have servers implemented.

  • Clojure
  • C++
  • Elixir / Phoenix
  • Go
  • Haskell
  • Java
  • Javascript / NodeJS
  • Ruby / EventMachine
  • Ruby / Rails
  • Rust

Build Instructions

Some dependencies are tracked via git submodules. First step is to pull them down.

git submodule init
git submodule update

Look for a README.md in each projects directory for instructions on building and running the servers.

Benchmark

As part of this comparison a benchmark tool websocket-bench was built to test the performance of these websocket servers. websocket-bench is designed to find how many connections a server can handle while providing an acceptable level of performance. For example, given the requirement that 4 broadcast requests are served concurrently and 95% of broadcasts be completed within 500ms, how many connections can the server handle?

Here is an example benchmark run:

% % bin/websocket-bench broadcast ws://earth.local:3334/ws --concurrent 10 --sample-size 100 --step-size 1000 --limit-percentile 95 --limit-rtt 250ms
clients:  1000    95per-rtt:  47ms    min-rtt:   9ms    median-rtt:  20ms    max-rtt:  66ms
clients:  2000    95per-rtt:  87ms    min-rtt:   9ms    median-rtt:  43ms    max-rtt: 105ms
clients:  3000    95per-rtt: 121ms    min-rtt:  21ms    median-rtt:  58ms    max-rtt: 201ms
clients:  4000    95per-rtt: 163ms    min-rtt:  30ms    median-rtt:  76ms    max-rtt: 325ms
clients:  5000    95per-rtt: 184ms    min-rtt:  37ms    median-rtt:  95ms    max-rtt: 298ms

The above benchmark starts by connecting 1000 websocket clients to ws://earth.local:3334/ws. Then it sends 100 broadcast requests with a concurrency of 10. It increases by 1000 clients at a time until the 95th percentile round-trip time exceeds 250ms.

Run make to build the benchmark tool. Ensure you have previously initialized git submodules or the build will fail. The benchmark tool will be built to bin/websocket-bench.

Open file limits

Most servers have sufficient performance to encounter OS level open file limits. Here is how to increase those limits.

Ubuntu 16.04

Add the following to /etc/sysctl.conf:

fs.file-max = 2097152

Add the following to /etc/security/limits.conf:

*    soft nofile 1048576
*    hard nofile 1048576

In the shell run (or put in .profile or the like):

ulimit -n 1048576

Running Benchmarks

It is highly recommended that websocket-bench and the server be run on separate machines connected with at least GB ethernet.

Run websocket-bench with the --help parameter for detailed info.

% bin/websocket-bench --help

Outbound connection limits

A host can only establish a few ten-thousands of outbound connections before it suffer port exhaustion. To be more accurate that limit is per IP address. websocket-bench can use multiple IP addresses to establish more connections.

bin/websocket-bench broadcast ws://earth.local:3334/ws -c 4 -s 40 -l 192.168.50.5 -l 192.168.50.246 -l 192.168.50.247 --step-size 1000

The above command would use addresses 192.168.50.5, 192.168.50.246, and 192.168.50.247.

Of course, this requires that the host have multiple IP addresses. On Ubuntu 16.04 additional addresses can be bound to an interface by adding configuration to /etc/network/interfaces (this may require disabling network-manager if the machine is a desktop installation).

Example /etc/network/interfaces snippet:

...
up /sbin/ip addr add 192.168.50.246/24 dev eth0
up /sbin/ip addr add 192.168.50.247/24 dev eth0

down /sbin/ip addr del 192.168.50.246/24 dev eth0
down /sbin/ip addr del 192.168.50.247/24 dev eth0
...

Results

Results are in the results directory.

Contributing

This project is complete and is no longer actively maintained. We'll leave pull requests demonstrating other implementations open for educational purposes.

About

Hashrocket logo

websocket-shootout is supported by the team at Hashrocket, a multidisciplinary design and development consultancy. If you'd like to work with us or join our team, don't hesitate to get in touch.

More Repositories

1

decent_exposure

A helper for creating declarative interfaces in controllers
Ruby
1,799
star
2

gatling

Deployment tool for Phoenix apps
Elixir
498
star
3

tilex

Today I Learned
Elixir
495
star
4

ws

websocket command line tool
Go
425
star
5

dotmatrix

Hashrocket Dot Files
Vim Script
363
star
6

hr-til

Today I Learned in Ruby
Ruby
341
star
7

slurper

Gem for slurping plain text stories into Pivotal Tracker
Ruby
127
star
8

view_specify

Auto-generate RSpec view specs by interrogating your existing views.
Ruby
55
star
9

unencumbered

Just enough Cucumber in RSpec.
Ruby
53
star
10

capybara-webmock

Mock external requests for Capybara JavaScript drivers
Ruby
48
star
11

mousetrap

CheddarGetter API Client in Ruby
Ruby
34
star
12

namecheap

ruby namecheap API wrapper
Ruby
32
star
13

coming-soon

App to register emails addresses. Intended as a coming soon, pre-launch, splash page.
JavaScript
27
star
14

acts_as_featured

Ruby
25
star
15

design_patterns_in_ruby

Wherein we reimplement the design patterns from the _Design Patterns in Ruby_ book, by Russ Olsen.
Ruby
25
star
16

vim-hashrocket

Mappings we find useful
Vim Script
23
star
17

localpolitics.in

Hashrocket's Martin Luther King Jr. Memorial 2009 Hack Day Apps for America Celebrity Rabies Awareness Pro-Am Fun Run Race For The Cure
JavaScript
18
star
18

terraformation

Generators with a Hashrocket twist
Ruby
15
star
19

vim-macdown

write markdown in Vim with live-reloads in MacDown
Vim Script
14
star
20

hashrocket-rails

Rails engine & generators for bootstrapping a Hashrocket project
Ruby
12
star
21

slack-command-api

A Sinatra API for processing custom Slack commands
Ruby
12
star
22

boot_devcards_example

Example of using devcards with boot
Clojure
10
star
23

graphql_way_rails

This is a Proof of Concept Rails project using GraphQL
Ruby
6
star
24

wowza

Ruby wrapper around the Wowza REST API
Ruby
5
star
25

hr_hotels

Example database for hotels.
Ruby
5
star
26

hr

Hashrocket Sub
Ruby
5
star
27

ecto_pg_extras

A collection of custom functions for PostgreSQL features in Ecto
Elixir
4
star
28

my_emma

Ruby wrapper for the MyEmma Remote Signup API
Ruby
4
star
29

university-bookstore

Hashrocket Training Project for Pre-RailsConf 2010 Workshop
Ruby
4
star
30

hashrocket-vr-example

JavaScript
3
star
31

dibs-ios

An iOS client for the Dibs "Online classifieds without the strangers" web application.
Objective-C
3
star
32

OffBot

πŸ”Œ πŸ€– Timely slack notifications for scheduled holidays and vacations
Ruby
3
star
33

learn_to_program

Curriculums and Programs for Teaching people how to build web applications
Ruby
2
star
34

concepts

A gallery for our side projects
JavaScript
2
star
35

wowza-webhooks

Java
2
star
36

vim-hr-psql

Open psql formatted table definitions from Vim Postgres
Vim Script
2
star
37

chime

Chimes in the Chicago office when someone comes in the front door
Arduino
1
star
38

hacktive

Github activity tracker
Ruby
1
star
39

homebrew-formulas

Hashrocket homebrew formulas
Ruby
1
star
40

spices

Ruby
1
star
41

metro_relic

Easily track custom newrelic metrics with a config file
Ruby
1
star
42

vostok

The Hashrocket middleman template
CSS
1
star
43

trybool

The value parsing tool to return a boolean that you never knew you needed!
Ruby
1
star
44

homebrew-fdw

FDW formulas for Homebrew package manager
Ruby
1
star
45

scored

Simple iPhone app
Objective-C
1
star
46

hashshake

Hashrocket shakes hands 🀝
Ruby
1
star