bench some web microframeworks (Ruby)
Because we don't have enough drama, let's bench some hello world apps!
Why?
As part of some research, collected some benchmark around Ruby web microframeworks.
The idea is bench the performance from Rack, throught the stack of the framework to the minimum representation of an application (Hello World apps).
It is not the goal to bechmark hits to the database or external services, nor to do a silly fibonacci-as-a-service, ok?
What?
Rack 2 compatible frameworks
- Cuba - cuba.ru
- Flame - flame.ru
- Grape - grape.ru
- Hanami (Router) - hanami-router.ru
- Hobbit - hobbit.ru
- Hobby - hobby.ru
- Newark - newark.ru
- Plezi - plezi.ru
- Rack - rack.ru + rack-response.ru
- Rack-App - rack-app.ru
- RackStep - rackstep.ru
- Rails (API + Metal) - rails-api.ru + rails-metal.ru
- Rambutan - rambutan.ru
- Roda - roda.ru
- Sinatra - sinatra.ru
- Syro - syro
- Watts - watts
Please note that while Rails has been added to the list, it is just a minimalistic representation with ActionPack gem (using API and Metal, no full middleware stack, etc). You shouldn't take the performance numbers mentioned here about Rails (or any other) as scientific and decision-taking references.
How?
Used wrk to benchmark, locally, a burst of requests (in 2 threads) over 10 seconds. The command line used was:
$ wrk -t 2 http://localhost:9292/All the frameworks using Puma on Ruby 2.4, in production mode with 16 threads:
$ puma -e production -t 16:16 apps/<framework.ru>Run benchmark for all frameworks
# use `bundle exec`, if needed
$ bench/summary-memory
$ bench/summary-speedHave some numbers around?
Yup, I do:
Requests/sec
Framework Requests/sec % from best
----------------------------------------------
rack 15839.64 100.00%
watts 13585.65 85.77%
syro 13306.01 84.00%
roda 11776.27 74.35%
cuba 11501.96 72.62%
rack-response 11480.11 72.48%
hobbit 11140.22 70.33%
hobby 10606.80 66.96%
hanami-router 10247.61 64.70%
newark 8896.35 56.17%
rambutan 8466.83 53.45%
plezi 8240.07 52.02%
rackstep 8187.32 51.69%
rack-app 7473.52 47.18%
rails-metal 6598.65 41.66%
flame 5454.46 34.44%
sinatra 3977.30 25.11%
grape 2937.03 18.54%
rails-api 1211.33 7.65%
Memory Allocation/Request
Framework Allocs/Req Memsize/Req
--------------------------------------
rack 41 2920
roda 45 3608
syro 45 3608
cuba 47 3688
watts 48 3200
hobbit 49 3928
hobby 53 4240
rack-response 56 4136
rails-metal 60 5728
plezi 66 5040
hanami-router 68 4880
newark 72 5928
rackstep 81 4944
rack-app 83 7184
rambutan 83 5944
flame 125 8536
sinatra 191 12512
grape 272 20800
rails-api 385 28017
Hardware
These numbers were collected on:
- Ubuntu 16.10 64-bit (kernel: 4.8.0-56-generic)
- Dell XPS 13 (9343, QHD, Developer Edition)
- Intel® Core™ i7-5500U CPU @ 2.40GHz × 4
- 8GB RAM
- 256GB SSD
- ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]