• Stars
    star
    894
  • Rank 51,062 (Top 2 %)
  • Language
    Ruby
  • License
    MIT License
  • Created about 11 years ago
  • Updated 3 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Pure Ruby implementation of HTTP/2 protocol

HTTP-2

Gem Version Build Status Coverage Status Analytics

Pure Ruby, framework and transport agnostic, implementation of HTTP/2 protocol and HPACK header compression with support for:

Protocol specifications:

Getting started

$> gem install http-2

This implementation makes no assumptions as how the data is delivered: it could be a regular Ruby TCP socket, your custom eventloop, or whatever other transport you wish to use - e.g. ZeroMQ, avian carriers, etc.

Your code is responsible for feeding data into the parser, which performs all of the necessary HTTP/2 decoding, state management and the rest, and vice versa, the parser will emit bytes (encoded HTTP/2 frames) that you can then route to the destination. Roughly, this works as follows:

require 'http/2'
socket = YourTransport.new

conn = HTTP2::Client.new
conn.on(:frame) {|bytes| socket << bytes }

while bytes = socket.read
 conn << bytes
end

Checkout provided client and server implementations for basic examples.

Connection lifecycle management

Depending on the role of the endpoint you must initialize either a Client or a Server object. Doing so picks the appropriate header compression / decompression algorithms and stream management logic. From there, you can subscribe to connection level events, or invoke appropriate APIs to allocate new streams and manage the lifecycle. For example:

# - Server ---------------
server = HTTP2::Server.new

server.on(:stream) { |stream| ... } # process inbound stream
server.on(:frame)  { |bytes| ... }  # encoded HTTP/2 frames

server.ping { ... } # run liveness check, process pong response
server.goaway # send goaway frame to the client

# - Client ---------------
client = HTTP2::Client.new
client.on(:promise) { |stream| ... } # process push promise

stream = client.new_stream # allocate new stream
stream.headers({':method' => 'post', ...}, end_stream: false)
stream.data(payload, end_stream: true)

Events emitted by the connection object:

:promise client role only, fires once for each new push promise
:stream server role only, fires once for each new client stream
:frame fires once for every encoded HTTP/2 frame that needs to be sent to the peer

Stream lifecycle management

A single HTTP/2 connection can multiplex multiple streams in parallel: multiple requests and responses can be in flight simultaneously and stream data can be interleaved and prioritized. Further, the specification provides a well-defined lifecycle for each stream (see below).

The good news is, all of the stream management, and state transitions, and error checking is handled by the library. All you have to do is subscribe to appropriate events (marked with ":" prefix in diagram below) and provide your application logic to handle request and response processing.

                      +--------+
                 PP   |        |   PP
             ,--------|  idle  |--------.
            /         |        |         \
           v          +--------+          v
    +----------+          |           +----------+
    |          |          | H         |          |
,---|:reserved |          |           |:reserved |---.
|   | (local)  |          v           | (remote) |   |
|   +----------+      +--------+      +----------+   |
|      | :active      |        |      :active |      |
|      |      ,-------|:active |-------.      |      |
|      | H   /   ES   |        |   ES   \   H |      |
|      v    v         +--------+         v    v      |
|   +-----------+          |          +-----------+  |
|   |:half_close|          |          |:half_close|  |
|   |  (remote) |          |          |  (local)  |  |
|   +-----------+          |          +-----------+  |
|        |                 v                |        |
|        |    ES/R    +--------+    ES/R    |        |
|        `----------->|        |<-----------'        |
| R                   | :close |                   R |
`-------------------->|        |<--------------------'
                      +--------+

For sake of example, let's take a look at a simple server implementation:

conn = HTTP2::Server.new

# emits new streams opened by the client
conn.on(:stream) do |stream|
  stream.on(:active) { } # fires when stream transitions to open state
  stream.on(:close)  { } # stream is closed by client and server

  stream.on(:headers) { |head| ... } # header callback
  stream.on(:data) { |chunk| ... }   # body payload callback

  # fires when client terminates its request (i.e. request finished)
  stream.on(:half_close) do

    # ... generate_response

    # send response
    stream.headers({
      ":status" => 200,
      "content-type" => "text/plain"
    })

    # split response between multiple DATA frames
    stream.data(response_chunk, end_stream: false)
    stream.data(last_chunk)
  end
end

Events emitted by the Stream object:

:reserved fires exactly once when a push stream is initialized
:active fires exactly once when the stream become active and is counted towards the open stream limit
:headers fires once for each received header block (multi-frame blocks are reassembled before emitting this event)
:data fires once for every DATA frame (no buffering)
:half_close fires exactly once when the opposing peer closes its end of connection (e.g. client indicating that request is finished, or server indicating that response is finished)
:close fires exactly once when both peers close the stream, or if the stream is reset
:priority fires once for each received priority update (server only)

Prioritization

Each HTTP/2 stream has a priority value that can be sent when the new stream is initialized, and optionally reprioritized later:

client = HTTP2::Client.new

default_priority_stream = client.new_stream
custom_priority_stream = client.new_stream(priority: 42)

# sometime later: change priority value
custom_priority_stream.reprioritize(32000) # emits PRIORITY frame

On the opposite side, the server can optimize its stream processing order or resource allocation by accessing the stream priority value (stream.priority).

Flow control

Multiplexing multiple streams over the same TCP connection introduces contention for shared bandwidth resources. Stream priorities can help determine the relative order of delivery, but priorities alone are insufficient to control how the resource allocation is performed between multiple streams. To address this, HTTP/2 provides a simple mechanism for stream and connection flow control.

Connection and stream flow control is handled by the library: all streams are initialized with the default window size (64KB), and send/receive window updates are automatically processed - i.e. window is decremented on outgoing data transfers, and incremented on receipt of window frames. Similarly, if the window is exceeded, then data frames are automatically buffered until window is updated.

The only thing left is for your application to specify the logic as to when to emit window updates:

conn.buffered_amount     # check amount of buffered data
conn.window              # check current window size
conn.window_update(1024) # increment connection window by 1024 bytes

stream.buffered_amount     # check amount of buffered data
stream.window              # check current window size
stream.window_update(2048) # increment stream window by 2048 bytes

Server push

An HTTP/2 server can send multiple replies to a single client request. To do so, first it emits a "push promise" frame which contains the headers of the promised resource, followed by the response to the original request, as well as promised resource payloads (which may be interleaved). A simple example is in order:

conn = HTTP2::Server.new

conn.on(:stream) do |stream|
  stream.on(:headers) { |head| ... }
  stream.on(:data) { |chunk| ... }

  # fires when client terminates its request (i.e. request finished)
  stream.on(:half_close) do
    promise_header = { ':method' => 'GET',
                       ':authority' => 'localhost',
                       ':scheme' => 'https',
                       ':path' => "/other_resource" }

    # initiate server push stream
    push_stream = nil
    stream.promise(promise_header) do |push|
      push.headers({...})
      push_stream = push
    end

    # send response
    stream.headers({
      ":status" => 200,
      "content-type" => "text/plain"
    })

    # split response between multiple DATA frames
    stream.data(response_chunk, end_stream: false)
    stream.data(last_chunk)
    
    # now send the previously promised data
    push_stream.data(push_data)
  end
end

When a new push promise stream is sent by the server, the client is notified via the :promise event:

conn = HTTP2::Client.new
conn.on(:promise) do |push|
  # process push stream
end

The client can cancel any given push stream (via .close), or disable server push entirely by sending the appropriate settings frame:

client.settings(settings_enable_push: 0)

Specs

To run specs:

rake

License

(MIT License) - Copyright (c) 2013 Ilya Grigorik GA

More Repositories

1

videospeed

HTML5 video speed controller (for Google Chrome)
JavaScript
3,812
star
2

ga-beacon

Google Analytics collector-as-a-service (using GA measurement protocol).
Go
3,536
star
3

gharchive.org

GH Archive is a project to record the public GitHub timeline, archive it, and make it easily accessible for further analysis.
Ruby
2,680
star
4

em-websocket

EventMachine based WebSocket server
Ruby
1,690
star
5

decisiontree

ID3-based implementation of the ML Decision Tree algorithm
Ruby
1,437
star
6

em-http-request

Asynchronous HTTP Client (EventMachine + Ruby)
Ruby
1,219
star
7

em-synchrony

Fiber aware EventMachine clients and convenience classes
Ruby
1,041
star
8

bugspots

Implementation of simple bug prediction hotspot heuristic
Ruby
853
star
9

agent

Agent is an attempt at modelling Go-like concurrency, in Ruby
Ruby
729
star
10

vimgolf

Real Vim ninjas count every keystroke - do you?
Ruby
678
star
11

em-proxy

EventMachine Proxy DSL for writing high-performance transparent / intercepting proxies in Ruby
Ruby
662
star
12

node-spdyproxy

SPDY forwarding proxy - fast and secure
JavaScript
527
star
13

bloomfilter-rb

BloomFilter(s) in Ruby: Native counting filter + Redis counting/non-counting filters
C
472
star
14

async-rails

async Rails 3 stack demo
Ruby
465
star
15

istlsfastyet.com

Is TLS fast yet? Yes, yes it is.
HTML
422
star
16

hackernews-button

Embeddable Hacker News button + vote counter for your site
Go
415
star
17

http-client-hints

Ruby
402
star
18

spdy

SPDY is a protocol designed to reduce latency of web pages
Ruby
315
star
19

hpbn.co

High Performance Browser Networking (O'Reilly)
HTML
299
star
20

webp-detect

WebP with Accept negotiation
C++
242
star
21

zeroconf-router

Zero-config reverse proxies: let's get there!
Ruby
206
star
22

autoperf

Ruby driver for httperf - automated load and performance testing
Ruby
179
star
23

PubSubHubbub

Asynchronous PubSubHubbub Ruby Client
Ruby
175
star
24

heroku-buildpack-dart

Heroku buildpack for Dart
Shell
170
star
25

rack-speedtracer

SpeedTracer middleware for server side debugging
Ruby
155
star
26

textquery

Evaluate any text against a collection of match rules
Ruby
145
star
27

tokyo-recipes

Lean & mean Tokyo Cabinet recipes (with Lua)
Lua
143
star
28

slowgrowl

Surface slow code paths in your Rails 3 app via Growl
Ruby
116
star
29

mneme

Mneme is an HTTP web-service for recording and identifying previously seen records - aka, duplicate detection.
Ruby
108
star
30

RRRDTool

Round robin database pattern via Redis sorted sets
Ruby
79
star
31

pregel

Single-node implementation of Google's Pregel framework for graph processing.
Ruby
74
star
32

gmetric

Pure Ruby interface for generating Ganglia gmetric packets
Ruby
69
star
33

rack-aggregate

Rack response-time statistics aggregator middleware
Ruby
67
star
34

em-jack

An Evented Beanstalk Client
Ruby
64
star
35

rb-pagerank

Code from RailsConf '09 pres: Building Mini Google in Ruby
Ruby
54
star
36

closure-sprockets

Sprockets processor for Google's Closure tools
Python
54
star
37

netinfo-monitor

Displays network quality as reported by Network Information API.
JavaScript
48
star
38

shopify-core-web-vitals

This embedded app provides a report on how real-world Google Chrome users experience the Shopify-powered storefront, as captured by the Chrome UX Report, and enables the site owner to benchmark their site against a custom list of competitors.
Ruby
48
star
39

libsnappy

Snappy, a fast compressor/decompressor (courtesy of Google)
Ruby
46
star
40

hydra5

Load-balanced (multi-headed) SOCKS5 proxy
Ruby
42
star
41

zdevice

ZDevice is a Ruby DSL for assembling ZeroMQ routing devices, with support for the ZDCF configuration syntax
Ruby
42
star
42

ruby2lolz

Ruby to Lolcode translator, kthnxbai.
Ruby
38
star
43

bmr-wordcount

Browser Map-Reduce: distributed word count example
Ruby
33
star
44

resource-hints

Moved to...
JavaScript
32
star
45

gitter

XML history generator for CodeSwarm
32
star
46

em-socksify

Transparent proxy support for any EventMachine protocol
Ruby
31
star
47

em-handlersocket

EventMachine HandlerSocket MySQL plugin for direct read/write of InnoDB tables
Ruby
29
star
48

canicrawl

Hosted robots.txt permissions verifier
Go
23
star
49

udacity-webperf

JavaScript
17
star
50

omnipipe

web pipes for your browser's omnibar!
Ruby
12
star
51

issue-tracker

W3C webperf issue tracker
JavaScript
11
star
52

contextual

runtime contextual HTML autoescaper
Ruby
10
star
53

presentations

Slides, notes, code examples from some of the bigger conferences & talks.
9
star
54

libgeohash

Ruby FFI wrapper for libgeohash
Ruby
7
star
55

performance-observer

JavaScript
7
star
56

ImageQuote

Convert text quotes to images
Ruby
7
star
57

resourcehints.info

HTML
2
star
58

igrigorik

1
star