• Stars
    star
    583
  • Rank 76,663 (Top 2 %)
  • Language
    Ruby
  • License
    MIT License
  • Created over 15 years ago
  • Updated over 2 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A simple TCP routing proxy built on EventMachine that lets you configure the routing logic in Ruby.

ProxyMachine

By Tom Preston-Werner ([email protected])

Description

ProxyMachine is a simple content aware (layer 7) TCP routing proxy built on EventMachine that lets you configure the routing logic in Ruby.

If you need to proxy connections to different backend servers depending on the contents of the transmission, then ProxyMachine will make your life easy!

The idea here is simple. For each client connection, start receiving data chunks and placing them into a buffer. Each time a new chunk arrives, send the buffer to a user specified block. The block's job is to parse the buffer to determine where the connection should be proxied. If the buffer contains enough data to make a determination, the block returns the address and port of the correct backend server. If not, it can choose to do nothing and wait for more data to arrive, close the connection, or close the connection after sending custom data. Once the block returns an address, a connection to the backend is made, the buffer is replayed to the backend, and the client and backend connections are hooked up to form a transparent proxy. This bidirectional proxy continues to exist until either the client or backend close the connection.

ProxyMachine was developed for GitHub's federated architecture and is successfully used in production to proxy millions of requests every day. The performance and memory profile have both proven to be excellent.

Installation

$ gem install proxymachine -s http://gemcutter.org

Running

Usage:
  proxymachine -c <config file> [-h <host>] [-p <port>]

Options:
  -c, --config CONFIG              Configuration file
  -h, --host HOST                  Hostname to bind. Default 0.0.0.0
  -p, --port PORT                  Port to listen on. Default 5432

Signals

QUIT - Graceful shutdown. Stop accepting connections immediately and
       wait as long as necessary for all connections to close.

TERM - Fast shutdown. Stop accepting connections immediately and wait
       up to 10 seconds for connections to close before forcing
       termination.

INT  - Same as TERM

Example routing config file

class GitRouter
  # Look at the routing table and return the correct address for +name+
  # Returns "<host>:<port>" e.g. "ae8f31c.example.com:9418"
  def self.lookup(name)
    ...
  end
end

# Perform content-aware routing based on the stream data. Here, the
# header information from the Git protocol is parsed to find the 
# username and a lookup routine is run on the name to find the correct
# backend server. If no match can be made yet, do nothing with the
# connection.
proxy do |data|
  if data =~ %r{^....git-upload-pack /([\w\.\-]+)/[\w\.\-]+\000host=\w+\000}
    name = $1
    { :remote => GitRouter.lookup(name) }
  else
    { :noop => true }
  end
end

Example SOCKS4 Proxy in 7 Lines

proxy do |data|
  return  if data.size < 9
  v, c, port, o1, o2, o3, o4, user = data.unpack("CCnC4a*")
  return { :close => "\0\x5b\0\0\0\0\0\0" }  if v != 4 or c != 1
  return  if ! idx = user.index("\0")
  { :remote => "#{[o1,o2,o3,o4]*'.'}:#{port}",
    :reply => "\0\x5a\0\0\0\0\0\0",
    :data => data[idx+9..-1] }
end

Valid return values

{ :remote => String } - String is the host:port of the backend server that will be proxied.
{ :remote => String, :data => String } - Same as above, but send the given data instead.
{ :remote => String, :data => String, :reply => String} - Same as above, but reply with given data back to the client { :noop => true } - Do nothing.
{ :close => true } - Close the connection.
{ :close => String } - Close the connection after sending the String.

Connection Errors and Timeouts

It's possible to register a custom callback for handling connection errors. The callback is passed the remote when a connection is either rejected or a connection timeout occurs:

proxy do |data|
  if data =~ /your thing/
    { :remote => 'localhost:1234', :connect_timeout => 1.0 }
  else
    { :noop => true }
  end
end

proxy_connect_error do |remote|
  puts "error connecting to #{remote}"
end

You must provide a :connect_timeout value in the proxy return value to enable connection timeouts. The :connect_timeout value is a float representing the number of seconds to wait before a connection is established. Hard connection rejections always trigger the callback, even when no :connect_timeout is provided.

Inactivity Timeouts

Inactivity timeouts work like connect timeouts but are triggered after the configured amount of time elapses without receiving the first byte of data from an already connected server:

proxy do |data|
  { :remote => 'localhost:1234', :inactivity_timeout => 10.0 }
end

proxy_inactivity_error do |remote|
  puts "#{remote} did not send any data for 10 seconds"
end

If no :inactivity_timeout is provided, the proxy_inactivity_error callback is never triggered.

Contribute

If you'd like to hack on ProxyMachine, start by forking my repo on GitHub:

http://github.com/mojombo/proxymachine

To get all of the dependencies, install the gem first. The best way to get your changes merged back into core is as follows:

  1. Clone down your fork
  2. Create a topic branch to contain your change
  3. Hack away
  4. Add tests and make sure everything still passes by running rake
  5. If you are adding new functionality, document it in the README.md
  6. Do not change the version number, I will do that on my end
  7. If necessary, rebase your commits into logical chunks, without errors
  8. Push the branch up to GitHub
  9. Send me (mojombo) a pull request for your branch

Copyright

Copyright (c) 2009 Tom Preston-Werner. See LICENSE for details.

More Repositories

1

chronic

Chronic is a pure Ruby natural language date parser.
Ruby
3,237
star
2

god

Ruby process monitor
Ruby
2,210
star
3

grit

**Grit is no longer maintained. Check out libgit2/rugged.** Grit gives you object oriented read/write access to Git repositories via Ruby.
Ruby
1,970
star
4

clippy

Clippy is a very simple Flash widget that makes it possible to place arbitrary text onto the client's clipboard.
939
star
5

mojombo.github.io

Jekyll source for my personal blog.
HTML
935
star
6

tpw

697
star
7

ernie

Ernie is an Erlang/Ruby BERT-RPC Server.
Erlang
463
star
8

erlectricity

Erlectricity exposes Ruby to Erlang and vice versa.
Ruby
349
star
9

tomdoc

A flexible code documentation specification with human readers in mind.
333
star
10

primer

Primer is a Flash-like API built on top of Canvas using jQuery.
JavaScript
280
star
11

mustache.erl

Mustache template engine for Erlang.
Erlang
239
star
12

bert

BERT (Binary ERlang Term) serialization library for Ruby.
Ruby
206
star
13

github-flavored-markdown

GitHub's Flavor of Markdown
183
star
14

bertrpc

BERTRPC is a Ruby BERT-RPC client library.
Ruby
164
star
15

sf-vegetarian-restaurants

A list of awesome vegetarian-friendly restaurants in SF
141
star
16

rakegem

Simple gem and release management with customizable Rake tasks.
Ruby
130
star
17

egitd

The Erlang git-daemon
Erlang
115
star
18

bert.erl

Erlang BERT encoder/decoder
Erlang
98
star
19

asteroids

Destroy your Atom editor, Asteroids style!
JavaScript
94
star
20

gollum-demo

Gollum test repo
Perl
75
star
21

rebar

Ruby to Erlang Bridge And Runner
Erlang
63
star
22

yaws

YAWS is an erlang web server
Erlang
42
star
23

omniship

Ruby API for Shipper APIs (UPS, FedEx, DHL)
Ruby
37
star
24

glowstick

A realtime, OpenGL graphing library for Ruby
Ruby
35
star
25

mastering-git-basics

A presentation on Git basics using ShowOff
JavaScript
35
star
26

octobeer

The GitHub Kegerator
27
star
27

cubesixel

A pixel font (native size 7x7) I made in a different life
27
star
28

vanhelsing

Super streamlined memory profiler with real time graphs for Ruby programs
Ruby
23
star
29

erlang_pipe

A pipe implementation in pure Erlang
Erlang
21
star
30

fixture-scenarios

This plugin allows you to create 'scenarios' which are collections of fixtures and ruby files that represent a context against which you can run tests.
Ruby
18
star
31

v8

C++
18
star
32

scoped

Scope access for multiple services in Ruby. Warning: OLDSCHOOL CODE!
Ruby
10
star
33

pyberry

A tool for systems analysis.
10
star
34

30daysoflaptops.github.io

CSS
8
star
35

conceptual_algorithms

6
star
36

mojombo.github.com

Old location of Jekyll source for tom.preston-werner.com
6
star
37

erlectricity-presentation

5
star
38

endo

Test repo for talk.
4
star
39

fakegem

This is a fake gem so I can test GitHub's gem building when I need to
Ruby
4
star
40

mojombo

3
star