• Stars
    star
    171
  • Rank 222,266 (Top 5 %)
  • Language
    Ruby
  • Created over 10 years ago
  • Updated over 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Trusterd: High Performance HTTP/2 Web Server scripting with mruby

logo

Special thanks to @maybehelpy for creating trusterd logo!!

Trusterd HTTP/2 Web Server

Gitter Build Status

Trusterd is a high performance HTTP/2 Web Server scripting with mruby using nghttp2 and mruby-http2. You can get HTTP/2 Web Server quickly which is high performance and customizable with mruby. The HTTP/2 server and client function are pluggable. So, you can embed these functions into your C applications.

TODO

This is a very early version, please test and report errors. Welcome pull-request.

  • Server Push

Requirements

rake bison git gperf libev-dev libevent-dev libjansson-dev libjemalloc-dev \
  libxml2-dev libssl-dev zlib1g-dev libc-ares-dev

After reading .travis.yml, you might easy to understand the install

Quick install

Manual Build

1. Install qrintf (Optional, but recommended)

Please see qrintf-gcc

or comment out this line in build_config.rb

cc.command = ENV['CC'] || 'qrintf-gcc'

2. Install jemalloc (Optional, but recommended)

Ubuntu
sudo apt-get install libjemalloc-dev

or comment out this line in build_config.rb

linker.flags_after_libraries << '-ljemalloc'

3. Download trusterd

git clone https://github.com/matsumotory/trusterd.git
cd trusterd

4. Build trusterd

make

5. Install

make install INSTALL_PREFIX=/usr/local/trusterd

6. Write config $(INSTALL_PREFIX)/conf/trusterd.conf.rb

SERVER_NAME = "Trusterd"
SERVER_VERSION = "0.0.1"
SERVER_DESCRIPTION = "#{SERVER_NAME}/#{SERVER_VERSION}"

root_dir = "/usr/local/trusterd"

# use env value
debug_opt = (ENV["RELEASE"] == "production") ? false : true

s = HTTP2::Server.new({

  #
  # required config
  #

  :port           => 8080,
  :document_root  => "#{root_dir}/htdocs",
  :server_name    => SERVER_DESCRIPTION,

  # support prefork only when linux kernel supports SO_REUSEPORT
  # :worker         => 4,

  # detect cpu thread automatically
  # If don't support SO_REUSEPORT of Linux, the number of worker is 0
  :worker         => "auto",

  # required when tls option is true.
  # tls option is true by default.
  #:key            => "#{root_dir}/ssl/server.key",
  #:crt            => "#{root_dir}/ssl/server.crt",
  #:dh_params_file => "#{root_dir}/ssl/dh.pem",

  # listen ip address
  # default value is 0.0.0.0
  # :server_host  => "127.0.0.1",

  #
  # optional config
  #

  # debug default: false
  :debug  =>  debug_opt,

  # tls default: true
  :tls => false,

  # daemon default: false
  # :daemon => true,

  # callback default: false
  # :callback => true,

  # connection_record default: true
  # :connection_record => false,

  # running user, start server with root and change to run_user
  # :run_user => "daemon",

  # Tuning RLIMIT_NOFILE, start server with root and must set run_user instead of root
  # :rlimit_nofile => 65535,

  # Set TCP_NOPUSH (TCP_CORK) option
  # :tcp_nopush => true,

  # expand buffer size before writing packet. decrease the number of small packets. That may be usefull for TLS session
  # :write_packet_buffer_expand_size => 4096 * 4,

  # limit buffer size before writing packet. write packet beyond the value. That may be usefull for TLS session
  # :write_packet_buffer_limit_size => 4096,

  # measuring server status: default false
  # :server_status => true,

  # use reverse proxy methods: default false
  # :upstream => true,

})

#
# when :callback option is true,
#
# # custom request headers
# # getter
# s.r.headers_in[":method"] #=> GET
# s.r.request_headers[":method"] #=> GET
#
# # custom response headers
# # setter
# s.r.headers_out["hoge"] = fuga
# s.r.response_headers["hoge"] = fuga
#
# # getter
# s.r.headers_out["hoge] #=> fuga
# s.r.response_headers["hoge] #=> fuga
#
#
# s.set_map_to_storage_cb {
#
#   p "callback block at set_map_to_storage_cb"
#   p s.filename            #=> /path/to/index.html
#   p s.uri                 #=> /index.html
#   p s.unparsed_uri        #=> /index.html?a=1&b=2
#   p s.percent_encode_uri  #=> /index.html?id=%E3%83%9F%E3
#   p s.args                #=> ?a=1&b=2
#   p s.body                #=> "post data"
#   p s.method              #=> "GET"
#   p s.scheme              #=> "https"
#   p s.authority           #=> "matsumoto-r.jp"
#   p s.host                #=> "matsumoto-r.jp"
#   p s.hostname            #=> "matsumoto-r.jp"
#   p s.client_ip           #=> "192.168.12.5"
#   p s.document_root       #=> "/path/to/htdocs"
#   p s.user_agent          #=> "trusterd_client"
#
#   p "if :server_status is true, use server status method"
#   p "the number of total requesting stream #{s.total_stream_request}"
#   p "the number of total requesting session #{s.total_session_request}"
#   p "the number of current connected sessions #{s.connected_sessions}"
#   p "the number of current processing streams #{s.active_stream}"
#
#   # location setting
#   if s.request.uri == "/index.html"
#     s.request.filename = "#{root_dir}/htdocs/hoge"
#   end
#   p s.request.filename
#
#   # you can use regexp if you link regexp mrbgem.
#   # Or, you can use KVS like mruby-redis or mruby-
#   # vedis and so on.
#
#   # reverse proxy config
#   # receive front end with HTTP/2 and proxy upstream server with HTTP/1.x
#   # TODO: don't support connection with TLS to upstream server
#   #
#   # need :upstream => true
#
#   if s.request.uri =~ /^/$/
#     # upstream request uri default: /
#     s.upstream_uri = s.percent_encode_uri
#     s.upstream_host = "127.0.0.1"
#
#     # upstream port default: 80
#     #s.upstream_port = 8080
#
#     # upstream connection timeout default: 600 sec
#     #s.upstream_timeout = 100
#
#     # use keepalive default: true
#     #s.upstream_keepalive = false
#
#     # use HTTP/1.0 protocol default: HTTP/1.1
#     #s.upstream_proto_major = 1
#     #s.upstream_proto_minor = 0
#   end
#
#   # dynamic content with mruby
#   if s.request.filename =~ /^.*\.rb$/
#     s.enable_mruby
#   end
#
#   # dynamic content with mruby sharing mrb_state
#   if s.request.filename =~ /^.*\_shared.rb$/
#     s.enable_shared_mruby
#   end
#
#  if s.request.uri =~ /hellocb/
#    s.set_content_cb {
#      s.rputs "hello trusterd world from cb"
#      s.echo "+ hello trusterd world from cb with \n"
#    }
#  end
#
# }

# extend response just before send response
# s.set_fixups_cb {
#  s.r.response_headers["server"] = "other server"
#}

# #If define set_content_cb this scope, callback only once
# s.set_content_cb {
#   s.rputs "hello trusterd world from cb"
#   s.echo "+ hello trusterd world from cb with \n"
# }

#
# f = File.open "#{root_dir}/logs/access.log", "a"
#
# s.set_logging_cb {
#
#   p "callback block after send response"
#
#   f.write "client_ip:'#{s.conn.client_ip}' date:'#{s.r.date}' status:#{s.r.status} content_length:#{s.r.content_length} uri:'#{s.r.uri}' filename:'#{s.r.filename}' user_agent:'#{s.r.user_agent}'\n"
#
# }

#
# or use access logging methods
#s.setup_access_log({
#  :file   => "/usr/local/trusterd/logs/access.log",
#
#  # :default or :custom
#  # if using :custom, set logging format to write_access_log method arg
#  # s.write_access_log "client_ip: #{s.client_ip}"
#  :format => :default,
#
#  # :plain or :json if using :default
#  :type   => :plain,
#})
#
#s.set_logging_cb {
#  s.write_access_log
#}
#


s.run

7. Run trusterd

make start INSTALL_PREFIX=/usr/local/trusterd

or

$(INSTALL_PREFIX)/bin/trusterd $(INSTALL_PREFIX)/conf/trusterd.conf.rb

or if you want to support both HTTP/1 and HTTP/2, we recommend to use nghttpx as reverse proxy

$(INSTALL_PREFIX)/bin/trusterd $(INSTALL_PREFIX)/conf/trusterd.conf.rb
sudo nghttpx -f,443 -b127.0.0.1,8080 --http2-bridge --backend-no-tls $(INSTALL_PREFIX)/ssl/server.key $(INSTALL_PREFIX)/ssl/server.crt

trusterd listen 127.0.0.1:8080 with HTTP/2 and nghttpx listen 0.0.0.0:443 as front with both HTTP/1 and HTTP/2.

8. Check by nghttp

nghttp is a client tool for HTTP/2.

$ ${GIT_CLONE_DIR}/mruby/build/host/mrbgems/mruby-http2/nghttp2/src/nghttp -v http://127.0.0.1:8080/index.html
hello trusterd world.

Clean

make clean

Using Docker

Using Docker image

Pulling
docker pull matsumotory/trusterd
Running
docker run -d -p 8080:8080 matsumotory/trusterd
Access
nghttp -v http://127.0.0.1:8080/index.html

Docker Image Build

Building
docker build -t local/trusterd .
Running

You can run trusted directly.

$ docker run --rm local/trusterd --help
Usage: ./bin/trusterd [switches] programfile
  switches:
  -b           load and execute RiteBinary (mrb) file
  -c           check syntax only
  -e 'command' one line of script
  -v           print version number, then run in verbose mode
  --verbose    run in verbose mode
  --version    print the version
  --copyright  print the copyright

Run with default configuration. (docker/conf/trusterd.conf.rb)

docker run -d -p 8080:8080 local/trusterd

Run with your configuration.

mkdir localconf
vi localconf/your_config.rb ## Write your configuration.
$ docker run -d -v `pwd`/localconf:/usr/local/trusterd/localconf -p 8080:8080 local/trusterd ./localconf/your_config.rb
Access
nghttp -v http://127.0.0.1:8080/index.html

Embed into your C application

HTTP/2 Server function

See src/mini_trusterd.c

HTTP/2 Client function

See src/mini_trusterd_client.c

Performance

Machine

  • Ubuntu14.04 on VMWare
  • Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz 4core
  • Memory 8GB

Config

trusterd.conf.rb

SERVER_NAME = "Trusterd"
SERVER_VERSION = "0.0.1"
SERVER_DESCRIPTION = "#{SERVER_NAME}/#{SERVER_VERSION}"

root_dir = "/usr/local/trusterd"

s = HTTP2::Server.new({

  :port           => 8081,
  :document_root  => "#{root_dir}/htdocs",
  :server_name    => SERVER_DESCRIPTION,
  :tls            => false,

})

s.run

Benchmarks

h2load is a benchmark tool for HTTP/2.

4 worker mode benchmark demo

500,000 requests/sec is very faster!!

4 worker mode cpu usage by top demo

trusterd worker processes use cpu resources of full core mostly.

HTTP/2

Server \ size of content 6 bytes 4,096 bytes
nghttpd (nghttpd @ a08ce38) single thread 148,841 73,812
nghttpd (nghttpd @ a08ce38) multi thread 347,152 104,244
tiny-nghttpd (nghttpd @ a08ce38) single thread 190,223 82,047
Trusterd @ 2432cc5 single process 204,769 92,068
Trusterd @ 2432cc5 multi process 509,059 134,542
H2O @ 529be4e single thread 216,453 112,356
H2O @ 529be4e multi thread 379,623 146,343

h2load -c 500 -m 100 -n 2000000

Ref: HTTP/1.1 on same benchmark environment

Server \ size of content 6 bytes 4,096 bytes
nginx single process 21,708 22,366
nginx multi process 67,349 56,203

weighttp -k -c 500 -n 200000

Memory

Startup

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     62085  0.0  0.0  46668  2288 pts/4    S+   16:41   0:00  |   \_ /usr/local/trusterd/bin/trusterd /usr/local/trusterd/conf/trusterd.conf.rb

After processing ten million request

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root     62085 63.3  0.0  49200  5144 pts/4    S+   16:41   0:47  |   \_ /usr/local/trusterd/bin/trusterd /usr/local/trusterd/conf/trusterd.conf.rb

License

under the MIT License:

More Repositories

1

ngx_mruby

ngx_mruby - A Fast and Memory-Efficient Web Server Extension Mechanism Using Scripting Language mruby for nginx
C
985
star
2

rcon

rcon is a lightweight resource virtualization tool for linux processes. This is one-binary.
Ruby
378
star
3

mod_mruby

mod_mruby: A Fast and Memory-Efficient Apache httpd Extension Mechanism Scripting with mruby
C
202
star
4

http-dos-detector

detect huge number of http access like DoS for Apache and nginx using mruby
Ruby
90
star
5

pmilter

Pmilter: Programmable Mail Filter Server Scripting with mruby
C++
81
star
6

mruby-redis

redis class for mruby.
C
49
star
7

mruby-http2

HTTP2 Module for mruby
C
44
star
8

mruby-mrbgem-template

mrbgem template generater
Ruby
35
star
9

mruby-vedis

vedis binding by mruby
C
34
star
10

dovecot-mruby-plugin

Programmable IMAP Server Scripting with mruby
C
30
star
11

libcgroup

C
30
star
12

mruby-simplehttpserver

mruby-simplehttpserver is a HTTP Server with less dependency for mruby
Ruby
26
star
13

k2i

Kernel Parameters Interface using HTTP
Rust
25
star
14

mod_process_security

mod_process_security is an access control module for CGI and DSO. Improvement of mod_ruid2(vulnerability) and mod_suexec(performance).
C
23
star
15

pfds

pfds - report a snapshot of the current processes fd
Ruby
19
star
16

mruby-cross-compile-on-mac-osx

Cross compile osx, linux or win32 binary of mruby on Mac OSX.
Ruby
19
star
17

mruby-virtualing

creating jail environment, limiting resouces, assigning IP address and separating filesystem
Ruby
19
star
18

mruby-cgroup

mruby-cgroups
C
18
star
19

ab-mruby

ab configured by mruby script.
C
18
star
20

mruby-httprequest

HttpRequest of iij/mruby support mruby/mruby using mruby-uv and mruby-http
Ruby
17
star
21

mruby-simplehttp

SimpleHttp of iij/mruby support mruby/mruby using mruby-uv and mruby-http
Ruby
17
star
22

matsumotory

my research and development archivements
HTML
16
star
23

tcpriv

tcpriv: Access Control Architecture Separating Privilege Transparently via TCP Connection Based on Process Information
C
16
star
24

docker-ngx_mruby

ngx_mruby using docker
Dockerfile
14
star
25

mruby-changefinder

ChangeFinder class detect change points via continuous outlier and smoothing.
Ruby
14
star
26

qos-control

qos-control.pl
Perl
13
star
27

mod_vhost_maxclients

MaxClients per vhost, no using shared memory and global lock. mod_vhost_maxclients use only scoreboad for stability, high compatibility and simple implementation.
C
13
star
28

mruby-userdata

When shared mrb_state, you can share userdata objects between one Ruby code and another is shared mrb_state.
C
12
star
29

mruby-on-Lua

mruby on Lua.
C
11
star
30

mruby-criu

CRIU, Checkpoint Restart In Userspace for Linux, bindig for mruby
C
10
star
31

procps-sys

Interface procps library from Rust
Rust
10
star
32

http-access-limiter

Count the number of references to the requested parameter on Apache and Nginx using mruby code.
Ruby
9
star
33

mruby-rcon

lightweight virtualization tool for linux processes module for mruby
Ruby
9
star
34

resources-managed-tools

managed resources tools using cfs and cgroups.
Perl
9
star
35

mruby-discount

mruby html generater from markdown
C
9
star
36

synciga

Assembly
8
star
37

ngx_custom_worker_module

ngx_custom_worker_module can set the number of processes with any factor of worker_processes auto.
C
8
star
38

mruby-capability

mruby-capability
C
8
star
39

mruby-memcached

libmemcached bindings by mruby
C
8
star
40

mod_extract_forwarded_for_2.4

mod_extract_forwarded for Apache httpd 2.4.*
C
7
star
41

mruby-mutex

mutex class for mruby
C
7
star
42

mruby-oauth

refactored oauth of iij/mruby using mruby-uv and mruby-http
Ruby
7
star
43

mruby-localmemcache

Mruby Inter Process Share Memory. Exchange memory space with mmap for multi mruby process.
C
7
star
44

mruby-sleep

mruby-sleep
C
7
star
45

middlecon

middleware conversation using Rust
Rust
6
star
46

h2o_mruby

h2o_mruby - A Fast and Memory-Efficient Web Server Extension Mechanism Using Scripting Language mruby for h2o
C
5
star
47

piotop

piotop
Perl
5
star
48

mruby-fast-remote-check

FastRemoteCheck can perform port listening check at high speed using raw socket
C
5
star
49

build-kernel-4.x-for-centos6

kernel-4.x build system for CentOS6 on CentOS6
Makefile
5
star
50

fish-ghq-vscode

Search and open a git repository as vscode project or a specified file of the repository by vscode.
Shell
5
star
51

mruby-lruc

C
4
star
52

mruby-disque

Disque client for mruby
Ruby
4
star
53

mod_request_dumper

output request_rec, server_rec, conn_rec on earch hook phases.
C
4
star
54

mruby-spdy

SPDY Module for mruby
C
4
star
55

kernel-4-build-system-for-centos6

kernel rpm of CentOS6 build system on OS X
Ruby
4
star
56

mruby-eventfd

Eventfd class
C
4
star
57

mruby-config

C
4
star
58

mod_resource_checker

Process Resource Logging Module using JSON format into file or piped program
C
4
star
59

mruby-pointer

Provide mruby C API which shared pointer between two mrb_states
C
3
star
60

mruby-random

mruby-random
C
3
star
61

mruby-tinymt

rand() and srand() method with TinyMT
C
3
star
62

check-tcport

Listen check tool via ipaddr, port and uid
C
3
star
63

mod_fileownercheck

This module resolve TOCTOU with FollowSymlinks
C
3
star
64

mruby-ngx-mruby-ext

Extended Nginx class for ngx_mruby. You can exnted Nginx class by mrbgem.
Ruby
2
star
65

mruby-netlink

libnetlink binding for mruby
C
2
star
66

mod_resource_manager

mod_resource_manager
C
2
star
67

release-code

release code by matsumoto_r
C
2
star
68

mruby-simpletest

Simple Test Using assert
Ruby
2
star
69

mod_process_name

mod_process_name
2
star
70

tcp_save_syn_listen

listen() with TCP_SAVE_SYN using LD_PRELOAD
C
2
star
71

capcon

Ruby
2
star
72

ipmi-control-tool

ipmi-control-tool
2
star
73

mruby-zabbix

zabbix 2.0 API client for mruby using mruby-httprequest.
Ruby
2
star
74

Create-IP-List-from-JPNIC

Create-IP-List-from-JPNIC
Perl
2
star
75

mruby-growthforecast

growthforecast class for mruby
Ruby
1
star
76

auth_tcpriv

C
1
star
77

Zabbix-API-Client

Zabbix-API-Client for perl
Perl
1
star
78

test-code

test
1
star
79

mod_zabbix

mod_zabbix is .....
1
star
80

mruby-mod-mruby-ext

Ruby
1
star
81

mod_vlimit

mod_vlimit
C
1
star
82

lua-get-resources

lua-get-resources lib for mod_lua
1
star
83

node-sample-chat

websocket sample chat for node.js and socket.io.
1
star
84

patch-for-apache

patch for Apache HTTP Server and Apache modules
1
star
85

Perl-Supporter

Perl module supporting code.
1
star
86

mruby-uname

system uname bindings
C
1
star
87

mod_vlimitconn

mod_vlimitconn
1
star
88

mod_load_monitor

mod_load_monitor - exec command over threshold of load average
C
1
star
89

mruby-geoip

GeoIP mrbgem using GeoIPCity.dat
C
1
star
90

ngx_mrubyc

ngx_mrubyc has less memory usage than ngx_mruby. ngx_mruby is a simpler extension mechanism of nginx with Ruby without GC.
1
star
91

randammer

put irc message from tweet of randam users.
Ruby
1
star
92

mod_lalimit

mod_lalimit
C
1
star
93

ngx_http_mruby

mruby shim for nginx
C
1
star
94

build_my_mruby

build my mruby
Shell
1
star
95

rcheck-analyzer

analyze mod_resource_checker log
Ruby
1
star