• Stars
    star
    2,496
  • Rank 18,403 (Top 0.4 %)
  • Language
    Go
  • License
    Apache License 2.0
  • Created over 9 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

An efficient, extensible and easy-to-use RPC framework.

eRPC

tag Go Version GoDoc Build Status Go report License

eRPC is an efficient, extensible and easy-to-use RPC framework.

Suitable for RPC, Microservice, Peer-to-Peer, IM, Game and other fields.

简体中文

eRPC-Framework

Install

  • go vesion ≥ 1.18

  • install

GO111MODULE=on go get -u -v -insecure github.com/andeya/erpc/v7
  • import
import "github.com/andeya/erpc/v7"

Feature

  • Use peer to provide the same API package for the server and client
  • Provide multi-layout abstractions such as:
    • peer
    • session/socket
    • router
    • handle/context
    • message
    • protocol
    • codec
    • transfer filter
    • plugin
  • Support reboot and shutdown gracefully
  • HTTP-compatible message format:
    • Composed of two parts, the Header and the Body
    • Header contains metadata in the same format as HTTP header
    • Body supports for custom codec of Content Type-Like, already implemented:
      • Protobuf
      • Thrift
      • JSON
      • XML
      • Form
      • Plain
    • Support push, call-reply and more message types
  • Support custom message protocol, and provide some common implementations:
    • rawproto - Default high performance binary protocol
    • jsonproto - JSON message protocol
    • pbproto - Ptotobuf message protocol
    • thriftproto - Thrift message protocol
    • httproto - HTTP message protocol
  • Optimized high performance transport layer
    • Use Non-block socket and I/O multiplexing technology
    • Support setting the size of socket I/O buffer
    • Support setting the size of the reading message (if exceed disconnect it)
    • Support controling the connection file descriptor
  • Support a variety of network types:
    • tcp
    • tcp4
    • tcp6
    • unix
    • unixpacket
    • kcp
    • quic
    • other
      • websocket
      • evio
  • Provide a rich plug-in point, and already implemented:
    • auth
    • binder
    • heartbeat
    • ignorecase(service method)
    • overloader
    • proxy(for unknown service method)
    • secure
  • Powerful and flexible logging system:
    • Detailed log information, support print input and output details
    • Support setting slow operation alarm threshold
    • Support for custom implementation log component
  • Client session support automatically redials after disconnection

Benchmark

Self Test

  • A server and a client process, running on the same machine

  • CPU: Intel Xeon E312xx (Sandy Bridge) 16 cores 2.53GHz

  • Memory: 16G

  • OS: Linux 2.6.32-696.16.1.el6.centos.plus.x86_64, CentOS 6.4

  • Go: 1.9.2

  • Message size: 581 bytes

  • Message codec: protobuf

  • Sent total 1000000 messages

  • erpc

client concurrency mean(ms) median(ms) max(ms) min(ms) throughput(TPS)
100 1 0 16 0 75505
500 9 11 97 0 52192
1000 19 24 187 0 50040
2000 39 54 409 0 42551
5000 96 128 1148 0 46367
  • erpc/socket
client concurrency mean(ms) median(ms) max(ms) min(ms) throughput(TPS)
100 0 0 14 0 225682
500 2 1 24 0 212630
1000 4 3 51 0 180733
2000 8 6 64 0 183351
5000 21 18 651 0 133886

Comparison Test

EnvironmentThroughputsMean LatencyP99 Latency

More Detail

  • Profile torch of erpc/socket

erpc_socket_profile_torch

svg file

  • Heap torch of erpc/socket

erpc_socket_heap_torch

svg file

Example

server.go

package main

import (
	"fmt"
	"time"

	"github.com/andeya/erpc/v7"
)

func main() {
	defer erpc.FlushLogger()
	// graceful
	go erpc.GraceSignal()

	// server peer
	srv := erpc.NewPeer(erpc.PeerConfig{
		CountTime:   true,
		ListenPort:  9090,
		PrintDetail: true,
	})
	// srv.SetTLSConfig(erpc.GenerateTLSConfigForServer())

	// router
	srv.RouteCall(new(Math))

	// broadcast per 5s
	go func() {
		for {
			time.Sleep(time.Second * 5)
			srv.RangeSession(func(sess erpc.Session) bool {
				sess.Push(
					"/push/status",
					fmt.Sprintf("this is a broadcast, server time: %v", time.Now()),
				)
				return true
			})
		}
	}()

	// listen and serve
	srv.ListenAndServe()
}

// Math handler
type Math struct {
	erpc.CallCtx
}

// Add handles addition request
func (m *Math) Add(arg *[]int) (int, *erpc.Status) {
	// test meta
	erpc.Infof("author: %s", m.PeekMeta("author"))
	// add
	var r int
	for _, a := range *arg {
		r += a
	}
	// response
	return r, nil
}

client.go

package main

import (
  "time"

  "github.com/andeya/erpc/v7"
)

func main() {
  defer erpc.SetLoggerLevel("ERROR")()

  cli := erpc.NewPeer(erpc.PeerConfig{})
  defer cli.Close()
  // cli.SetTLSConfig(&tls.Config{InsecureSkipVerify: true})

  cli.RoutePush(new(Push))

  sess, stat := cli.Dial(":9090")
  if !stat.OK() {
    erpc.Fatalf("%v", stat)
  }

  var result int
  stat = sess.Call("/math/add",
    []int{1, 2, 3, 4, 5},
    &result,
    erpc.WithAddMeta("author", "andeya"),
  ).Status()
  if !stat.OK() {
    erpc.Fatalf("%v", stat)
  }
  erpc.Printf("result: %d", result)
  erpc.Printf("Wait 10 seconds to receive the push...")
  time.Sleep(time.Second * 10)
}

// Push push handler
type Push struct {
  erpc.PushCtx
}

// Push handles '/push/status' message
func (p *Push) Status(arg *string) *erpc.Status {
  erpc.Printf("%s", *arg)
  return nil
}

More Examples

Usage

NOTE:

  • It is best to set the packet size when reading: SetReadLimit
  • The default packet size limit when reading is 1 GB

Peer(server or client) Demo

// Start a server
var peer1 = erpc.NewPeer(erpc.PeerConfig{
ListenPort: 9090, // for server role
})
peer1.Listen()

...

// Start a client
var peer2 = erpc.NewPeer(erpc.PeerConfig{})
var sess, err = peer2.Dial("127.0.0.1:8080")

Call-Struct API template

type Aaa struct {
    erpc.CallCtx
}
func (x *Aaa) XxZz(arg *<T>) (<T>, *erpc.Status) {
    ...
    return r, nil
}
  • register it to root router:
// register the call route
// HTTP mapping: /aaa/xx_zz
// RPC mapping: Aaa.XxZz
peer.RouteCall(new(Aaa))

// or register the call route
// HTTP mapping: /xx_zz
// RPC mapping: XxZz
peer.RouteCallFunc((*Aaa).XxZz)

Service method mapping

  • The default mapping(HTTPServiceMethodMapper) of struct(func) name to service methods:

    • AaBb -> /aa_bb
    • ABcXYz -> /abc_xyz
    • Aa__Bb -> /aa_bb
    • aa__bb -> /aa_bb
    • ABC__XYZ -> /abc_xyz
    • Aa_Bb -> /aa/bb
    • aa_bb -> /aa/bb
    • ABC_XYZ -> /abc/xyz
    erpc.SetServiceMethodMapper(erpc.HTTPServiceMethodMapper)
  • The mapping(RPCServiceMethodMapper) of struct(func) name to service methods:

    • AaBb -> AaBb
    • ABcXYz -> ABcXYz
    • Aa__Bb -> Aa_Bb
    • aa__bb -> aa_bb
    • ABC__XYZ -> ABC_XYZ
    • Aa_Bb -> Aa.Bb
    • aa_bb -> aa.bb
    • ABC_XYZ -> ABC.XYZ
    erpc.SetServiceMethodMapper(erpc.RPCServiceMethodMapper)

Call-Function API template

func XxZz(ctx erpc.CallCtx, arg *<T>) (<T>, *erpc.Status) {
    ...
    return r, nil
}
  • register it to root router:
// register the call route
// HTTP mapping: /xx_zz
// RPC mapping: XxZz
peer.RouteCallFunc(XxZz)

Push-Struct API template

type Bbb struct {
    erpc.PushCtx
}
func (b *Bbb) YyZz(arg *<T>) *erpc.Status {
    ...
    return nil
}
  • register it to root router:
// register the push handler
// HTTP mapping: /bbb/yy_zz
// RPC mapping: Bbb.YyZz
peer.RoutePush(new(Bbb))

// or register the push handler
// HTTP mapping: /yy_zz
// RPC mapping: YyZz
peer.RoutePushFunc((*Bbb).YyZz)

Push-Function API template

// YyZz register the handler
func YyZz(ctx erpc.PushCtx, arg *<T>) *erpc.Status {
    ...
    return nil
}
  • register it to root router:
// register the push handler
// HTTP mapping: /yy_zz
// RPC mapping: YyZz
peer.RoutePushFunc(YyZz)

Unknown-Call-Function API template

func XxxUnknownCall (ctx erpc.UnknownCallCtx) (interface{}, *erpc.Status) {
    ...
    return r, nil
}
  • register it to root router:
// register the unknown call route: /*
peer.SetUnknownCall(XxxUnknownCall)

Unknown-Push-Function API template

func XxxUnknownPush(ctx erpc.UnknownPushCtx) *erpc.Status {
    ...
    return nil
}
  • register it to root router:
// register the unknown push route: /*
peer.SetUnknownPush(XxxUnknownPush)

Plugin Demo

// NewIgnoreCase Returns a ignoreCase plugin.
func NewIgnoreCase() *ignoreCase {
    return &ignoreCase{}
}

type ignoreCase struct{}

var (
    _ erpc.PostReadCallHeaderPlugin = new(ignoreCase)
    _ erpc.PostReadPushHeaderPlugin = new(ignoreCase)
)

func (i *ignoreCase) Name() string {
    return "ignoreCase"
}

func (i *ignoreCase) PostReadCallHeader(ctx erpc.ReadCtx) *erpc.Status {
    // Dynamic transformation path is lowercase
    ctx.UriObject().Path = strings.ToLower(ctx.UriObject().Path)
    return nil
}

func (i *ignoreCase) PostReadPushHeader(ctx erpc.ReadCtx) *erpc.Status {
    // Dynamic transformation path is lowercase
    ctx.UriObject().Path = strings.ToLower(ctx.UriObject().Path)
    return nil
}

Register above handler and plugin

// add router group
group := peer.SubRoute("test")
// register to test group
group.RouteCall(new(Aaa), NewIgnoreCase())
peer.RouteCallFunc(XxZz, NewIgnoreCase())
group.RoutePush(new(Bbb))
peer.RoutePushFunc(YyZz)
peer.SetUnknownCall(XxxUnknownCall)
peer.SetUnknownPush(XxxUnknownPush)

Config

type PeerConfig struct {
    Network            string        `yaml:"network"              ini:"network"              comment:"Network; tcp, tcp4, tcp6, unix, unixpacket, kcp or quic"`
    LocalIP            string        `yaml:"local_ip"             ini:"local_ip"             comment:"Local IP"`
    ListenPort         uint16        `yaml:"listen_port"          ini:"listen_port"          comment:"Listen port; for server role"`
    DialTimeout time.Duration `yaml:"dial_timeout" ini:"dial_timeout" comment:"Default maximum duration for dialing; for client role; ns,µs,ms,s,m,h"`
    RedialTimes        int32         `yaml:"redial_times"         ini:"redial_times"         comment:"The maximum times of attempts to redial, after the connection has been unexpectedly broken; Unlimited when <0; for client role"`
	RedialInterval     time.Duration `yaml:"redial_interval"      ini:"redial_interval"      comment:"Interval of redialing each time, default 100ms; for client role; ns,µs,ms,s,m,h"`
    DefaultBodyCodec   string        `yaml:"default_body_codec"   ini:"default_body_codec"   comment:"Default body codec type id"`
    DefaultSessionAge  time.Duration `yaml:"default_session_age"  ini:"default_session_age"  comment:"Default session max age, if less than or equal to 0, no time limit; ns,µs,ms,s,m,h"`
    DefaultContextAge  time.Duration `yaml:"default_context_age"  ini:"default_context_age"  comment:"Default CALL or PUSH context max age, if less than or equal to 0, no time limit; ns,µs,ms,s,m,h"`
    SlowCometDuration  time.Duration `yaml:"slow_comet_duration"  ini:"slow_comet_duration"  comment:"Slow operation alarm threshold; ns,µs,ms,s ..."`
    PrintDetail        bool          `yaml:"print_detail"         ini:"print_detail"         comment:"Is print body and metadata or not"`
    CountTime          bool          `yaml:"count_time"           ini:"count_time"           comment:"Is count cost time or not"`
}

Optimize

  • SetMessageSizeLimit sets max message size. If maxSize<=0, set it to max uint32.

    func SetMessageSizeLimit(maxMessageSize uint32)
  • SetSocketKeepAlive sets whether the operating system should send keepalive messages on the connection.

    func SetSocketKeepAlive(keepalive bool)
  • SetSocketKeepAlivePeriod sets period between keep alives.

    func SetSocketKeepAlivePeriod(d time.Duration)
  • SetSocketNoDelay controls whether the operating system should delay message transmission in hopes of sending fewer messages (Nagle's algorithm). The default is true (no delay), meaning that data is sent as soon as possible after a Write.

    func SetSocketNoDelay(_noDelay bool)
  • SetSocketReadBuffer sets the size of the operating system's receive buffer associated with the connection.

    func SetSocketReadBuffer(bytes int)
  • SetSocketWriteBuffer sets the size of the operating system's transmit buffer associated with the connection.

    func SetSocketWriteBuffer(bytes int)

Extensions

Codec

package import description
json "github.com/andeya/erpc/v7/codec" JSON codec(erpc own)
protobuf "github.com/andeya/erpc/v7/codec" Protobuf codec(erpc own)
thrift "github.com/andeya/erpc/v7/codec" Form(url encode) codec(erpc own)
xml "github.com/andeya/erpc/v7/codec" Form(url encode) codec(erpc own)
plain "github.com/andeya/erpc/v7/codec" Plain text codec(erpc own)
form "github.com/andeya/erpc/v7/codec" Form(url encode) codec(erpc own)

Plugin

package import description
auth "github.com/andeya/erpc/v7/plugin/auth" An auth plugin for verifying peer at the first time
binder "github.com/andeya/erpc/v7/plugin/binder" Parameter Binding Verification for Struct Handler
heartbeat "github.com/andeya/erpc/v7/plugin/heartbeat" A generic timing heartbeat plugin
proxy "github.com/andeya/erpc/v7/plugin/proxy" A proxy plugin for handling unknown calling or pushing
secure "github.com/andeya/erpc/v7/plugin/secure" Encrypting/decrypting the message body
overloader "github.com/andeya/erpc/v7/plugin/overloader" A plugin to protect erpc from overload

Protocol

package import description
rawproto "github.com/andeya/erpc/v7/proto/rawproto A fast socket communication protocol(erpc default protocol)
jsonproto "github.com/andeya/erpc/v7/proto/jsonproto" A JSON socket communication protocol
pbproto "github.com/andeya/erpc/v7/proto/pbproto" A Protobuf socket communication protocol
thriftproto "github.com/andeya/erpc/v7/proto/thriftproto" A Thrift communication protocol
httproto "github.com/andeya/erpc/v7/proto/httproto" A HTTP style socket communication protocol

Transfer-Filter

package import description
gzip "github.com/andeya/erpc/v7/xfer/gzip" Gzip(erpc own)
md5 "github.com/andeya/erpc/v7/xfer/md5" Provides a integrity check transfer filter

Mixer

package import description
multiclient "github.com/andeya/erpc/v7/mixer/multiclient" Higher throughput client connection pool when transferring large messages (such as downloading files)
websocket "github.com/andeya/erpc/v7/mixer/websocket" Makes the eRPC framework compatible with websocket protocol as specified in RFC 6455
evio "github.com/andeya/erpc/v7/mixer/evio" A fast event-loop networking framework that uses the erpc API layer
html html "github.com/xiaoenai/tp-micro/helper/mod-html" HTML render for http client

Projects based on eRPC

project description
TP-Micro TP-Micro is a simple, powerful micro service framework based on eRPC
Pholcus Pholcus is a distributed, high concurrency and powerful web crawler software

Business Users

深圳市梦之舵信息技术有限公司    平安科技
北京风行在线技术有限公司    北京可即时代网络公司 快手短视频平台

License

eRPC is under Apache v2 License. See the LICENSE file for the full license text

More Repositories

1

pholcus

Pholcus is a distributed high-concurrency crawler software written in pure golang
Go
7,471
star
2

faygo

Faygo is a fast and concise Go Web framework that can be used to develop high-performance web app(especially API) with fewer codes. Just define a struct handler, faygo will automatically bind/verify the request parameters and generate the online API doc.
Go
1,593
star
3

lessgo

Lessgo 是一款简单、稳定、高效、灵活的 golang web 开发框架,支持动态路由、自动化API测试文档、热编译、热更新等,实现前后端分离、系统与业务分离,完美兼容MVC与MVVC等多种开发模式,非常利于企业级应用与API接口的开发。[A simple, stable, efficient and flexible web framework.]
Go
479
star
4

goutil

Golang common tool functions and components.
Go
369
star
5

surfer

Package surfer is a high level concurrency http client. It has surf andphantom download engines, highly simulated browser behavior, the function of analog login and so on.
Go
214
star
6

aster

Easily get the golang syntax tree and modify the code.
Go
151
star
7

faydoc

User manual of faygo frame.
117
star
8

gust

A Rust-inspired declarative-programming and generic-type module for Golang that helps avoid bugs and improve development efficiency.
Go
89
star
9

algorithm

algorithm library
Go
58
star
10

erpc-doc

eRPC Documents
Go
55
star
11

tp-micro

TP-Micro is a simple, powerful micro service framework based on Teleport.
Go
54
star
12

pholcus_lib

仅供学习的Pholcus爬虫规则库
Go
43
star
13

ameda

Powerful toolbox for golang data types.
Go
31
star
14

lessgo_demo

A demo of Lessgo.
Go
31
star
15

rester

Fast and concise RESTful web framework based on fasthttp
Go
27
star
16

apiware

Apiware binds the specified parameters of the Golang net/http and fasthttp requests to the structure and verifies the validity of the parameter values.
Go
24
star
17

wasmy

wasmy, easily customize my wasm app!
Rust
22
star
18

lessgoext

Lessgo's expansion module.
Go
22
star
19

timer

Go语言各种定时器的实现。
Go
21
star
20

rpc-benchmark

golang RPC framework benchmark
Go
20
star
21

go-crawler-practice

📕《Go语言爬虫编写实战》
19
star
22

lessgo_doc

Lessgo user's manual.
Go
18
star
23

pool

通用资源池,动态增加资源实例,并支持空闲资源定时回收。
Go
18
star
24

opay

以订单为主线、面向接口开发的在线支付通用模块
Go
17
star
25

mahonia

Copy from http://code.google.com/p/mahonia/
Go
17
star
26

pholcus_dependent

Pholcus第三方依赖包源码下载
Go
16
star
27

gofield

High-performance struct field accessor based on unsafe pointers.
Go
14
star
28

wasmesh

wasmesh is a WebAssembly service mesh framework.
Rust
14
star
29

erpc-book

Handbook of eRPC which is an efficient, extensible and easy-to-use RPC framework.
Go
12
star
30

beelogs

Upgrade from beego logs !
Go
12
star
31

tp-ext

Teleport v3 custom extensions collection.
Go
12
star
32

tconfig

源自beego的升级版配置文件解析库,目前支持解析的文件格式有 ini、json、xml、yaml。
Go
10
star
33

faygos

A micro service framework developed using 'faygo' and 'etcd'
Go
10
star
34

ipc

IPC Inter-Process Communication.
Go
9
star
35

cfgo

Cfgo from the YAML document, bi-directional synchronous multi-module configuration.
Go
9
star
36

beauty-go

Go解答《编程之美》系列
Go
9
star
37

less

lessgo 项目部署工具
Go
9
star
38

TRRW-Chrome-Extension-Template

Use React + TypeScript + Antd + Rust-Wasm to develop Chrome Extension.
TypeScript
8
star
39

flagx

Standard flag package extension with more features, such as struct flag, app framework, etc.
Go
8
star
40

easysync2

The EasySync2 Algorithm
Rust
7
star
41

utils

Go语言中的一些常用函数。
Go
7
star
42

lessgo_dependency

Lessgo‘s dependency package.
Go
7
star
43

env

Package env provides a convenient way to initialize variables from the environment.
Go
7
star
44

currip

currip helps you get current extranet ip or intranet ip.
Go
7
star
45

widerror

Error type with wide attributes, used for API unified error code
Rust
5
star
46

mysql-aes

MySQL Transparent Data Encryption
Go
5
star
47

henrylee2cn.github.com

Share Notes !
CSS
5
star
48

tp-p2p

A teleport-based p2p application framework that penetrates the intranet through TCP tunnel.
Go
5
star
49

option

Go-generics option module inspired by rust.
Go
4
star
50

sulfa

Basic algorithm library implemented by rust.
Rust
4
star
51

lycee

lycee is a key-value storage system being developed for me to learn rust.
Rust
4
star
52

codec_protobuf

Go
4
star
53

chinaid

Make or verify China ID
Go
4
star
54

andeya

3
star
55

slinbo

A personal chat application.
C++
2
star
56

result

Go-generics result module inspired by rust.
Go
2
star
57

sqlab

The project waiting for planning, some SQL tool functions are temporarily deposited.
Go
2
star
58

enum_const

enum const trait
Rust
2
star
59

vscode_debug_go

Configure my own local launch.json to improve ease of use
Go
2
star
60

api-response

A consistent structure for API responses, including success and error handling.
Rust
1
star
61

cubix

Standardized building block components born for Rust craft.
Rust
1
star
62

py-layout

Python Project Generator
Python
1
star
63

imgs-repo

我的图床仓库
1
star
64

convert_traits

Define your own conversion traits to solve the problem of converting two external types without using new types.
Rust
1
star