• Stars
    star
    199
  • Rank 196,105 (Top 4 %)
  • Language
    Erlang
  • License
    Apache License 2.0
  • Created about 10 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Erlang general non-blocking TCP/SSL socket server

esockd Build status

Erlang General Non-blocking TCP/SSL Socket Server.

Features

  • General Non-blocking TCP/SSL Socket Server
  • Acceptor Pool and Asynchronous TCP Accept
  • UDP/DTLS Server
  • Max connections management
  • Allow/Deny by peer address
  • Proxy Protocol V1/V2
  • Keepalive Support
  • Rate Limit
  • IPv6 Support

Usage

A Simple TCP Echo Server:

-module(echo_server).

-export([start_link/2, init/2]).

start_link(Transport, Sock) ->
    {ok, spawn_link(?MODULE, init, [Transport, Sock])}.

init(Transport, Sock) ->
    case Transport:wait(Sock) of
        {ok, NewSock} ->
            loop(Transport, NewSock);
        Error -> Error
    end.

loop(Transport, Sock) ->
    case Transport:recv(Sock, 0) of
        {ok, Data} ->
            {ok, Peername} = Transport:peername(Sock),
            Transport:send(Sock, Data),
            loop(Transport, Sock);
        {error, Reason} ->
            io:format("TCP Error: ~s~n", [Reason]),
            {stop, Reason}
    end.

Setup Echo Server:

%% Start esockd application
ok = esockd:start().
Options = [{acceptors, 10}, {max_connections, 1024}, {tcp_options, [binary, {reuseaddr, true}]}].
MFArgs = {echo_server, start_link, []},
esockd:open(echo, 5000, Options, MFArgs).

Examples

Example Description
examples/async_recv prim_net async recv/send
examples/gen_server gen_server behaviour
examples/simple simple echo server
examples/ssl ssl echo server
examples/proxy_protocol proxy protocol v1/2
examples/udp udp echo server
examples/dtls dtls echo server

API

Open a listener

esockd:open(echo, 5000, [{tcp_options, [binary, {reuseaddr, true}]}],
            {echo_server, start_link, []}).

esockd:open(echo, {"127.0.0.1", 6000}, [{tcp_options, [binary, {reuseaddr, true}]}],
            {echo_server, start_link, []}).

Spec:

-spec(open(Protocol, ListenOn, Options, MFArgs) -> {ok, pid()} | {error, term()} when
           Protocol :: atom(),
           ListenOn :: inet:port_number() | {host(), inet:port_number()}),
           Options  :: [option()],
           MFArgs   :: esockd:mfargs()).

Option:

-type(option() :: {acceptors, pos_integer()}
                | {max_connections, pos_integer()}
                | {max_conn_rate, pos_integer()}
                | {access_rules, [esockd_access:rule()]}
                | {shutdown, brutal_kill | infinity | pos_integer()}
                | tune_buffer | {tune_buffer, boolean()}
                | proxy_protocol | {proxy_protocol, boolean()}
                | {proxy_protocol_timeout, timeout()}
                | {ssl_options, [ssl:ssl_option()]}
                | {udp_options, [gen_udp:option()]}
                | {dtls_options, [gen_udp:option() | ssl:ssl_option()]}).

MFArgs:

-type(mfargs() :: atom() | {atom(), atom()} | {module(), atom(), [term()]}).

Get Setting and Stats

Get stats:

esockd:get_stats({echo, 5000}).

Get acceptors:

esockd:get_acceptors({echo, {"127.0.0.1", 6000}}).

Get/Set max connections:

esockd:get_max_connections({echo, 5000}).
esockd:set_max_connections({echo, 5000}, 100000).

Allow/Deny

Same to Allow/Deny Syntax of nginx:

allow address | CIDR | all;

deny address | CIDR | all;

allow/deny by options:

esockd:open(echo, 5000, [{access, [{deny, "192.168.1.1"}, {allow, "192.168.1.0/24"}, {deny, all}]}], MFArgs).

allow/deny by API:

esockd:allow({echo, 5000}, all).
esockd:allow({echo, 5000}, "192.168.0.1/24").
esockd:deny({echo, 5000}, all).
esockd:deny({echo, 5000}, "10.10.0.0/16").

Close a listener

esockd:close(echo, 5000).
esockd:close(echo, {"127.0.0.1", 6000}).

Spec:

-spec(close(Protocol, ListenOn) -> ok when Protocol :: atom(), ListenOn :: inet:port_number() | {host(), inet:port_number()}).

SSL

Connecting to ssl_echo_server:

openssl s_client -connect 127.0.0.1:5000 -ssl3

openssl s_client -connect 127.0.0.1:5000 -tls1

Design

Supervisor Tree

esockd_sup
    -> esockd_listener_sup
        -> esockd_listener
        -> esockd_acceptor_sup
            -> esockd_acceptor
            -> esockd_acceptor
            -> ......
        -> esockd_connection_sup
            -> esockd_connection
            -> esockd_connection
            -> ......

Acceptor

  1. Acceptor Pool

  2. Suspend for one second when e{n, m}file errors happened

Connection Sup

  1. Create a connection, and let it run...

  2. Control maximum connections

  3. Count active connections

  4. Count shutdown reasons

CIDR

CIDR Wiki: https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing

Benchmark

Benchmark 2.1.0-alpha release on one 8 cores, 32G memory ubuntu/14.04 server::

250K concurrent connections, 50K messages/sec, 40Mbps In/Out consumed 5G memory, 20% CPU/core

License

Apache License Version 2.0

Author

EMQ X Team.

More Repositories

1

emqx

The most scalable open-source MQTT broker for IoT, IIoT, and connected vehicles
Erlang
13,975
star
2

MQTTX

A Powerful and All-in-One MQTT 5.0 client toolbox for Desktop, CLI and WebSocket.
TypeScript
3,843
star
3

CocoaMQTT

MQTT 5.0 client library for iOS and macOS written in Swift
Swift
1,589
star
4

neuron

Open source industrial IoT connectivity server
C
1,023
star
5

MQTT-Client-Examples

MQTT Client Examples
Kotlin
743
star
6

qmqtt

MQTT client for Qt
C++
693
star
7

emqtt-bench

Lightweight MQTT benchmark tool written in Erlang
Erlang
511
star
8

mqtt-jmeter

MQTT JMeter Plugin
Java
438
star
9

emqtt

Erlang MQTT 5.0 Client
Erlang
365
star
10

quic

QUIC protocol for Erlang & Elixir
Erlang
220
star
11

emqx-operator

A Kubernetes Operator for EMQX
Go
209
star
12

emqx-rel

Release Project for EMQX Broker prior to 4.3. Newer releases are built here: https://github.com/emqx/emqx
Shell
189
star
13

emqx-docs

EMQX product documentation
HTML
156
star
14

NanoSDK

NanoSDK - MQTT 5.0-compliant SDK with QUIC support in NNG flavor
C
126
star
15

mria

Asynchronously replicated Mnesia-like database for Erlang/Elixir
Erlang
117
star
16

qmqtt-client

MQTT Client GUI Written with Qt
C++
97
star
17

ekka

Autocluster and Autoheal for EMQX Broker
Erlang
94
star
18

emqx-web-hook

EMQX Webhook Plugin
Erlang
79
star
19

emqx-rule-engine

EMQX Rule Engine
Erlang
69
star
20

emqx-plugin-template

EMQ X Plugin Template and Demo
Erlang
63
star
21

emqx-coap

EMQX CoAP Gateway
Erlang
61
star
22

emqx-auth-mysql

Authentication, ACL with MySQL Database
Erlang
55
star
23

emqx-dashboard

EMQ X Dashboard
CSS
52
star
24

MQTT-Web-Toolkit

EMQ Web Toolkit
Vue
44
star
25

emqx-extension-examples

Examples for ExHook & ExProto
Erlang
44
star
26

emqx-auth-http

EMQX HTTP Authentication/ACL Plugin
Erlang
43
star
27

emqx-dashboard5

The management console used to monitor and manage EMQX clusters.
Vue
42
star
28

neuron-dashboard

Neuron Dashboard
TypeScript
41
star
29

minirest

A Mini RESTful API Framework
Erlang
37
star
30

emqx-bridge-mqtt

Bridge of MQTT (deprecated since EMQX v5)
Erlang
37
star
31

replayq

Generic on-disk persistent queue implementation for Erlang
Erlang
35
star
32

emqx-auth-pgsql

EMQ X PostgreSQL Authentication Plugin
Erlang
35
star
33

edge-stack

Shell
34
star
34

emqx-auth-redis

EMQ X Redis Authentication/ACL Plugin
Erlang
33
star
35

hocon

HOCON configuration Parser for Erlang/OTP
Erlang
32
star
36

mqttbs

MQTT Benchmark Suite
32
star
37

mqtt5.0-cn

MQTT Version 5.0 Chinese
31
star
38

emqx-docker

An Open-Source, Cloud-Native, Distributed MQTT Message Broker for IoT.
Shell
30
star
39

emqx-lwm2m

EMQ X LwM2M Gateway
Erlang
30
star
40

emqx-sn

EMQX MQTT-SN Gateway
Erlang
29
star
41

emqx-auth-jwt

EMQ X JWT Authentication Plugin
Erlang
28
star
42

emqx-lua-hook

EMQ X Lua Hook
Erlang
28
star
43

ecpool

Erlang Connection/Client Pool Library
Erlang
28
star
44

emqx-exporter

Prometheus exporter for EMQX
Python
26
star
45

emqx-dashboard-frontend

EMQ X Dashboard Frontend
Vue
25
star
46

emqx-prometheus

EMQX Statsd
Erlang
24
star
47

emqx-exhook

The Multiple-Language extension plugin for EMQ X Hooks
Erlang
23
star
48

blog

EMQ Blog
Python
22
star
49

emqx-retainer

EMQ X Retainer
Erlang
21
star
50

emqx-auth-mongo

EMQ X Authentication/ACL with MongoDB
Erlang
21
star
51

coap-jmeter

CoAP JMeter Plugin
Java
20
star
52

eip

EMQX Improvement Proposals
Makefile
19
star
53

emqx-elixir-plugin

An Example Plugin for EMQX in Elixir
Elixir
19
star
54

emqx-exproto

The Multiple-Language extension plugin for Protocol access
Erlang
18
star
55

openai-mqtt-nodejs

Node.js project that combines OpenAI's API with MQTT for real-time AI-driven interactions in IoT ecosystems.
JavaScript
18
star
56

emqx-auth-username

EMQ X Authentication with Username and Password
Erlang
17
star
57

mqtt-to-timescaledb

Ingesting time series data into TimescaleDB using MQTT and EMQX | MQTT Timescale Integration
Makefile
16
star
58

erlang-rocksdb

Upstream is here: https://gitlab.com/barrel-db/erlang-rocksdb
C++
16
star
59

emqx-auth-clientid

EMQ X ClientId Authentication Plugin
Erlang
16
star
60

jq

jq (nif) for beam
C
15
star
61

emqx-dashboard-web

Frontend Source Code for EMQ X dashboard Enterprise v4
Vue
15
star
62

emqx-tutorial-cn

EMQ X ไธญๆ–‡Tutorial
14
star
63

emqx-extension-java-sdk

The Java SDK for emqx-extension-hook
Java
14
star
64

emqttb

A scriptable autotuning load generator for MQTT
Erlang
14
star
65

neuron-docs

Neuron Documentation
Shell
14
star
66

emqx-extension-python-sdk

The Python SDK for emqx-extension-hook
Python
13
star
67

HBridge

High-performance MQTT Bridge written in Haskell
Haskell
12
star
68

cdk-emqx-cluster

Python
11
star
69

pulsar-client-erl

Apache Pulsar client for Erlang/Elixir
Erlang
11
star
70

emqx-management

EMQ X Management API
Erlang
10
star
71

wormhole

The channel for routing request from cloud to edge.
Go
10
star
72

local-storage-encryption

Obfuscate key-value pairs in localStorage.
JavaScript
10
star
73

deploy-emqx-to-aws-with-terraform

HCL
10
star
74

mqtt-to-kafka

Streaming IoT data into Confluent/Kafka using MQTT and EMQX | MQTT Kafka Integration
Makefile
10
star
75

emqx-ui

EMQX UI Library
Vue
9
star
76

cloud-docs

EMQX Cloud Documentations
CSS
9
star
77

emqx-meter-cn

EMQ X Meter Benchmark
Python
9
star
78

emqx-usage-example

Using Docker Compose to show different usages of EMQX
9
star
79

mqtt-to-influxdb

Ingesting time series data into InfluxDB using MQTT and EMQX | MQTT InfluxDB Integration
JavaScript
8
star
80

example

EMQ x IoT project sample code
Java
8
star
81

ehttpc

HTTP Client for Erlang/OTP
Erlang
7
star
82

mqtt-security-scanner

MQTT Security Scanner
Go
7
star
83

rebar3_emqx_plugin

A rebar3 plugin for automatically generate emqx plugin template
Erlang
7
star
84

ipqueue

High Performance Interleaved Priority Queue for Erlang/Elixir
Erlang
7
star
85

ecql

Erlang Cassandra CQL Driver
Erlang
7
star
86

deploy-emqx-to-alibabacloud-with-terraform

HCL
6
star
87

influxdb-client-erl

Erlang
6
star
88

lc

Load Control for beam
Erlang
6
star
89

emqx-recon

Recon debug/optimize plugin
Erlang
6
star
90

tf-emqx-performance-test

HCL
6
star
91

emqx-builder

Dockerfile
5
star
92

emqx-reloader

Erlang Module Reloader
Erlang
5
star
93

emqx-exproto-java-sdk

The Java SDK for emqx-exproto
Java
5
star
94

emqx-stomp

EMQ X Stomp Protocol Plugin
Erlang
5
star
95

emqx-webinars

Webinar materials
Shell
5
star
96

bootcamp

The learning center of EMQ products
C
5
star
97

mqtt-to-redis

Ingesting IoT data into Redis using MQTT and EMQX | MQTT Redis Integration
JavaScript
5
star
98

emqx-auth-ldap

EMQ X LDAP Authentication Plugin
Erlang
5
star
99

nanomq-docker

An ultra-lightweight and blazing-fast MQTT broker for IoT edge
Dockerfile
4
star
100

emqx-10million-test-cn

EMQ X 10MILLION TEST
4
star