• Stars
    star
    174
  • Rank 219,104 (Top 5 %)
  • Language
    Erlang
  • License
    Apache License 2.0
  • Created almost 7 years ago
  • Updated 8 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Distributed Erlang without the EPMD

Erlang (and Elixir) distribution without EPMD, aka EPMDLESS

Shellcheck Hex pm

Allows to connect erlang nodes, via Erlang distribution and without epmd, using tcp or tls.

Requirements

  • Erlang >= 21

Usage

epmdless has 2 options for how ports are handled when Erlang distribution requests the port to use to connect to another node. With epmdless_client (variable ports) each node must first be added with a call to epmdless_client:add_node and this information is not automatically propagated between nodes like with EPMD, so a adding b and c and then connecting to them will not result in b and c creating a full mesh unless one of them has the other added to its mapping of nodes to ports with epmdless_client:add_node.

The other option is epmdless_static (static port). With this module the same port is used for every node it attempts to connect to, meaning when a connects to b and c a full mesh will be created because b will be able to look up c's port and connect to it.

Static Port

As of OTP-23.0 EPMD is still required to use a static port for distribution. This will be changed in an upcoming release, but for OTP releases before the upcoming support and for more dynamic options (see below) epmdless can be used and will continue to work on newer OTP-23.x versions.

At this time rebar3 nightly is required for generating a release that will have working remote console and rpc.

Docker Usage: Connecting Nodes

An example project examples/erlang_docker_example contains a project setup to use epmdless_static as the epmd_module and a docker-compose.yml config that will bring up multiple nodes that all use the same port for distribution each on the same network with specific IP addresses.

The release's vm.args.src uses epmdless_static for epmd_module which results in the same port being used for any lookup of a node. Since the same port is used for each node the port is simply set in vm.args.src with -erl_epmd_port 8001:

-sname epmdless_test

-setcookie epmdless_test

-start_epmd false
-epmd_module epmdless_static
-erl_epmd_port 8001

In docker-compose.yml 3 nodes are created node_a, node_b and node_c:

  node_a:
    container_name: node_a
    hostname: node_a
    build: .

After running docker-compse up we can show that the full mesh network is created in this case when we connect node_a to node_b and node_c:

$ docker exec -ti node_a bin/epmdless_test remote_console

(epmdless_test@node_a)1> net_adm:ping(epmdless_test@node_b).
pong
(epmdless_test@node_a)2> net_adm:ping(epmdless_test@node_c).
pong
(epmdless_test@node_a)3> nodes().
[epmdless_test@node_b,epmdless_test@node_c]

Now open a remote_console on node_c to see that it is connected to both node_a and node_b:

$ docker exec -ti node_c bin/epmdless_test remote_console

(epmdless_test@node_c)1> nodes().
[epmdless_test@node_a,epmdless_test@node_b]

Release: Variable Ports

Below in section Manual Usage you'll find details on how to use epmdless without building a release, this is useful for playing around or developing on epmdless, in this section we will be using a release built with the latest rebar3 nightly, OTP-23 and

The example project is under examples/erlang_variable_ports_example. The rebar3 configuration shows how to include epmdless in your project and release:

{erl_opts, [debug_info]}.
{deps, [epmdless]}.

{relx, [{release, {epmdless_test, "0.1.0"},
         [epmdless,
          epmdless_test]},
        {dev_mode, false},
        {include_erts, true},
        {extended_start_script, true},
        {vm_args_src, "config/vm.args.src"}]}.

Note that epmdless is kept in the release's list of applications and not in a .app.src list of applications. This is because it is not an actual runtime dependency of the application epmdless_test but a dependency of the particular release that applications is being used in.

In vm.args.src you'll find epmdless_client being setup as the epmd_module and the epmd daemon being disabled:

-sname ${NAME}@localhost

-setcookie epmdless_test

-start_epmd false
-epmd_module epmdless_client

Note that in this example there is no -erl_epmd_port in the vm.args.src because we will not use the same port for every node.

Build the release as usual:

$ rebar3 release
===> Plugin rebar3_hex not available. It will not be used.
===> Verifying dependencies...
===> Compiling epmdless
===> Compiling epmdless_test
===> Assembling release... epmdless_test-0.1.0
===> Warnings generating release:
*WARNING* Missing application sasl. Can not upgrade with this release
===> Release successfully assembled: _build/default/rel/epmdless_test

To run a release on a specific port set the ERL_DIST_PORT environment variable and run the console:

$ NAME=a ERL_DIST_PORT=8081 _build/default/rel/epmdless_test/bin/epmdless_test console
(a@localhost)1>

Connecting with a remote shell works the same:

$ NAME=a ERL_DIST_PORT=8081 _build/default/rel/epmdless_test/bin/epmdless_test remote_console
(a@localhost)1>

Or running an RPC:

$ NAME=a ERL_DIST_PORT=8081 _build/default/rel/epmdless_test/bin/epmdless_test rpc 'erlang nodes [hidden]'
[c371@rosa]

In this case, since the release is running on OTP-23, erl_call is used for rpc and eval commands and assign themselves a hidden nodename and in this case it gives itself the name c371@rosa.

Run another instance on a separate port and connect it to the first

$ NAME=b ERL_DIST_PORT=8082 _build/default/rel/epmdless_test/bin/epmdless_test console
(b@localhost)1> epmdless_client:add_node(a@localhost, 8081).
ok
(b@localhost)2> epmdless_client:list_nodes().
[{{"a",{127,0,0,1}},{"localhost",8081}}]
(b@localhost)3> net_adm:ping(a@localhost).
pong
(b@localhost)4> nodes().
[a@localhost]

Since the mapping of nodes to ports must be kept manually by calling add_node/2 if a third node c is added and connected to a it will not create a full mesh as you'd expect with regular usage of Erlang distribution and EPMD. That is, unless you first use add_node/2 to add b and its port to the mapping in c, then it will be able to automatically create the full mesh.

Manual Usage

The following details how to use epmdless with erl directly.

Example Usage with Multiple Ports

To play around with epmdless and get a feel for how it works the simplest way is to run a couple separate erl shells and connect them.

Only run the Erlang shell commands after havng start both a, b, and c, each on a different port 8001, 8002 and 8003:

ERL_DIST_PORT=8002 erl -sname b@localhost -start_epmd false -epmd_module epmdless_client -pa _build/default/lib/epmdless/ebin

> 
ERL_DIST_PORT=8001 erl -sname a@localhost -start_epmd false -epmd_module epmdless_client -pa _build/default/lib/epmdless/ebin

(a@localhost)1> epmdless_client:add_node(b@localhost, 8002).
ok
(a@localhost)2> epmdless_client:list_nodes().
[{{"b",{127,0,0,1}},{"localhost",8002}}]

ERL_DIST_PORT=8003 erl -sname c@localhost -start_epmd false -epmd_module epmdless_client -pa _build/default/lib/epmdless/ebin

(c@localhost)1> epmdless_client:add_node(a@localhost, 8001).                                                ok
(c@localhost)2> epmdless_client:list_nodes().
[{{"a",{127,0,0,1}},{"localhost",8001}}]
(c@localhost)3> net_adm:ping(a@localhost).
pong
(c@localhost)4> 2020-05-17T09:14:44.300102-06:00 error: ** Cannot get connection id for node c@localhost
2020-05-17T09:14:51.299676-06:00 warning: global: c@localhost failed to connect to b@localhost
(c@localhost)5> nodes().
[a@localhost]
(c@localhost)6> epmdless_client:list_nodes().
[{{"a",{127,0,0,1}},{"localhost",8001}}]

Node a is connected to b and c but c and b are not able to connect to each other because they have no had their respective ports added to the epmdless_client state with add_node/2. This results in the warnings about failed to connect to b@localhost on node c.

Connecting to epmdless nodes

As of OTP-23 there is an option -dist_listen which keeps a remote shell from binding to a listen port -- and implies -hidden so this argument is not needed. Before this option a port had to be bound to even when wanting to simply use erl -remsh a@localhost. This complicated use of epmdless where we want to tell it the port of the remote node a@localhost and not care about any port for the new node that will make the remote connection.

So with OTP-23 it is possible to use the same ERL_DIST_PORT environment variable as used above but in this case it will not be used to listen but only to connect to a@localhost:

$ ERL_DIST_PORT=8001 erl -dist_listen false -remsh a@localhost -epmd_module epmdless_client -pa _build/default/lib/epmdless/ebin

(a@localhost)1> nodes(hidden).
['NAXDANHFJWVF@rosa']

For backwards compatibility reasons the environment variable EPMDLESS_REMSH_PORT is also still supported and can be used in place of ERL_DIST_PORT and will need to if on OTP before 23 because the new node will have to bind to a listen port.

Also in OTP-23 the option -address [Hostname:]Port was added to erl_call, a program for communicating with a distributed Erlang node. This is how the relx script does the rpc and eval commands if run on OTP-23:

$ _build/default/rel/epmdless_test/erts-11.0/bin/erl_call  -r -address :8081 -c epmdless_test -a 'erlang nodes [hidden]'
[c899@rosa]

With OTP-22 and earlier a remote shell requires setting EPMDLESS_REMSH_PORT to the port the node you want to connect to is using and optionally give a port for ERL_DIST_PORT -- if not given then 0 is uesd and a random port is chosen:

$ EPMDLESS_REMSH_PORT=8081 erl -sname a_remsh@localhost -remsh epmdless_test@localhost -epmd_module epmdless_client -hidden -setcookie epmdless_test -pa _build/default/checkouts/epmdless/ebin

(epmdless_test@localhost)1> nodes(hidden).
[a_remsh@localhost]

Tests

Testing is done with shelltestrunner and an example project under shelltests/epmdless_test. From that directory run shelltest -c --diff --all --execdir -- epmdless_test.test. It is also run with the latest rebar3 and relx in github actions for this repo.

Other example projects

Note: These will be updated for the latest epmdless options and moved to the examples/ directory. But for now refer to these projects

Please also refer: https://github.com/oltarasenko/erlang_distribution_in_docker for an example of complete project running epmdless.

TLS example for Elixir

Here is a small project which shows how to setup EPMDLess with TLS for Elixir: https://github.com/oltarasenko/epmdless-elixir-example

(Please also see a discussion here: #11)

More Repositories

1

erlastic_search

An Erlang app for communicating with Elastic Search's rest interface.
Erlang
162
star
2

grpcbox

Erlang grpc on chatterbox
Erlang
138
star
3

kuberl

Erlang Kubernetes client
Erlang
53
star
4

erl_tidy

Automatically format Erlang code.
Erlang
46
star
5

epubnub

Erlang PubNub API
Erlang
28
star
6

OpaDo

Opa port of the TodoMVC app
Opa
23
star
7

rebar3_run

Run a release with one simple command
Makefile
23
star
8

augle

Auth + Google = Augle
Erlang
22
star
9

grafana-operator

Grafana Operator creates and manages Grafana deployments on Kubernetes.
Go
18
star
10

eCloudEdit

A port of CloudEdit to use Webmachine instead of Rails
JavaScript
16
star
11

sctp_dist

Erlang
16
star
12

epmdlessless

Erlang
15
star
13

ramler

Erlang generators for RAML
Erlang
14
star
14

erlup

Erlang toolchain installer
Rust
14
star
15

pg_types

Erlang library for encoding and decoding postgres data types
Erlang
13
star
16

eindexer

Simple search engine written in Erlang
Erlang
12
star
17

throttle

Library for checking if a process is overloaded
Erlang
11
star
18

rebar3_erlydtl_plugin

Rebar3 Erlydtl Plugin
Erlang
11
star
19

rebar3.org

Rebar3.org
CSS
10
star
20

ctx

Erlang context library
Erlang
10
star
21

vanguard

Discover Cadre in the Vanguard
Erlang
10
star
22

dynamo_heroku_demo

Sample Heroku app for Elixir web framework Dynamo
Elixir
10
star
23

providers

Providers provide.
Erlang
10
star
24

jsonlog

A custom formatter for the Erlang logger that converts logs to json
Erlang
10
star
25

beerenthusiasts

Beer Enthusiasts is the easiest way to share, discuss and discover new recipes for homebrewing beer. Built using Erlang, Yaws, Nitrogen and CouchDB.
JavaScript
10
star
26

grpcbox_plugin

Rebar3 plugin for generating grpcbox behaviours
Erlang
9
star
27

erls

Build and manage multiple Erlang versions
Rust
9
star
28

egit_sshd

SSH dameon for hosting git repos
Erlang
7
star
29

docker_compose_cth

Erlang Common Test Hook for starting docker-compose services before running test suites
Erlang
7
star
30

heroku_weber_example

Weber example for Heroku
Elixir
7
star
31

rebar_docker

rebar3 plugin for generating docker images
Erlang
7
star
32

elna

Erlang HTTP 1.1 and 2 server using Elli and Chatterbox
Erlang
6
star
33

heroku-buildpack-opa

Opa buildpack for Heroku
Shell
6
star
34

otel_getting_started

Erlang
6
star
35

mirrormaster

Hex mirror and private repo
Erlang
6
star
36

kakapo

riak_core based router
Erlang
6
star
37

wts

Warping the Timestamps
Erlang
6
star
38

erlangdc2013

Erlang on Heroku Workshop ErlangDC 2013
Erlang
5
star
39

rebar3_tests

Rebar3 ShellTestRunner tests
Erlang
5
star
40

chatterbox_tiles

Erlang
5
star
41

mmmbot

Erlang Irc bot
Erlang
5
star
42

epubnub_chat

And example for using epubnub to talk to the example chat app on http://www.pubnub.com/blog/build-real-time-web-apps-easy
Erlang
5
star
43

rebar_alias

Rebar3 plugin for defining aliases
Erlang
5
star
44

elli_chatterbox

Combine Elli and Chatterbox into one http monster
Erlang
4
star
45

cs542-prims-algorithm

CS542 Project: Implementation of Prim's Algorithm
Erlang
4
star
46

dwight

Hermes' son
Erlang
4
star
47

webmachine_examples

Webmachine Examples
Erlang
4
star
48

rebar_publish

Publish apps
Erlang
4
star
49

postgres_pool_example

Erlang Postgres Connection Pool Example
Erlang
4
star
50

cowboy_template

Cowboy app template
Erlang
4
star
51

erlchestrate

Erlang client for orchestrate.io
Erlang
4
star
52

rebar3_elixirc

Erlang
4
star
53

logfmt-erlang

Logfmt log parsing for Erlang
Erlang
4
star
54

odoric

Odoric
Erlang
3
star
55

elli_template

Elli template for Rebar3
Erlang
3
star
56

opa_pubnub_plugin

Opa plugin for client side interaction with PubNub
JavaScript
3
star
57

rebar3_ex_compiler

Example rebar3 compile provider
Erlang
3
star
58

ceug_4_2012

Chicago Erlang User Group April, 2012 Presentation on Cowboy and Batman.js
3
star
59

CEUG

Chicago Erlang User Group Demos
JavaScript
3
star
60

session_8

ErlangCamp Session 8
Erlang
3
star
61

riak_pb_msgcodegen

Erlang
3
star
62

roll_dice

Dice rolling OpenTelemetry Erlang example
Erlang
2
star
63

erlanger_evolution

The evolution of an erlang programmer. From non-OTP to OTP.
Erlang
2
star
64

rebar3_orb

Orb for building and testing rebar3 projects in CircleCI
2
star
65

lambda_jam_2013

Lambda Jam 2013 Presentation
Ruby
2
star
66

erl_new

Script for Erlang templates
Shell
2
star
67

ember_template

Erlang Ember App Template
JavaScript
1
star
68

lambda_jam_2013_maze

Maze
Erlang
1
star
69

heroku-buildpack-erlang-dialyzer

Run Dialyzer on your project and host the results
Shell
1
star
70

seresye.agner

Seresye package
1
star
71

gen_leader_zk

Erlang
1
star
72

eMmmbot

New mmmbot web interface and backend
JavaScript
1
star
73

county

Sheriff patrols the County
Erlang
1
star
74

reports

JavaScript
1
star
75

erlangdc2013_workshop

Workshop for ErlangDC2013
Ruby
1
star
76

orchestrate-erlang-client

A super simple (more planned) Erlang client to query the Orchestrate.io service.
Erlang
1
star
77

elenchus

JavaScript
1
star
78

dep_tracking

Dependency tracking example for rebar3
Erlang
1
star
79

mashape-erlang-client-library

Mashape Erlang client library
Erlang
1
star
80

jqheroku

Webapp to fetch json from a url and run a jsonpath against it
Erlang
1
star
81

erlware_commons.agner

Erlware Commons agner package
1
star
82

beamup

Install and control active BEAM languages
Rust
1
star