• Stars
    star
    179
  • Rank 207,208 (Top 5 %)
  • Language
    JavaScript
  • License
    MIT License
  • Created almost 12 years ago
  • Updated over 8 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

mutiplex-demultiplex multiple streams through a single text Stream

MuxDemux

multiplex-demultiplex object streams across any text stream.

[Build Status] (https://travis-ci.org/dominictarr/mux-demux)

Stability

Stable: Expect patches, possible features additions.

Example

var MuxDemux = require('mux-demux')
var net = require('net')

net.createServer(function (con) {
  con.pipe(MuxDemux(function (stream) {
    stream.on('data', console.log.bind(console))
  })).pipe(con)
}).listen(8642, function () {
  var con = net.connect(8642), mx
  con.pipe(mx = MuxDemux()).pipe(con)

  var ds = mx.createWriteStream('times')

  setInterval(function () {
    ds.write(new Date().toString())
  }, 1e3)
})

Binary Support

By encoding JSON as msgpack, binary is supported. just require from mux-demux/msgpack instead of mux-demux.

var MuxDemux = require('mux-demux/msgpack')

Gotchas

take care to create a MuxDemux instance per connection, do not connect many connections to one `MuxDemux'.

Right

net.createServer(function (stream) {
  stream.pipe(MuxDemux(function (_stream) {

  }).pipe(stream)
}).listen(port)

WRONG!

var mx = MuxDemux()
net.createServer(function (stream) {
  //this will connect many streams to the OUTER MuxDemux Stream!
  stream.pipe(mx).pipe(stream)
}).listen(port)

Right

var mx = MuxDemux()
mx.close(function () {
  console.log("connection close");
})

var A = mx.createStream()
A.end()

WRONG!

var mx = MuxDemux()

var A = mx.createStream()
A.end()

mx.close(function () {
  console.log("connection close");
})

actually this will not close mx

Errors, and use in PRODUCTION

mux-demux parses a JSON protocol, and so you must handle any errors that may result from someone connecting, and sending invalid data.

net.createServer(function (stream) {
  var mx = MuxDemux()
  stream.pipe(mx).pipe(stream)
  mx.on('error', function () {
    stream.destroy()
  })
  stream.on('error', function () {
    mx.destroy()
  })
}).listen(9999)

#API

the API browser-stream

var MuxDemux = require('mux-demux')
var a = MuxDemux()
var b = MuxDemux()

a.pipe(b).pipe(a)

b.on('connection', function (stream) {
  // inspect stream.meta to decide what this stream is.
})

a.createWriteStream(meta)
a.createReadStream(meta)
a.createStream(meta)

there is actually no distinction between clients and servers. if both sides are listening on('connection',...) then both sides may call create{Write,Read,}Stream(meta) and initiate new streams.

MuxDemux(options, onConnection)

Creates a MuxDemux stream. Optionally pass in an options hash

{
    error: Boolean,
    wrapper: function (stream) {...}
}

If the error option is set to true then MuxDemux will emit errors on the streams on unexpected disconnects. othewise, it will just emit 'end' on those streams.

wrapper be used to change the serialization format used by mux-demux, by default, line seperated json is used. see examples below both mux-demux end points must use the same wrapper.

options is optional. MuxDemux(onConnection) is a shortcut for MuxDemux().on('connection', onConnection)

createReadStream (meta)

open a ReadableStream from the other side. returns a ReadableStream. the other side of connection will emit a writable stream that is connected to this stream.

createWriteStream (meta)

open a WritableStream to the other side. returns a WritableStream, the other side will emit a ReadableStream connected to this stream.

createStream (meta, opts)

open a Stream to the other side which is both readable and writable. returns a Stream, the other side will emit a Stream connected to this stream.

opts may be {allowHalfOpen: true}, if this is not set, the stream will emit 'end' when end() is called. this may cause the stream to loose some data from the other end. If allowHalfOpen is true then the remote end must call end().

note to self, references to a class (Stream) should be capitalized, and in backticks. references to an instance should be lowercase, and not in backticks unless refuring to a specific variable in a code example.

close(cb)

asks mux-demux to emit end once all the sub-streams have closed. this will wait untill they have ended, closed, or errored, just like net.Server#close.

Takes an optional callback, and emits the 'end' event.

Wrapper Examples

A stream of plain old js objects.

new MuxDemux({wrapper: function (stream) { return stream } })

A stream of msgpack.

var es = require('event-stream')
var ms = require('msgpack-stream')

new MuxDemux({wrapper: function (stream) {
  return es.pipeline(ms.createDecodeStream(), stream, ms.createEncodeStream())
}})

MuxDemuxStream#error

there is one addition to the stream interface. call stream.error(err) will send an error that will be emitted at the other side of the stream. this is useful for sending 404 like messages to clients, etc.

More Repositories

1

event-stream

EventStream is like functional programming meets IO
JavaScript
2,189
star
2

JSON.sh

a pipeable JSON parser written in Bash
Shell
1,981
star
3

JSONStream

rawStream.pipe(JSONStream.parse()).pipe(streamOfObjects)
JavaScript
1,900
star
4

scuttlebutt

peer-to-peer replicatable data structure
JavaScript
1,310
star
5

rc

The non-configurable configuration loader for lazy people.
JavaScript
995
star
6

crdt

Commutative Replicated Data Types for easy collaborative/distributed systems.
JavaScript
836
star
7

through

simple way to create a ReadableWritable stream that works
JavaScript
667
star
8

your-web-app-is-bloated

measuring memory usage of popular webapps
514
star
9

npmd

JavaScript
450
star
10

split

JavaScript
346
star
11

curry

simple curry module, with nothing *too clever*, and full test coverage
JavaScript
314
star
12

random-name

JavaScript
296
star
13

hashlru

JavaScript
237
star
14

wifi.sh

Shell
216
star
15

level-sublevel

no longer maintained, sorry!
JavaScript
194
star
16

noderify

official fork: https://github.com/staltz/noderify
JavaScript
157
star
17

feedopensource

Iteratively Fund Open Source Projects With Bitcoin
JavaScript
142
star
18

excel-stream

JavaScript
137
star
19

stream-spec

executable specification for Stream (make testing streams easy)
JavaScript
125
star
20

map-stream

JavaScript
122
star
21

map-reduce

async map-reduce functions for nodejs
JavaScript
121
star
22

cyphernet

115
star
23

observable

A Mutable Value represented as a Function.
HTML
111
star
24

stream-combiner

JavaScript
103
star
25

rpc-stream

JavaScript
98
star
26

bench-lru

JavaScript
85
star
27

pull-box-stream

One way streaming encryption based on libsodium's secretbox primitive
JavaScript
84
star
28

level-live-stream

JavaScript
79
star
29

stack-expression

inspired by regular expressions but can do nested structures
JavaScript
77
star
30

hipster

JavaScript
72
star
31

snob

distributed version control system implemented in javascript.
JavaScript
71
star
32

xdiff

diff complex javascript objects
JavaScript
70
star
33

from

Easy way to create a Readable Stream
JavaScript
70
star
34

scalable-secure-scuttlebutt

HTML
68
star
35

explain-error

JavaScript
67
star
36

fsm

Finite State Machines in javascript
JavaScript
66
star
37

r-edit

JavaScript
64
star
38

readme

JavaScript
62
star
39

tiles

JavaScript
61
star
40

indexhtmlify

JavaScript
59
star
41

tacodb

JavaScript
57
star
42

adiff

diff and patch operations on arrays.
JavaScript
57
star
43

map-filter-reduce

JavaScript
57
star
44

browser-stream

open pipable streams to and from the browser, with Socket.io
JavaScript
55
star
45

reconnect

JavaScript
53
star
46

electro

JavaScript
51
star
47

level-replicate

JavaScript
51
star
48

d64

JavaScript
50
star
49

on-change-network

JavaScript
49
star
50

lock

lock asynchronous resources
JavaScript
47
star
51

crypto-bench

HTML
47
star
52

monotonic-timestamp

JavaScript
44
star
53

mynosql

JavaScript
44
star
54

pause-stream

JavaScript
43
star
55

json-select

JavaScript
43
star
56

json-buffer

JavaScript
41
star
57

bittodo

JavaScript
40
star
58

stream-punks

discussion repo for streams
39
star
59

charwise

JavaScript
39
star
60

proxy-by-url

custom logic for node-http-proxy to proxy based on incoming url
JavaScript
38
star
61

sentimental-versioning

version numbers with meaning
HTML
38
star
62

coherence

JavaScript
38
star
63

level-hooks

JavaScript
37
star
64

sodium-browserify

JavaScript
37
star
65

secret-handshake-paper

TeX
36
star
66

browselectrify

create browserify bundle that also works in electron
JavaScript
36
star
67

kv

simple kv store for streams
JavaScript
35
star
68

c2wasm

C++
35
star
69

level-trigger

triggers for levelup
JavaScript
33
star
70

deploy

scripts to setup continuous deployment with git push
Shell
33
star
71

presentations

JavaScript
32
star
72

rumours

Intergration of scuttlebutt family.
JavaScript
32
star
73

web-bootloader

HTML
28
star
74

remote-events

connect EventEmitters through Streams.
JavaScript
28
star
75

indexes-of

JavaScript
27
star
76

mpg123

JavaScript
27
star
77

level-master

JavaScript
27
star
78

what-is-scuttlebutt

spec for defining "scuttlebutt" as a living changing protocol
27
star
79

h

JavaScript
26
star
80

testbed

continuous integration for nodejs
JavaScript
25
star
81

canvas-browserify

HTML
25
star
82

level-merkle

JavaScript
25
star
83

it-is

assertion DSL based on functional idioms.
JavaScript
25
star
84

semver-ftw

Simple Description of SemVer
HTML
25
star
85

level-inverted-index

JavaScript
24
star
86

computer-modern

CSS
24
star
87

hyperaudio

JavaScript
24
star
88

level-scuttlebutt

leveldb persistence for scuttlebutts (scuttlebutt/crdt/append-only and friends)
JavaScript
24
star
89

level-search

JavaScript
24
star
90

level-couch-sync

JavaScript
23
star
91

simple-xlsx

maintained fork is at https://github.com/zeke/simple-xlsx
JavaScript
23
star
92

content-addressable-store

JavaScript
23
star
93

shasum

JavaScript
23
star
94

ticket-auth

JavaScript
22
star
95

ssh-key-to-pem

JavaScript
21
star
96

private-groups-paper

21
star
97

scuttlebucket

JavaScript
21
star
98

looper

JavaScript
20
star
99

deterministic-tar

JavaScript
20
star
100

npm-browserify

JavaScript
20
star