• Stars
    star
    43
  • Rank 645,348 (Top 13 %)
  • Language
    Erlang
  • License
    Other
  • Created about 9 years ago
  • Updated about 5 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

distributed system stress and load testing tool

Typhoon





Makes latency assessments of distributed systems build from microservices.

Build Status GitHub release

Typhoon is a stress and load testing tool for distributed systems that simulates traffic from a test cluster toward a system-under-test (SUT) and visualizes infrastructure-, protocol- and application-related latencies. It provides an out-of-the-box, cross-platform solution for investigating protocols and microservices latencies, and is operable as a standalone application. For scalability and accuracy, its runtime environment is Erlang.

Key Features and Functionality

Typhoon uses Cubism.js to visualize latencies. The visualizations help you to make quick decisions on optimal software configuration, the number of servers and concurrent connections you need, and other short-term considerations. Long-term, they can inform how you develop and extend your data and service architecture, choose new technologies, etc.

Typhoon also:

  • provides a REST API.
  • uses pure functional expressions to define workload scenarios that don't require any compilation or native package builds; read more here.
  • scales up to dozens of individual nodes hosted in any cloud environment.
  • uses peer-to-peer clustering, based on consistent hashing, to assemble and orchestrate load toward SUT. It helps to deal with possible network failures, and provides high availability for synthetic load and telemetry collections.
  • approximates network delay, round-trip time, TLS handshakes, Time to First Byte and Time to Meaningful Response.
  • evaluates protocol overhead by approximating packet metrics.
  • estimates application performance.
  • validates system performance and scalability while spawning a huge number of concurrent sessions.
  • acceptance testing; read more here.

You can read more about Typhoon's features in this blog post.

Inspiration

Typhoon's architecture and design reflect the principles of incremental scalability, decentralization and optimistic replication. It derives inspiration from related efforts driven by Nokia (latency analysis on cellular networks), Google (web protocol enhancement and evolution), and other companies working in the mobile app space.

Getting Started

Typhoon supplies pre-built releases for Linux/x86_64, MacOS/10.10.x and Docker platforms. Instructions for using these binaries are on the GitHub releases page.

Build the latest version of Typhoon from the master branch. The build process requires Erlang/OTP version 18.0 or later. All development, including new features and bug fixes, take place on the master branch using forking and pull requests as described in these contribution guidelines.

Install directions with AWS-related details are here.

Running Typhoon

The easiest way to run a standalone instance is with the Docker container. The option is viable only if you've configured either Docker Toolbox or the Docker daemon. Use the latest release version:

docker run -it --name typhoon --rm -p 8080:8080 registry.opensource.zalan.do/hunt/typhoon:latest

This A) starts a single Typhoon node as a Docker container and B) exposes services using the REST API on port 8080. By default it is bound to localhost IP address on Mac OS and Linux. If you're using a different platform, please check your Docker configuration.

You can also spawn Typhoon using native platform binaries. See the installation instructions on the GitHub releases page:

/usr/local/typhoon-x.y.z foreground

This brings Typhoon up and running. The application uses local IP address 127.0.0.1 and port 8080 to offer services. Open the Typhoon dashboard http://localhost:8080 in your web browser to manage the workload and analyze the results.

Next, define a simple workload scenario and publish it to Typhoon:

docker exec -it typhoon tyrl \
   -i example \
   -H "Connection: keep-alive" \
   http://localhost:8080/health/peer

Open Typhoon dashboard at web browser http://localhost:8080 to analyze measurements: network delay, roundtrip time, TLS handshake, Time to First Byte, and Time to Meaningful Response. It also evaluates protocol overhead at this time by approximating packet metrics, and estimates application performance.

Congrats! You have successfully started a Typhoon, written a stress test scenario, deployed it to a cluster and analyzed your system's behavior.

Running a local Typhoon cluster

Use docker containers to spawn three node cluster on local environment. The following examples shows how to spawn cluster seeder and other nodes. Use the latest release version

Let's spawn a seeder node, this node is used by other Typhoon peers to discover each other.

docker run -d --name typhoon --rm -p 8080:8080 registry.opensource.zalan.do/hunt/typhoon:latest

Next, the identity of seeder node is needed to spawn other peers. We can use health check api for this

curl http://localhost:8080/health/peer

The application should return list of cluster peers ["[email protected]"]. This identity shall be passed to other containers within EK_SEED environment variable. The following example spawn more cluster peers:

docker run -d -p 8080 -e "[email protected]" registry.opensource.zalan.do/hunt/typhoon:latest
docker run -d -p 8080 -e "[email protected]" registry.opensource.zalan.do/hunt/typhoon:latest

We can validate that all peers joins the cluster using health check api

curl http://localhost:8080/health/peer

The application should return list of cluster peers ["[email protected]","[email protected]","[email protected]"].

Next steps

Contributing/Bugs

Typhoon is Apache 2.0 licensed and accepts contributions via GitHub pull requests:

  • Fork the repository on GitHub
  • Read the README.md for build instructions

commit message

The commit message helps us to write a good release note, speed-up review process. The message should address two question what changed and why. The project follows the template defined by chapter Contributing to a Project of Git book.

Short (50 chars or less) summary of changes

More detailed explanatory text, if necessary. Wrap it to about 72 characters or so. In some contexts, the first line is treated as the subject of an email and the rest of the text as the body. The blank line separating the summary from the body is critical (unless you omit the body entirely); tools like rebase can get confused if you run the two together.

Further paragraphs come after blank lines.

Bullet points are okay, too

Typically a hyphen or asterisk is used for the bullet, preceded by a single space, with blank lines in between, but conventions vary here

bugs

If you experience any issues with Typhoon, please let us know via GitHub issues. We appreciate detailed and accurate reports that help us to identity and replicate the issue.

  • Specify the configuration of your environment. Include which operating system you use and the versions of runtime environments.

  • Attach logs, screenshots and exceptions, in possible.

  • Reveal the steps you took to reproduce the problem.

Contacts

Changelog

Typhoon uses semantic versions to identify stable releases.

  • 1.0.0 - Simplify scenario format, New Single Page Application
  • 0.8.0 - hackweek UI improvments and bug fixes
  • 0.7.3 - support local clustering

License

Copyright 2015 Zalando SE

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0.

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

More Repositories

1

datalog

simplified query engine based on logic programming paradigm
Erlang
139
star
2

cache

Erlang in-memory cache
Erlang
136
star
3

datum

pure functional and generic programming for Erlang
Erlang
119
star
4

aws-cdk-pure

Purely Functional and high-order cloud components with AWS CDK
TypeScript
93
star
5

esq

simple persistent queues for Erlang
Erlang
54
star
6

golem

pure functional and generic programming for Go
Go
46
star
7

serverless

Serverless Erlang runtime for AWS Lambda Service
Erlang
29
star
8

csv

csv parser, optimized for performance
Makefile
22
star
9

dynamo

Generic Golang Key/Value trait for AWS storage services
Go
18
star
10

pts

The library provides hashtable-like interface to manipulate data distributed across Erlang processes.
Erlang
15
star
11

hash

collection of hash functions for Erlang applications
Erlang
12
star
12

feta

fogfish erlang toolkit archive
Erlang
10
star
13

node-lambda-typescript-template

The bare minimum for a TypeScript app running on Amazon Lambda.
TypeScript
10
star
14

guid

K-ordered unique identifiers in lock-free and decentralised manner for Golang applications
Go
10
star
15

makefile

Erlang workflow
Makefile
10
star
16

word2vec

Golang "native" implementation of word2vec algorithm (word2vec++ port)
C++
10
star
17

oauth2

oauth2 authorization server
Erlang
9
star
18

ek

Erlang clustering
Erlang
8
star
19

nebula

Erlang nodes discovery agent
Erlang
8
star
20

ddb

AWS Generic Storage Drivers
Erlang
8
star
21

relog

Redis datalog support
Erlang
8
star
22

m_http

A class of Erlang monads which can do http requests
Erlang
7
star
23

gurl

α΅πŸ†„πŸ†πŸ…» is a combinator library for network I/O
Go
6
star
24

blueprint-serverless-golang

AWS CDK template for serverless Golang
Go
6
star
25

hyperion

Erlang Vanilla Node
Erlang
6
star
26

uid

erlang fault tolerant service to generate unique identities
Erlang
6
star
27

svg

Erlang
5
star
28

certbot-on-aws

Serverless integration with https://letsencrypt.org for microservices hosted on AWS
Makefile
5
star
29

schemaorg

Go types of schema.org ontology
Go
5
star
30

semantic

Semantic Web ToolKit for Erlang applications
Erlang
4
star
31

socat

Command line utility to cat files via network socket
Erlang
4
star
32

ring

consistent hashing data structure
Go
4
star
33

elata

Erlang LATency Agent: software solution to gather telemetry of software components (written on erlang)
Shell
4
star
34

swirl

swirl is the Erlang port of whiskers.js template library.
Erlang
4
star
35

streamfs

Append-only file system access with stream abstraction
Erlang
3
star
36

esh

Erlang to Shell binding
Erlang
3
star
37

crdts

Convergent replicated data type for Erlang
Erlang
3
star
38

hnsw

Hierarchical Navigable Small World Graphs
Go
3
star
39

curie

The type CURIE (compact URI) for Golang
Go
3
star
40

erlang-in-docker

Erlang/OTP container
Dockerfile
3
star
41

elasticnt

N-triple intake to Elastic Search
Erlang
3
star
42

chronolog

time series database
Erlang
3
star
43

ecsd

AWS ECS microservice supervisor
Erlang
3
star
44

permit

a high-level api for security tokens management.
Erlang
3
star
45

stdio

Creating streams and performing input and output operations on them
Erlang
3
star
46

s3am

Stream I/O to aws s3 buckets
Erlang
3
star
47

faults

Type safe constructs to annotate Golang errors with the context
Go
3
star
48

gouldian

Go combinator library for building HTTP serverless applications
Go
3
star
49

datomic-aws

AWS Appliance to managed Datomic solutions
Shell
2
star
50

serverless-runtime

Experimental runtime for serverless Erlang application (Use fogfish/serverless) instead
Erlang
2
star
51

swarm

Go channels for distributed queueing and event-driven systems
Go
2
star
52

it

Human-friendly unit tests assertions for Go
Go
2
star
53

code-build-bot

Serverless CI/CD with AWS CodeBuild
TypeScript
2
star
54

aae

active anti-entropy library
Erlang
2
star
55

schemacli

schema.org ontology command-line
Go
2
star
56

d3

direct distributed dets interface
Erlang
2
star
57

tslab

A slab-like allocator library for generic type T
Go
2
star
58

dive

ephemeral and persistent b-tree interface
Erlang
1
star
59

esio

HTTP client to Elastic Search for Erlang application.
Erlang
1
star
60

homebrew-qemu-9pfs

Homebrew Tap install QEMU with 9P filesystem
Ruby
1
star
61

geojson

GeoJSON / RFC7946 codec for Golang
Go
1
star
62

kv8.rel

simple erlang benchmark utility
Shell
1
star
63

elasticlog

Elastic Search datalog support
Erlang
1
star
64

pq

Erlang process queues (pool of workers)
Erlang
1
star
65

znap

Build and replay snapshots from your asynchronous event stream(s)
Scala
1
star
66

ambit

Erlang
1
star
67

cryptex

Semi-automatic cipher for Algebraic Data Types in Golang
Go
1
star
68

erlcc

Erlang Code Compile: wrapper of native compile module
Makefile
1
star
69

tf-workspace

A personal workspace for TensorFlow.
Jupyter Notebook
1
star
70

runscript

tiny wrapper to run script / shell command as root
C
1
star
71

skiplist

Golang SkipList data structure
Go
1
star
72

clue

system and application status repository for Erlang
Erlang
1
star
73

clot

cloud toolkit for Erlang applications
Erlang
1
star
74

hornlog

define and evaluate horn clauses
Makefile
1
star
75

hexer

Hexastore: Sextuple Indexing for Semantic Web Data Management
Go
1
star
76

pns

process namespace
Erlang
1
star
77

go-check-updates

upgrades your go.mod dependencies to the latest versions
Go
1
star
78

stream

Golang file system abstraction tailored for AWS S3, enabling seamless streaming of binary objects along with their corresponding metadata.
Go
1
star