• Stars
    star
    561
  • Rank 77,586 (Top 2 %)
  • Language
    JavaScript
  • Created over 12 years ago
  • Updated almost 4 years ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

A caffeine driven, simplistic approach to benchmarking.

Matcha

A caffeine driven, simple approach to benchmarking.

Matcha allow you to design experiments that will measure the performance of your code. It is recommended that each bench focus on a specific point of impact in your application.

Matcha Report

Installation

Matcha is available on npm.

  $ npm install matcha

Writing Async Benchmarks

Though suites/benches are executed serially, the benches themselves can be asyncronous. Furthermore, suites ran with the matcha command line runner have a number of globals to simplify bench definitions. Take the following code, for example:

suite('Make Tea', function () {
  var tea = new CupOfTea('green');

  bench('boil water', function(next) {
    tea.boil('85℃', function (err, h20) {
      // perfect temperature!
      next();
    });
  });

  // add tea, pour, ...  

  bench('sip tea', function() {
    tea.sip('mmmm');
  });
});

Async vs. Sync

Since boiling water takes time, a next function was provided to each iteration in our bench to be called when the async function completes. Since the consumption of tea provides instant gratification, no next needed to be provided, even though we still wish to measure it.

Setup/Teardown

Arbitray functions may be specified for setup or teardown for each suite by using the before and after keywords. These function may be sync or async.

suite('DB', function() {
  before(function(next) {
    db.connect('localhost:9090', next);
  });

  bench(function(next) {
    // ...
  });

  after(function() {
    db.close();
  });
});

Setting Options

As not all code is equal, we need a way to change the running conditions for our benches. Options can currently be changed for any given suite, and will be retained for any nested suites or benches of that suite.

To set an option:

suite('Make Tea', function () {
  set('iterations', 10000);
  set('type', 'static');
  // ...
Defaults

Here are all available options and the default values:

set('iterations', 100);     // the number of times to run a given bench
set('concurrency', 1);      // the number of how many times a given bench is run concurrently
set('type', 'adaptive');    // or 'static' (see below)
set('mintime', 500);        // when adaptive, the minimum time in ms a bench should run
set('delay', 100);          // time in ms between each bench
Static vs Adaptive

There are two modes for running your benches: 'static' and 'adaptive'. Static mode will run exactly the set number of iterations. Adaptive will run the set iterations, then if a minimal time elapsed has not passed, will run more another set of iterations, then check again (and repeat) until the requirement has been satisfied.

Running Benchmarks

Running of your benchmarks is provided through ./bin/matcha. The recommended approach is to add a devDependancy in your package.json and then add a line to a Makefile or build tool. The matcha bin will accept a list of files to load or will look in the current working directory for a folder named benchmark and load all files.

  $ matcha suite1.js suite2.js

Options

    -h, --help               view matcha usage information
    -v, --version            view matcha version
    -R, --reporter [clean]   specify the reporter to use
    -I, --interface [bdd]    specify the interface to expect
    --interfaces             display available interfaces
    --reporters              display available reporters

-I, --interface

The --interface option lets you specify the interface to use, defaulting to "bdd".

-R, --reporter

The --reporter option allows you to specify the reporter that will be used, defaulting to "clean".

Interfaces

Matcha "interface" system allows developers to choose their style of DSL. Shipping with bdd, and exports flavoured interfaces.

bdd

suite('suite name', function(){
    set('iterations', 10);
    bench('bench name', function(done){
        some_fn(done);
  });
});

exports

exports['suite name'] = {
    options: {
      iterations: 10
    },
    bench: {
        'bench name': function (doen) {
            some_fn(done);
        }
    }
};

Reporters

Matcha reporters adjust to the terminal window.

clean

Good-looking default reporter with colors on the terminal screen.

plain

Similar to clean reporter but without colors and other ANSI sequences.

csv

Completely different, create csv formated rows for later processing.

Contributing

Interested in contributing? Fork to get started. Contact @logicalparadox if you are interested in being regular contributor.

Contibutors

Shoutouts

  • mocha inspired the suite/bench definition language.

License

(The MIT License)

Copyright (c) 2011-2012 Jake Luer [email protected]

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

More Repositories

1

apnagent

Node.js adapter for Apple Push Notification (APN) Service.
JavaScript
216
star
2

codex

Static site and documentation generator. Markdown for content. Jade/stylus for templates.
JavaScript
64
star
3

apnagent-ios

iOS application for testing Apple Push Notification (APN) service payloads.
Objective-C
46
star
4

drag.js

DEPRECIATED: A micro javascript utility for ui drag behavior.
JavaScript
42
star
5

electron

DEPRECIATED: A simple command-line interface framework for node.js.
JavaScript
34
star
6

filtr

Array filtering for node.js and the browser inspired by MongoDB.
JavaScript
24
star
7

v8-argv

Proxy v8 argv (including harmony) to v8/node while forwarding the remaining arguments to a custom script.
JavaScript
19
star
8

apnagent-playground

Sample application for article: "Delivering iOS Push Notifications with Node.js"
JavaScript
18
star
9

codex-hub

Codex template for node.js/javascript GitHub project documentation sites.
JavaScript
15
star
10

oath

Thunk-based flow control with fallback support for callbacks.
JavaScript
13
star
11

fbagent

Thin & stateless Facebook Graph wrapper for node.
JavaScript
13
star
12

folio

Tiny static javascript build and serve utility.
JavaScript
12
star
13

puppet-nodejs

Puppet manifest for installating node.js from source download.
Puppet
11
star
14

exifdata

Read and (soon) write JPEG EXIF metadata in pure JavaScript for Node.js.
JavaScript
11
star
15

tea-error

Custom error constructor factory.
JavaScript
11
star
16

transmute

No-inheritance transform streams.
JavaScript
10
star
17

createsend

Abandoned. Use:
JavaScript
6
star
18

comm

Rust-inspired communication library for asynchronous message passing.
JavaScript
6
star
19

dragonfly

Tiny error manager for big applications.
JavaScript
4
star
20

readdir-stream

Recursively read a directory and stream stats objects.
JavaScript
4
star
21

kettle.js

Javascript scroll widgets based on drag.js.
4
star
22

hakaru

Measure everything.
JavaScript
4
star
23

puppet-redis

A puppet manifest for redis install from source.
Puppet
3
star
24

chai-getting-started

JavaScript
3
star
25

gryn

Procfile process management on node.
JavaScript
3
star
26

co-mutex

Mutex primitive for generator flow control.
JavaScript
3
star
27

fez

css3 framework in stylus
JavaScript
2
star
28

co-semaphore

Counted semaphore primitive for generator flow-control.
JavaScript
2
star
29

pcrypt

Password salt/hash encryption and comparison using node's pbkdf2.
JavaScript
2
star
30

roundabout

Stack several transform streams behind a single stream for reusable piping magic.
JavaScript
2
star
31

fsagent

A few extras for the node.js `fs` module.
JavaScript
2
star
32

argvee

argv parsing (that is actually useful)
TypeScript
2
star
33

flynn

Process/daemon manager.
JavaScript
1
star
34

geospatial

JavaScript
1
star
35

co-condvar

Conditional variable primitive for generator flow-control.
JavaScript
1
star
36

bufs

Manage collections of buffers as a single buffer.
JavaScript
1
star
37

contraption

Finite State Machines for Node.js
JavaScript
1
star
38

logicalparadox-site

... by Codex
JavaScript
1
star
39

.dockerfiles

All my Dockerfiles.
Shell
1
star
40

co-future

Wrap a generator function in a future for fine-grained resolution control.
JavaScript
1
star
41

goodwin

JavaScript
1
star
42

geek

Some tools for node geeks.
JavaScript
1
star
43

.redux

The (new) .dotfiles.
Python
1
star