• Stars
    star
    103
  • Rank 333,046 (Top 7 %)
  • Language
    JavaScript
  • License
    Apache License 2.0
  • Created over 5 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

πŸ€– Repeat tests. Repeat tests. Repeat tests.
test-each logo

Node Browsers TypeScript Codecov Minified size Mastodon Medium

πŸ€– Repeat tests. Repeat tests. Repeat tests.

Repeats tests using different inputs (Data-Driven Testing):

  • test runner independent: works with your current setup
  • generates test titles that are descriptive, unique, for any JavaScript type (not just JSON)
  • loops over every possible combination of inputs (cartesian product)
  • can use random functions (fuzz testing)
  • snapshot testing friendly

Example

// The examples use Ava but any test runner works (Jest, Mocha, Jasmine, etc.)
import test from 'ava'

import multiply from './multiply.js'

import { each } from 'test-each'

// The code we are testing

// Repeat test using different inputs and expected outputs
each(
  [
    { first: 2, second: 2, output: 4 },
    { first: 3, second: 3, output: 9 },
  ],
  ({ title }, { first, second, output }) => {
    // Test titles will be:
    //    should multiply | {"first": 2, "second": 2, "output": 4}
    //    should multiply | {"first": 3, "second": 3, "output": 9}
    test(`should multiply | ${title}`, (t) => {
      t.is(multiply(first, second), output)
    })
  },
)

// Snapshot testing. The `output` is automatically set on the first run,
// then re-used in the next runs.
each(
  [
    { first: 2, second: 2 },
    { first: 3, second: 3 },
  ],
  ({ title }, { first, second }) => {
    test(`should multiply outputs | ${title}`, (t) => {
      t.snapshot(multiply(first, second))
    })
  },
)

// Cartesian product.
// Run this test 4 times using every possible combination of inputs
each([0.5, 10], [2.5, 5], ({ title }, first, second) => {
  test(`should mix integers and floats | ${title}`, (t) => {
    t.is(typeof multiply(first, second), 'number')
  })
})

// Fuzz testing. Run this test 1000 times using different numbers.
each(1000, Math.random, ({ title }, index, randomNumber) => {
  test(`should correctly multiply floats | ${title}`, (t) => {
    t.is(multiply(randomNumber, 1), randomNumber)
  })
})

Demo

You can try this library:

Install

npm install -D test-each

This package works in both Node.js >=16.17.0 and browsers.

This is an ES module. It must be loaded using an import or import() statement, not require(). If TypeScript is used, it must be configured to output ES modules, not CommonJS.

Usage

import { each } from 'test-each'

const inputs = [
  ['red', 'blue'],
  [0, 5, 50],
]
each(...inputs, (info, color, number) => {})

Fires callback once for each possible combination of inputs.

Each input can be an array, a function or an integer.

A common use case for callback is to define tests (using any test runner).

info is an object whose properties can be used to generate test titles.

Test titles

Each combination of parameters is stringified as a title available in the callback's first argument.

Titles should be included in test titles to make them descriptive and unique.

Long titles are truncated. An incrementing counter is appended to duplicates.

Any JavaScript type is stringified, not just JSON.

You can customize titles either by:

import { each } from 'test-each'

each([{ color: 'red' }, { color: 'blue' }], ({ title }, param) => {
  // Test titles will be:
  //    should test color | {"color": "red"}
  //    should test color | {"color": "blue"}
  test(`should test color | ${title}`, () => {})
})

// Plain objects can override this using a `title` property
each(
  [
    { color: 'red', title: 'Red' },
    { color: 'blue', title: 'Blue' },
  ],
  ({ title }, param) => {
    // Test titles will be:
    //    should test color | Red
    //    should test color | Blue
    test(`should test color | ${title}`, () => {})
  },
)

// The `info` argument can be used for dynamic titles
each([{ color: 'red' }, { color: 'blue' }], (info, param) => {
  // Test titles will be:
  //    should test color | 0 red
  //    should test color | 1 blue
  test(`should test color | ${info.index} ${param.color}`, () => {})
})

Cartesian product

If several inputs are specified, their cartesian product is used.

import { each } from 'test-each'

// Run callback five times: a -> b -> c -> d -> e
each(['a', 'b', 'c', 'd', 'e'], (info, param) => {})

// Run callback six times: a c -> a d -> a e -> b c -> b d -> b e
each(['a', 'b'], ['c', 'd', 'e'], (info, param, otherParam) => {})

// Nested arrays are not iterated.
// Run callback only twice: ['a', 'b'] -> ['c', 'd', 'e']
each(
  [
    ['a', 'b'],
    ['c', 'd', 'e'],
  ],
  (info, param) => {},
)

Input functions

If a function is used instead of an array, each iteration fires it and uses its return value instead. The function is called with the same arguments as the callback.

The generated values are included in test titles.

import { each } from 'test-each'

// Run callback with a different random number each time
each(['red', 'green', 'blue'], Math.random, (info, color, randomNumber) => {})

// Input functions are called with the same arguments as the callback
each(
  ['02', '15', '30'],
  ['January', 'February', 'March'],
  ['1980', '1981'],
  (info, day, month, year) => `${day}/${month}/${year}`,
  (info, day, month, year, date) => {},
)

Fuzz testing

Integers can be used instead of arrays to multiply the number of iterations.

This enables fuzz testing when combined with input functions and libraries like faker.js, chance.js or json-schema-faker.

import faker from 'faker'

// Run callback 1000 times with a random UUID and color each time
each(
  1000,
  faker.random.uuid,
  faker.random.arrayElement(['green', 'red', 'blue']),
  (info, randomUuid, randomColor) => {},
)

// `info.index` can be used as a seed for reproducible randomness.
// The following series of 1000 UUIDs will remain the same across executions.
each(
  1000,
  ({ index }) => faker.seed(index) && faker.random.uuid(),
  (info, randomUuid) => {},
)

Snapshot testing

This library works well with snapshot testing.

Any library can be used (snap-shot-it, Ava snapshots, Jest snapshots, Node TAP snapshots, etc.).

import { each } from 'test-each'

// The `output` is automatically set on the first run,
// then re-used in the next runs.
each(
  [
    { first: 2, second: 2 },
    { first: 3, second: 3 },
  ],
  ({ title }, { first, second }) => {
    test(`should multiply outputs | ${title}`, (t) => {
      t.snapshot(multiply(first, second))
    })
  },
)

Side effects

If callback's parameters are directly modified, they should be copied to prevent side effects for the next iterations.

import { each } from 'test-each'

each(
  ['green', 'red', 'blue'],
  [{ active: true }, { active: false }],
  (info, color, param) => {
    // This should not be done, as the objects are re-used in several iterations
    param.active = false

    // But this is safe since it's a copy
    const newParam = { ...param }
    newParam.active = false
  },
)

Iterables

iterable() can be used to iterate over each combination instead of providing a callback.

import { iterable } from 'test-each'

const combinations = iterable(
  ['green', 'red', 'blue'],
  [{ active: true }, { active: false }],
)

for (const [{ title }, color, param] of combinations) {
  test(`should test color | ${title}`, () => {})
}

The return value is an Iterable. This can be converted to an array with the spread operator.

const array = [...combinations]

array.forEach(([{ title }, color, param]) => {
  test(`should test color | ${title}`, () => {})
})

API

each(...inputs, callback)

inputs: Array | function | integer (one or several)
callback: (info, ...params) => void

Fires callback with each combination of params.

iterable(...inputs)

inputs: Array | function | integer (one or several)
Return value: Iterable<[info, ...params]>

Returns an Iterable looping through each combination of params.

info

Type: object

info.title

Type: string

Like params but stringified. Should be used in test titles.

info.titles

Type: string[]

Like info.title but for each param.

info.index

Type: integer

Incremented on each iteration. Starts at 0.

info.indexes

Type: integer[]

Index of each params inside each initial input.

params

Type: any (one or several)

Combination of inputs for the current iteration.

Support

For any question, don't hesitate to submit an issue on GitHub.

Everyone is welcome regardless of personal background. We enforce a Code of conduct in order to promote a positive and inclusive environment.

Contributing

This project was made with ❀️. The simplest way to give back is by starring and sharing it online.

If the documentation is unclear or has a typo, please click on the page's Edit button (pencil icon) and suggest a correction.

If you would like to help us fix a bug or add a new feature, please check our guidelines. Pull requests are welcome!

More Repositories

1

cross-platform-node-guide

πŸ“— How to write cross-platform Node.js code
JavaScript
1,331
star
2

modern-errors

Handle errors in a simple, stable, consistent way
JavaScript
1,273
star
3

nve

Run any command on specific Node.js versions
JavaScript
614
star
4

wild-wild-path

🀠 Object property paths with wildcards and regexps 🌡
JavaScript
608
star
5

log-process-errors

Show some ❀️ to Node.js process errors
JavaScript
471
star
6

human-signals

Human-friendly process signals
JavaScript
261
star
7

autoserver

Create a full-featured REST/GraphQL API from a configuration file
JavaScript
200
star
8

safe-json-value

⛑️ JSON serialization should never fail
JavaScript
198
star
9

cross-platform-terminal-characters

All the characters that work on most terminals
JavaScript
196
star
10

unix-permissions

Swiss Army knife for Unix permissions
JavaScript
119
star
11

Notes

Technologies I've learned
sed
58
star
12

gulp-execa

Gulp.js command execution for humans
JavaScript
55
star
13

fast-cartesian

Fast cartesian product
TypeScript
53
star
14

get-bin-path

Get the current package's binary path
JavaScript
34
star
15

wild-wild-utils

🀠 Functional utilities using object property paths with wildcards and regexps 🌡
JavaScript
28
star
16

get-node

Download a specific version of Node.js
JavaScript
19
star
17

handle-cli-error

πŸ’£ Error handler for CLI applications πŸ’₯
JavaScript
15
star
18

keep-func-props

Wrap a function without changing its name and other properties
JavaScript
13
star
19

node-version-alias

Resolve Node.js version aliases like `latest`, `lts` or `erbium`
JavaScript
12
star
20

big-cartesian

Cartesian product for big inputs
TypeScript
12
star
21

eslint-config

ESLint configuration for my own projects
JavaScript
10
star
22

preferred-node-version

Get the preferred Node.js version of a project or user
JavaScript
9
star
23

all-node-versions

List all available Node.js versions
JavaScript
9
star
24

error-serializer

Convert errors to/from plain objects.
JavaScript
9
star
25

modern-errors-http

`modern-errors` plugin to create HTTP error responses.
TypeScript
8
star
26

normalize-node-version

Normalize and validate Node.js versions
JavaScript
8
star
27

cv-website

Static page with my CV website
HTML
8
star
28

truncate-json

Truncate a JSON string.
JavaScript
8
star
29

merge-error-cause

Merge an error with its `cause`
JavaScript
7
star
30

get-node-cli

Download a specific version of Node.js (CLI)
JavaScript
7
star
31

global-cache-dir

Get the global cache directory
JavaScript
6
star
32

dev-tasks

Automated development tasks for my own projects
Shell
5
star
33

modern-errors-serialize

`modern-errors` plugin to serialize/parse errors.
JavaScript
5
star
34

portuguese-conjugation-cheat-sheet

Portuguese conjugation cheat sheet
CSS
5
star
35

spyd

Complete yet simple benchmark runner
JavaScript
5
star
36

string-byte-length

Get the UTF-8 byte length of a string.
JavaScript
5
star
37

template-javascript

JavaScript library template
JavaScript
5
star
38

colors-option

Let users toggle colors
JavaScript
5
star
39

modern-errors-winston

`modern-errors` plugin for Winston.
JavaScript
5
star
40

normalize-exception

Normalize exceptions/errors
JavaScript
5
star
41

abstract-parser

Abstraction layer for JavaScript parsers
JavaScript
4
star
42

modern-errors-cli

`modern-errors` plugin to handle errors in CLI modules.
TypeScript
4
star
43

error-custom-class

Create custom error classes
JavaScript
4
star
44

fetch-node-website

Fetch releases on nodejs.org
JavaScript
4
star
45

winston-error-format

Log errors with Winston
JavaScript
4
star
46

is-json-value

Check if a value is valid JSON.
JavaScript
4
star
47

error-cause-polyfill

Polyfill `error.cause`
JavaScript
4
star
48

modern-errors-switch

`modern-errors` plugin to execute class-specific logic.
JavaScript
4
star
49

modern-errors-clean

`modern-errors` plugin to clean stack traces.
JavaScript
4
star
50

wild-wild-parser

🀠 Parser for object property paths with wildcards and regexps 🌡
JavaScript
4
star
51

precise-now

Like `performance.now()` but in nanoseconds
TypeScript
4
star
52

error-http-response

Create HTTP error responses.
JavaScript
3
star
53

test-api

[WIP] Automated API testing
JavaScript
3
star
54

modern-errors-process

`modern-errors` plugin to handle process errors.
JavaScript
3
star
55

guess-json-indent

Guess the indentation of a JSON string.
JavaScript
3
star
56

time-resolution

Find the process's time resolution
TypeScript
3
star
57

is-error-instance

Check if a value is an `Error` instance.
TypeScript
3
star
58

declarative-merge

Merge objects/arrays declaratively
JavaScript
3
star
59

template-typescript

TypeScript library template
TypeScript
3
star
60

modern-errors-bugs

`modern-errors` plugin to print where to report bugs.
JavaScript
3
star
61

set-error-class

Properly update an error's class.
JavaScript
2
star
62

dev-parser

Parse JavaScript using a terminal
JavaScript
2
star
63

ehmicky

Node.js back-end developer
2
star
64

set-error-stack

Properly update an error's stack.
JavaScript
2
star
65

set-array

Set array items declaratively
JavaScript
2
star
66

set-error-message

Properly update an error's message.
JavaScript
2
star
67

set-error-props

Properly update an error's properties
JavaScript
2
star
68

string-byte-slice

Like `string.slice()` but bytewise.
JavaScript
2
star
69

redefine-property

Better `Object.defineProperty()`
JavaScript
2
star
70

terminal-theme

🎨 Use a color theme for your code's terminal output
JavaScript
2
star
71

wrap-error-message

Properly wrap an error's message.
JavaScript
2
star
72

chalk-string

Chalk with style strings.
JavaScript
2
star
73

error-class-utils

Properly create error classes.
JavaScript
2
star
74

create-error-types

Create multiple error types.
JavaScript
1
star
75

oh-oh

This is an.
JavaScript
1
star
76

design

Logos of my projects
1
star
77

prettier-config

Prettier configuration for my own projects
JavaScript
1
star