π€ Repeat tests. Repeat tests. Repeat tests.
Repeats tests using different inputs (Data-Driven Testing):
- test runner independent: works with your current setup
- generates test titles that are descriptive, unique, for any JavaScript type (not just JSON)
- loops over every possible combination of inputs (cartesian product)
- can use random functions (fuzz testing)
- snapshot testing friendly
Example
// The examples use Ava but any test runner works (Jest, Mocha, Jasmine, etc.)
import test from 'ava'
import multiply from './multiply.js'
import { each } from 'test-each'
// The code we are testing
// Repeat test using different inputs and expected outputs
each(
[
{ first: 2, second: 2, output: 4 },
{ first: 3, second: 3, output: 9 },
],
({ title }, { first, second, output }) => {
// Test titles will be:
// should multiply | {"first": 2, "second": 2, "output": 4}
// should multiply | {"first": 3, "second": 3, "output": 9}
test(`should multiply | ${title}`, (t) => {
t.is(multiply(first, second), output)
})
},
)
// Snapshot testing. The `output` is automatically set on the first run,
// then re-used in the next runs.
each(
[
{ first: 2, second: 2 },
{ first: 3, second: 3 },
],
({ title }, { first, second }) => {
test(`should multiply outputs | ${title}`, (t) => {
t.snapshot(multiply(first, second))
})
},
)
// Cartesian product.
// Run this test 4 times using every possible combination of inputs
each([0.5, 10], [2.5, 5], ({ title }, first, second) => {
test(`should mix integers and floats | ${title}`, (t) => {
t.is(typeof multiply(first, second), 'number')
})
})
// Fuzz testing. Run this test 1000 times using different numbers.
each(1000, Math.random, ({ title }, index, randomNumber) => {
test(`should correctly multiply floats | ${title}`, (t) => {
t.is(multiply(randomNumber, 1), randomNumber)
})
})
Demo
You can try this library:
- either directly in your browser.
- or by executing the
examples
files in a terminal.
Install
npm install -D test-each
This package works in both Node.js >=16.17.0 and browsers.
This is an ES module. It must be loaded using
an import
or import()
statement,
not require()
. If TypeScript is used, it must be configured to
output ES modules,
not CommonJS.
Usage
import { each } from 'test-each'
const inputs = [
['red', 'blue'],
[0, 5, 50],
]
each(...inputs, (info, color, number) => {})
Fires callback
once for each possible combination of inputs
.
Each input
can be an array
, a
function
or an integer
.
A common use case for callback
is to define tests (using any test runner).
info
is an object
whose properties can be used to generate
test titles.
Test titles
Each combination of parameters is stringified as a title
available in the
callback
's first argument.
Titles should be included in test titles to make them descriptive and unique.
Long titles are truncated. An incrementing counter is appended to duplicates.
Any JavaScript type is stringified, not just JSON.
You can customize titles either by:
- defining
title
properties ininputs
that are plain objects - using the
info
argument
import { each } from 'test-each'
each([{ color: 'red' }, { color: 'blue' }], ({ title }, param) => {
// Test titles will be:
// should test color | {"color": "red"}
// should test color | {"color": "blue"}
test(`should test color | ${title}`, () => {})
})
// Plain objects can override this using a `title` property
each(
[
{ color: 'red', title: 'Red' },
{ color: 'blue', title: 'Blue' },
],
({ title }, param) => {
// Test titles will be:
// should test color | Red
// should test color | Blue
test(`should test color | ${title}`, () => {})
},
)
// The `info` argument can be used for dynamic titles
each([{ color: 'red' }, { color: 'blue' }], (info, param) => {
// Test titles will be:
// should test color | 0 red
// should test color | 1 blue
test(`should test color | ${info.index} ${param.color}`, () => {})
})
Cartesian product
If several inputs
are specified, their
cartesian product is used.
import { each } from 'test-each'
// Run callback five times: a -> b -> c -> d -> e
each(['a', 'b', 'c', 'd', 'e'], (info, param) => {})
// Run callback six times: a c -> a d -> a e -> b c -> b d -> b e
each(['a', 'b'], ['c', 'd', 'e'], (info, param, otherParam) => {})
// Nested arrays are not iterated.
// Run callback only twice: ['a', 'b'] -> ['c', 'd', 'e']
each(
[
['a', 'b'],
['c', 'd', 'e'],
],
(info, param) => {},
)
Input functions
If a function
is used instead of an array, each iteration fires it and uses
its return value instead. The function
is called with the
same arguments
as the callback
.
The generated values are included in test titles.
import { each } from 'test-each'
// Run callback with a different random number each time
each(['red', 'green', 'blue'], Math.random, (info, color, randomNumber) => {})
// Input functions are called with the same arguments as the callback
each(
['02', '15', '30'],
['January', 'February', 'March'],
['1980', '1981'],
(info, day, month, year) => `${day}/${month}/${year}`,
(info, day, month, year, date) => {},
)
Fuzz testing
Integers can be used instead of arrays to multiply the number of iterations.
This enables fuzz testing when combined with input functions and libraries like faker.js, chance.js or json-schema-faker.
import faker from 'faker'
// Run callback 1000 times with a random UUID and color each time
each(
1000,
faker.random.uuid,
faker.random.arrayElement(['green', 'red', 'blue']),
(info, randomUuid, randomColor) => {},
)
// `info.index` can be used as a seed for reproducible randomness.
// The following series of 1000 UUIDs will remain the same across executions.
each(
1000,
({ index }) => faker.seed(index) && faker.random.uuid(),
(info, randomUuid) => {},
)
Snapshot testing
This library works well with snapshot testing.
Any library can be used
(snap-shot-it
,
Ava snapshots,
Jest snapshots,
Node TAP snapshots, etc.).
import { each } from 'test-each'
// The `output` is automatically set on the first run,
// then re-used in the next runs.
each(
[
{ first: 2, second: 2 },
{ first: 3, second: 3 },
],
({ title }, { first, second }) => {
test(`should multiply outputs | ${title}`, (t) => {
t.snapshot(multiply(first, second))
})
},
)
Side effects
If callback
's parameters are directly modified, they should be
copied to prevent side effects for the next iterations.
import { each } from 'test-each'
each(
['green', 'red', 'blue'],
[{ active: true }, { active: false }],
(info, color, param) => {
// This should not be done, as the objects are re-used in several iterations
param.active = false
// But this is safe since it's a copy
const newParam = { ...param }
newParam.active = false
},
)
Iterables
iterable()
can be used to iterate over each combination
instead of providing a callback.
import { iterable } from 'test-each'
const combinations = iterable(
['green', 'red', 'blue'],
[{ active: true }, { active: false }],
)
for (const [{ title }, color, param] of combinations) {
test(`should test color | ${title}`, () => {})
}
The return value is an
Iterable
.
This can be converted to an array with the spread operator.
const array = [...combinations]
array.forEach(([{ title }, color, param]) => {
test(`should test color | ${title}`, () => {})
})
API
each(...inputs, callback)
inputs
: Array | function | integer
(one or several)
callback
: (info, ...params) => void
Fires callback
with each combination of params
.
iterable(...inputs)
inputs
: Array | function | integer
(one or several)
Return value: Iterable<[info, ...params]>
Returns an
Iterable
looping through each combination of params
.
info
Type: object
info.title
Type: string
Like params
but stringified. Should be used in
test titles.
info.titles
Type: string[]
Like info.title
but for each param
.
info.index
Type: integer
Incremented on each iteration. Starts at 0
.
info.indexes
Type: integer[]
Index of each params
inside each initial
input
.
params
Type: any
(one or several)
Combination of inputs for the current iteration.
Support
For any question, don't hesitate to submit an issue on GitHub.
Everyone is welcome regardless of personal background. We enforce a Code of conduct in order to promote a positive and inclusive environment.
Contributing
This project was made with β€οΈ. The simplest way to give back is by starring and sharing it online.
If the documentation is unclear or has a typo, please click on the page's Edit
button (pencil icon) and suggest a correction.
If you would like to help us fix a bug or add a new feature, please check our guidelines. Pull requests are welcome!