Master the art of the most powerful testing technique for backend
3 things to your benefit
Component/integration test is an hybrid between E2E and unit tests. It's gaining a lot of popularity and going by the testing diamond model it is considered as the default technique for modern backend. Its main idea is testing an entire component (e.g., Microservice) as-is, through the API, with all the layers including database but fake anything extraneous. This brings both high confidence and great developer experience. However, doing it right, fast, exhaustive and maximizing the value demand some learning and skills. This is the mission statement of this repo. Warning: You might fall in love with testing ๐
This repository contains:
1.
2. ๐ Example application - A Complete showcase of a typical Node.js backend with performant tests setup (50 tests in 4 seconds! including database!)
3.
๐จโ๐ซ Exciting news: I've just released my super-comprehensive testing course after two years of recording and editing. It's now on a ๐ limited-time sale during July
Table of contents
Best Practices Sections
Database And Infrastructure Setup- Optimizing your DB, MQ and other infra for testing (6 best practices)Web Server Setup- Good practices for starting and stopping the backend API (3 best practices)The Test Anatomy- The bread and butter of a component test (6 best practices)Integration- Techniques for testing collaborations with 3rd party components (8 best practices)Dealing With Data- Patterns and practices for testing the application data and database (8 best practices)Message Queue- Correctly testing flows that start or end at a queue (8 best practices)Development Workflow- Incorporating component tests into your daily workflow (5 best practices)
Example Application
Our Showcase- An example Node.js component that embodies selected list of important best practices
Other Recipes
More Examples And Platforms- A list of more examples that cover more platforms and topics
โ
Best Practices
Section 1: Infrastructure and database setup
โช๏ธ 1. Use Docker-Compose to host the database and other infrastructure
๐ทย Tags: #strategic
๐ ย Alternatives: A popular option is manual installation of local database - This results in developers working hard to get in-sync with each other ("Did you set the right permissions in the DB?") and configuring a different setup in CI
โ Code Examples
# docker-compose.yml
version: '3.6'
services:
database:
image: postgres:11
command: postgres -c fsync=off -c synchronous_commit=off -c full_page_writes=off -c random_page_cost=1.0
environment:
- POSTGRES_USER=myuser
- POSTGRES_PASSWORD=myuserpassword
- POSTGRES_DB=shop
container_name: 'postgres-for-testing'
ports:
- '54310:5432'
tmpfs: /var/lib/postgresql/dataโช๏ธ 2. Start docker-compose using code in the global setup process
๐ทย Tags: #strategic
โ
Do: In a typical multi-process test runner (e.g. Mocha, Jest), the infrastructure should be started in a global setup/hook (Jest global setup), Mocha global fixture using custom code that spin up the docker-compose file. This takes away common workflows pains - The DB is an explicit dependency of the test, no more tests failing because the DB is down. A new developer onboarded? Get them up to speed with nothing more than git clone && npm test. Everything happens automatically, no tedious README.md, no developers wonder what setup steps did they miss. In addition, going with this approach maximizes the test performance: the DB is not instantiated per process or per file, rather once and only once. On the global teardown phase, all the containers should shutoff (See a dedicated bullet below).
โ Code Examples
// jest.config.js
globalSetup: './example-application/test/global-setup.js'// global-setup.js
const dockerCompose = require('docker-compose');
module.exports = async () => {
await dockerCompose.upAll();
};โช๏ธ 3. Shutoff the infrastructure only in the CI environment
๐ทย Tags: #performance
โ Do: Keep the database and other infrastructure always alive in developers' machine so the next tests run will start at a glance, typically in 3-5ms. This super-fast start-up will encourage developers to run the tests continuously and treat them as a coding companion: It's an amazing coding experience to have the tests running all the time and watching your back as you type. Keeping the DB alive requires a clear data clean-up strategy, see our recommendation below. What about CI environment? This careful tune-up is mostly important in a developer machine where the test might get executed very frequently (e.g. after every editor save, once a minute), in a CI environnement the next tests execution might happen in a different machine and there is no motivation to keep the the docker-compose up.
โ Code Examples
// jest.config.js
globalTeardown: './example-application/test/global-teardown.js',// global-teardown.js - clean-up after all tests
const isCI = require('is-ci');
const dockerCompose = require('docker-compose');
module.exports = async () => {
// Check if running CI environment
if (isCI) {
dockerCompose.down();
}
};โช๏ธ 4. Optimize your real DB for testing, Don't fake it
#intermediate
โ Code Examples
Postgres
# docker-compose file
version: "3.6"
services:
db:
image: postgres:13
container_name: 'postgres-for-testing'
// fsync=off means don't wait for disc acknowledgement
command: postgres -c fsync=off -c synchronous_commit=off -c full_page_writes=off -c random_page_cost=1.0
tmpfs: /var/lib/postgresql/data
# ...โช๏ธ 5. Store test data in RAM folder
#performance
โ
Do: Use your real DB product, just store the data in a RAM folder to reduce IO and gain some performance boost. In Linux machine, this can be done quickly by mapping the data to the built-in tmpfs directory - This particular folder's content is stored in memory without disc involvement. In Mac and Windows, one should generate a RAM folder using a script that can be done once or automated. We have conducted multiple performance benchmarks and found that this only slightly improves the performance - The other optimizations that were covered above already minimize the IO work and modern SSD discs are blazing fast. Some specific databases like Mongo comes with a built-in memory engine, this is an additional option to consider.
โ Code Examples
# docker-compose file
version: "3.6"
services:
db:
image: postgres:13
container_name: 'postgres-for-testing'
// ๐ Stores the DB data in RAM folder. Works only in Linux
tmpfs: /var/lib/postgresql/data
# ...โช๏ธ 6. Build the DB schema using migrations, ensure it happens only once in dev
#intermediate
โ Code Examples
// jest.config.js
globalSetup: './example-application/test/global-setup.js'
// global-setup.js
const npm = require('npm');
const util = require('util');
module.exports = async () => {
// ...
const npmCommandAsPromise = util.promisify(npm.commands.run);
await npmCommandAsPromise(['db:migrate']); // Migrating the DB using a npm script before running any tests.
// ...
}Section 2: Web server setup
โช๏ธ 1. The test and the backend should live within the same process
๐ทย Tags: #basic, #strategic
โ ย Do: The tests should start the webserver within the same process, not in a remote environment or container. Failing to do so will result in a loss of critical features: A test won't be able to simulate various important events using test doubles (e.g. make some component throw an exception), customize environment variables, and make configuration changes. Also, the complexity of measuring code coverage and intercepting network calls will highly increase
โ Code Examples
const apiUnderTest = require('../api/start.js');
beforeAll(async () => {
//Start the backend in the same processโช๏ธ 2. Let the tests control when the server should start and shutoff
#basic, #strategic
โ Code Examples
const initializeWebServer = async () => {
return new Promise((resolve, reject) => {
// A typical Express setup
expressApp = express();
defineRoutes(expressApp);
connection = expressApp.listen(() => {
resolve(expressApp);
});
});
};
const stopWebServer = async () => {
return new Promise((resolve, reject) => {
connection.close(() => {
resolve();
})
});
};
beforeAll(async () => {
expressApp = await initializeWebServer();
});
afterAll(async () => {
await stopWebServer();
});
โช๏ธ 3. Specify a port in production, randomize in testing
#e
โ Code Examples
// api-under-test.js
const initializeWebServer = async () => {
return new Promise((resolve, reject) => {
// A typical Express setup
expressApp = express();
connection = expressApp.listen(webServerPort, () => {// No port
resolve(expressApp);
});
});
};
// test.js
beforeAll(async () => {
expressApp = await initializeWebServer();// No port
});Section 3: The test anatomy (basics)
โช๏ธ 1. Stick to unit testing best practices, aim for great developer-experience
๐ทย Tags: #basic, #strategic
โ Code Examples
// basic-tests.test.ts
test('When asked for an existing order, Then should retrieve it and receive 200 response', async () => {
// Arrange
const orderToAdd = {
userId: 1,
productId: 2,
mode: 'approved',
};
const {
data: { id: addedOrderId },
} = await axiosAPIClient.post(`/order`, orderToAdd);
// Act
const getResponse = await axiosAPIClient.get(`/order/${addedOrderId}`);
// Assert
expect(getResponse).toMatchObject({
status: 200,
data: {
userId: 1,
productId: 2,
mode: 'approved',
},
});
});โช๏ธ 2. Approach the API using a library that is a pure HTTP client (e.g. axios, not supertest)
#basic
๐ ย Alternatives: Supertest - Consider to avoid as it encourages direct bounding to express objects and promote different assertion syntax that your built-in assertion library
โ Code Examples
// basic-test.test.ts
const axios = require('axios');
let axiosAPIClient;
beforeAll(async () => {
const apiConnection = await initializeWebServer();
const axiosConfig = {
baseURL: `http://127.0.0.1:${apiConnection.port}`,
validateStatus: () => true,
};
// Create axios client for the whole test suite
axiosAPIClient = axios.create(axiosConfig);
// ...
});
test('When asked for an existing order, Then should retrieve it and receive 200 response', async () => {
const orderToAdd = {
userId: 1,
productId: 2,
mode: 'approved',
};
// Use axios to create an order
const {
data: { id: addedOrderId },
} = await axiosAPIClient.post(`/order`, orderToAdd);
// Use axios to retrieve the same order by id
const getResponse = await axiosAPIClient.get(`/order/${addedOrderId}`);
// ...
});โช๏ธ 3. Provide real credentials or token. If possible, avoid security back doors
#basics
๐ ย Alternatives: Mock the authentication middleware and disable it or trick it into authorizing the request - While not an awful option, it means that the 'real' authorization code is not part of the test (because the test stubbed /replaced it)
โช๏ธ 4. Assert on the entire HTTP response object, not on every field
๐ทย Tags: #basics
โ Code Examples
// basic-tests.test.ts
test('When asked for an existing order, Then should retrieve it and receive 200 response', async () => {
// ...
const getResponse = await axiosAPIClient.get(`/order/${addedOrderId}`);
// Assert on entire HTTP response object
expect(getResponse).toMatchObject({
status: 200,
data: {
userId: 1,
productId: 2,
mode: 'approved',
},
});
});โช๏ธ 5. Structure tests by routes and stories
#basics
โ ย Do: Organize your tests using 'describe' blocks representing API routes. Eventually, this will result in a tree of routes and tests underneath. For example describe('/API'), describe('POST /orders'). See the full example below. This common view of API end-points will likely look familiar and appeal to the occasional test report viewer. It resembles tooling that were proven to be popular like POSTMAN, OpenAPI docs, and others. Most, if not all, developers would know to map a test failure in a specific route with the corresponding code. A newly onboarded developer who is unfamiliar with the code would benefit from understanding the various routes and then easily start exploring the corresponding controller. Sometimes there are many scenario/cases under each route. In this case, consider creating another nested category (i.e. describe block) that represents a topic or user story. If the code under test is accessed using a message queue (see dedicated 4below), structure the routes by topics and queues.
โ Code Examples
// basic-tests.test.js
describe('/api', () => {
describe('GET /order', () => {
// ...
});
describe('POST /orders', () => {
// ...
});
});โก๏ธ Full code here
๐ฆ Learn all of these topics in an online course by Yoni Goldberg
โช๏ธ 6. Test the five potential outcomes
๐ทย Tags: #intermediate #strategic
โข Response - The test invokes an action (e.g., via API) and gets a response. It's now concerned with checking the response data correctness, schema, and HTTP status
โข A new state - After invoking an action, some data is probably modified. For example, when updating a user - It might be that the new data was not saved. Commonly and mistakenly, testers check only the response and not whether the data is updated correctly. Testing data and databases raises multiple interesting challenges that are greatly covered below in the
โข External calls - After invoking an action, the app might call an external component via HTTP or any other transport. For example, a call to send SMS, email or charge a credit card. Anything that goes outside and might affect the user - Should be tested. Testing integrations is a broad topic which is discussed in the
โข Message queues - The outcome of a flow might be a message in a queue. In our example application, once a new order was saved the app puts a message in some MQ product. Now other components can consume this message and continue the flow. This is very similar to testing integrations only working with message queues is different technically and tricky. The
โข Observability - Some things must be monitored, like errors or remarkable business events. When a transaction fails, not only we expect the right response but also correct error handling and proper logging/metrics. This information goes directly to a very important user - The ops user (i.e., production SRE/admin). Testing error handler is not very straighforward - Many types of errors might get thrown, some errors should lead to process crash, and there are many other corners to cover. We plan to write the ๐ section on 'Observability and errors' soon
This content is available also as a course or a workshop
Find here the same content as a course, online workshop, free webinar (TBD, follow here for specific date), or invite a private workshop to your team
Section 4: Integrations
โช๏ธ 1. Isolate the component from the world using HTTP interceptor
๐ทย Tags: #strategic #basic
๐ ย Alternatives: Some services provide a fake version that can be deployed by the caller locally, usually using Docker - This will ease the setup and boost the performance but won't help with simulating various responses ย
โ Code Examples
// Intercept requests for 3rd party APIs and return a predefined response
beforeEach(() => {
nock('http://localhost/user/').get(`/1`).reply(200, {
id: 1,
name: 'John',
});
});โช๏ธ 2. Define default responses before every test to ensure a clean slate
#basic
โ Code Examples
// Create a one-time interceptor before every test
beforeEach(() => {
nock('http://localhost/user/').get(`/1`).reply(200, {
id: 1,
name: 'John',
});
});
// Endure clean slate after each test
afterEach(() => {
nock.cleanAll();
});โก๏ธ Full code here
โช๏ธ 3. Override the happy defaults with corner cases using unique paths
#advanced, #draft
Remember that after every test everything is cleaned-up, see bullet about clean-up.
โ Code Examples
// Using an uncommon user id (7) and create a compatible interceptor
test('When the user does not exist, return http 404', async () => {
//Arrange
const orderToAdd = {
userId: 7,
productId: 2,
mode: 'draft',
};
nock('http://localhost/user/').get(`/7`).reply(404, {
message: 'User does not exist',
code: 'nonExisting',
});
//Act
const orderAddResult = await axiosAPIClient.post('/order', orderToAdd);
//Assert
expect(orderAddResult.status).toBe(404);
});โช๏ธ 4. Deny all outgoing requests by default
#basic
nock.disableNetConnect(). For any request that was not explicitly defined - the interceptor will throw an exception and make the tests fail. Why is this needed? To protect the component borders. It might be that some HTTP calls were not considered and trying to hit a real external server. When requests are not intercepted, it violates the component isolation, triggers flakiness, and degrades performance. Remember to exclude calls to the local API under test that should serve the tests` requests. When the test suite is done, remove this restriction to avoid leaving unexpected behaviour to other tests suites.
โ Code Examples
beforeAll(async () => {
// ...
// ๏ธ๏ธ๏ธEnsure that this component is isolated by preventing unknown calls
nock.disableNetConnect();
// Enable only requests for the API under test
nock.enableNetConnect('127.0.0.1');
});โช๏ธ 5. Simulate network chaos
#basic
โ
Do: Go beyond the happy and sad paths. Check not only errored responses (e.g., HTTP 500 error) but also network-level anomalies like slow and timed-out responses. This will prove that the code is resilient and can handle various network scenarios like taking the right path after a timeout, has no fragile race conditions, and contains a circuit breaker for retries. Reputable interceptor tools can easily simulate various network behaviors like hectic service that occasionally fail. It can even realize when the default HTTP client timeout value is longer than the simulated response time and throw a timeout exception right away without waiting
โ Code Examples
test('When users service replies with 503 once and retry mechanism is applied, then an order is added successfully', async () => {
//Arrange
nock.removeInterceptor(userServiceNock.interceptors[0])
nock('http://localhost/user/')
.get('/1')
.reply(503, undefined, { 'Retry-After': 100 });
nock('http://localhost/user/')
.get('/1')
.reply(200);
const orderToAdd = {
userId: 1,
productId: 2,
mode: 'approved',
};
//Act
const response = await axiosAPIClient.post('/order', orderToAdd);
//Assert
expect(response.status).toBe(200);
});โช๏ธ 6. Catch invalid outgoing requests by specifying the request schema
๐ทย Tags: #basic
โ Code Examples
// ๏ธ๏ธ๏ธAssert that the app called the mailer service appropriately with the right input
test('When order failed, send mail to admin', async () => {
//Arrange
// ...
let emailPayload;
nock('http://mailer.com')
.post('/send', (payload) => ((emailPayload = payload), true))
.reply(202);
const orderToAdd = {
userId: 1,
productId: 2,
mode: 'approved',
};
//Act
await axiosAPIClient.post('/order', orderToAdd);
// ๏ธ๏ธ๏ธAssert
expect(emailPayload).toMatchObject({
subject: expect.any(String),
body: expect.any(String),
recipientAddress: expect.stringMatching(
/^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/
),
});
});โก๏ธ Full code here
โช๏ธ 7. Record real outgoing requests for awareness
๐ทย Tags: #advanced
โช๏ธ 8. Fake the time to minimize network call duration
#basic, #draft
โ Code Examples
// use "fake timers" to simulate long requests.
test("When users service doesn't reply within 2 seconds, return 503", async () => {
//Arrange
const clock = sinon.useFakeTimers();
nock('http://localhost/user/')
.get('/1', () => clock.tick(5000))
.reply(200);
const orderToAdd = {
userId: 1,
productId: 2,
mode: 'approved',
};
//Act
const response = await axiosAPIClient.post('/order', orderToAdd);
//Assert
expect(response.status).toBe(503);
//Clean
clock.uninstall();
});โก๏ธ Full code here
Section 5: Dealing with data
โช๏ธ 1. Important: Each test should act on its own records only
๐ทย Tags: #strategic
โ Do: Any record that might affect the test results should be added at the beginning of the test. Exclamation mark. Doing so will result in short and self-contained test stories that the occasional reader can easily troubleshoot without skimming through the entire file. A common mistake is to seed the whole test data globally - This leads to high coupling and complexity. Specifically, failing to keep the tests self-contained will lead to the Domino effect: Understanding why test num #27 failed demands reading the 26 tests before. Each might have mutated the global data. Other undesired side effects: One can't run a single test because it depends on data that is generated by previous tests; It will get much harder to understand the test intent because the gun that is being shown on the last scene was never introduced before (The mystery guest syndrome). Are you concerned with performance? Based on our benchmarks, adding relevant data at the beginning of each test add ~1 second to the execution time - Absolutely worth the decreased complexity. This advice is valuable only to records that are the subject of the tests. Tests can have different types of data, see next bullet and this diagram.
โ Code Examples
test('When asked for an existing order, Then should retrieve it and receive 200 response', async () => {
//Arrange - Create a record so we can later query for it and assert for is existence
const orderToAdd = {
userId: 1,
productId: 2,
mode: 'approved',
};
await axiosAPIClient.post(`/order`, orderToAdd);
//Next -> Invoke the route under test and asssert for something
});โช๏ธ 2. Only metadata and context data should get pre-seeded to the database
None
-
Metadata - General purpose lists and lookups that are needed for the app to perform but are not related at all with the test's subject. For example, currencies list, countries, roles list, and similar. This data can get seeded once globally. There is no point in re-adding it per test or file
-
Context data - Required records that hold a relationship with the subject under test but are not being tested directly. For example, consider an e-commerce purchase flow tests: The User entity, Shop entity, Business entity are all a parent or sibling of the Order that is being tested. They might affect the test result (e.g., Trying to order goods when the user was deleted) but are not the direct subject of the test. To keep the tests short and focused, this data can be added per file, if they affect the test results - Add the data per test
-
Test records - This is the data that is actually being tested and likely to be added or mutated. The reader must directly see what data exists to understand the results of the test. For this reason, explicitly define and add this information inside the test. Going with the same e-commerce site example, when testing the purchase flow, add the order records within the test
โ Code Examples
// Adding metadata globally. Done once regardless of the amount of tests
module.exports = async () => {
console.time('global-setup');
// ...
await npmCommandAsPromise(['db:seed']); // Will create a countries (metadata) list. This is not related to the tests subject
// ...
// ๐๐ผ We're ready
console.timeEnd('global-setup');
};describe('/api', () => {
let user;
beforeAll(async () => {
// Create context data once before all tests in the suite
user = createUser();
});
describe('GET /order', () => {
test('When asked for an existing order, Then should retrieve it and receive 200 response', async () => {
//Arrange
const orderToAdd = {
userId: user.id, // Must provide a real user id but we don't care which user creates the order
productId: 2,
mode: 'approved',
};
const {
data: { id: addedOrderId },
} = await axiosAPIClient.post(`/order`, orderToAdd);
...
});
});
});test('When asked for an existing order, Then should retrieve it and receive 200 response', async () => {
//Arrange - Create a record so we can later query for it and assert for is existence
const orderToAdd = {
userId: 1,
productId: 2,
mode: 'approved',
};
await axiosAPIClient.post(`/order`, orderToAdd);
//Next -> Invoke the route under test and asssert for something
});โช๏ธ 3. Assert the new data state using the public API
๐ทย Tags: #basics
This design decision does not come without a caveat. The test invokes much more code than needed: Tests might fail because of failures in code not being directly tested. Our philosophy is to stick to user flows under realistic conditions at the cost of a slight increase in developer's sweat.
๐ ย Alternatives: Approach the DB directly - Miss bug in the query code, higher exposure to internal refactoring
โ Code Examples
test('When adding a new valid order, Then should be able to retrieve it', async () => {
//Arrange
const orderToAdd = {
userId: 1,
productId: 2,
mode: 'approved',
};
//Act
const {
data: { id: addedOrderId },
} = await axiosAPIClient.post('/order', orderToAdd);
//Assert by fetch the new order, and not only by the POST response
const { data, status } = await axiosAPIClient.get(
`/order/${addedOrderId}`
);
expect({
data,
status,
}).toMatchObject({
status: 200,
data: {
id: addedOrderId,
userId: 1,
productId: 2,
},
});
});โช๏ธ 4. Important: Choose a clear data clean-up strategy: After-all (recommended) or after-each
#strategic
The second option is to clean up after all the test files have finished (or even daily!). This approach means that the same DB with existing records serves all the tests and processes. To avoid stepping on each other's toes, the tests must add and act on specific records that they have added. Need to check that some record was added? Assume that there are other thousands of records and query for records that were added explicitly. Need to check that a record was deleted? Can't assume an empty table, check that this specific record is not there. This technique brings few powerful gains: It works natively in multi-process mode, when a developer wishes to understand what happened - the data is there and not deleted. It also increases the chance of finding bugs because the DB is full of records and not artificially empty. It's not perfect, though, since the DB is stuffed with data - Data that goes to unique columns might be duplicated. When adding 10 records and asserting their existence, a more sophisticated query will be needed. All of these challenges have reasonable resolutions (read the next bullets, for example, unique values can get random suffix). See the full comparison table here.
Who wins? There's no clear cut here. Both have their strength but also unpleasant implications. Both can result in great testing solution. Our recommended approach is cleaning up occasionally and accepting the non-deterministic DB state. This option resembles more the production environment, leads to more realistic tests and when done right will not show any flakiness. A bit of more sweat for more realism.
โ Code Examples
// After-all clean up (recommended)
// global-teardown.js
module.exports = async () => {
// ...
if (Math.ceil(Math.random() * 10) === 10) {
await new OrderRepository().cleanup();
}
};// After-each clean up
afterAll(async () => {
await new OrderRepository().cleanup();
});
// or
afterEach(async () => {
await new OrderRepository().cleanup();
});โช๏ธ 5. Add some randomness to unique fields
#intermediate
โ Code Examples
// Adding a short unique suffix to the externalIdentifier enable the writer to ignore other tests
// and the need to clean the db after each test
test('When adding a new valid order, Then should get back 200 response', async () => {
//Arrange
const orderToAdd = {
userId: 1,
productId: 2,
mode: 'approved',
externalIdentifier: `id-${getShortUnique()}`, //unique value
};
//Act
const receivedAPIResponse = await axiosAPIClient.post(
'/order',
orderToAdd
);
// ...
});โช๏ธ 6. Test also the response schema. Mostly when there are auto-generated fields
#advanced
When it is impossible to assert for specific data, check for mandatory field existence and types. Sometimes, the response contains important fields with dynamic data that can't be predicted when writing the test, like dates and incrementing numbers. If the API contract promises that these fields won't be null and hold the right types, it's imperative to test it. Most assertion libraries support checking types. If the response is small, check the return data and type together within the same assertion (see code example). One more option is to verify the entire response against an OpenAPI doc (Swagger). Most test runners have community extensions that validate API responses against their documentation.
โ Code Examples
test('When adding a new valid order, Then should get back approval with 200 response', async () => {
// ...
//Assert
expect(receivedAPIResponse).toMatchObject({
status: 200,
data: {
id: expect.any(Number), // Any number satisfies this test
mode: 'approved',
},
});
});โช๏ธ 7. Install the DB schema using the same technique like production
๐ ย Alternatives: Manually copy a DB dump - This is a great way to find installation issues only in production and also complicate the developer testing experience
โ Code Examples
// Create the DB schema. Done once regardless of the amount of tests
module.exports = async () => {
console.time('global-setup');
// ...
await npmCommandAsPromise(['db:migrate']);
// ...
// ๐๐ผ We're ready
console.timeEnd('global-setup');โช๏ธ 8. Test for undesired side effects
๐ทย Tags: #advanced
๐ ย Alternatives: Some apply Repository/ORM level protection that ensures that one tenant is not accessing another tenant's records. This is valuable but doesn't cover all the scenarios
โ Code Examples
test("When deleting an existing order, Then should get a successful message", async () => {
// Arrange
const orderToDelete = {
userId: 1,
productId: 2,
externalIdentifier: `id-${getShortUnique()}`, //unique value
};
const {
data: { id: orderToDeleteId },
} = await axiosAPIClient.post("/order", orderToDelete);
// Create another order to make sure the delete request deletes only the correct record
const anotherOrder = {
userId: 1,
productId: 2,
externalIdentifier: `id-${getShortUnique()}`, //unique value
};
nock("http://localhost/user/").get(`/1`).reply(200, {
id: 1,
name: "John",
});
const {
data: { id: anotherOrderId },
} = await axiosAPIClient.post("/order", anotherOrder);
// Act
const deleteResponse = await axiosAPIClient.delete(`/order/${orderToDeleteId}`);
const getOrderResponse = await axiosAPIClient.get(`/order/${anotherOrderId}`);
// Assert
expect(deleteResponse.status).toBe(204);
// Assert anotherOrder still exists
expect(getOrderResponse.status).toBe(200);
});โก๏ธ Full code here
This content is available also as a course or a workshop
Find here the same content as a course, online workshop, free webinar (TBD, follow here for specific date), or invite a private workshop to your team
Section 6: Message queues
โช๏ธ 1. Important: Use a fake MQ for the majority of testing
๐ทย Tags: #intermediate, #strategic
A better alternative is to use a simplistic fake that does nothing more than accepting messages, passing them to subscribers/consumers and emitting events when ack/delete happens. This fake will allow the tests to publish messages in-memory and subscribe to events to realize when interesting things happened (e.g., a message was acknowledged). Anyway, the primary mission statetement of the tests is to check how the app behaves and not the well-trusted MQ product. With a fake, all is stored in-memory with simple flows and super-fast performance. Writing a fake like this should not last more than few hours (See code example here and below). The only downside is that it is not suitable to check multi-legs flow like dead-letter queues, retries, and the production configurations. Since these specific tests are slow by nature, they anyway should be executed rarely. Given all of this background, a recommended MQ testing strategy is to use simplistic-fake for the majority of the tests, mostly the tests that cover the app flows. Then to cover other risks, write just a few E2E tests over a production-like environment with a real message queue system.
๐ ย Alternatives: Stub the message queue listener (the code that subscribes to the queue). Within the test, Mock this listener code to emit new fake MQ messages. While doable, this is not recommended. The listener layer is responsible for catching errors and mapping the result to some MQ action like acknowledgment or rejection. Leave this layer within the test scope
โ Code Examples
// fake-mq.js, Simplistic implementation of MQ client for testing purposes
// Note: This is code is even more simplified, see full example in the example application
class FakeMessageQueueProvider extends EventEmitter {
async ack() {
this.emit('message-acknowledged', { event: 'message-acknowledged' }); //Let the test know that this happened
}
async sendToQueue(queueName, message) {
this.emit('message-sent', message);
}
async consume(queueName, messageHandler) {
// We just save the callback (handler) locally, whenever a message will put into this queue
// we will fire this handler
this.messageHandler = messageHandler;
}
async pushMessageToQueue(queue, newMessage) {
this.messageHandler(newMessage);
}
}โช๏ธ 2. Promisify the test. Avoid polling, indentation, and callbacks
#advanced, #strategic
โ Code Examples
// message-queue-client.js. The MQ client/wrapper is throwing an event when the message handler is done
async consume(queueName, onMessageCallback) {
await this.channel.consume(queueName, async (theNewMessage) => {
await onMessageCallback(theNewMessage);
await this.ack(theNewMessage); // Handling is done, acknowledge the msg
this.emit('message-acknowledged', eventDescription); // Let the tests know that all is over
});
}// The test listen to the acknowledge/confirm message and knows when the operation is done
test('Whenever a user deletion message arrive, then this user orders are also deleted', async () => {
// Arrange
// ๐๐ผ HERE WE SHOULD add new orders to the system
const getNextMQEvent = once(MQClient, "message-acknowledged"); // Once function, part of Node, promisifies an event from EventEmitter
// Act
fakeMessageQueue.pushMessageToQueue('deleted-user', { id: addedOrderId });
// Assert
const eventFromMessageQueue = await getNextMQEvent; // This promise will resolve once the message handling is done
// Now we're certain that the operations is done and can start asserting for the results ๐
});โช๏ธ 3. Test message acknowledgment and 'nack-cknowledgment'
#advanced, #strategic
โ Code Examples
//Putting a delete-order message, checking the the app processed this correctly AND acknowledged
test('Whenever a user deletion message arrive, then his orders are deleted', async () => {
// Arrange
// Add here a test record - A new order of a specific user using the API
const fakeMessageQueue = await startFakeMessageQueue();
const getNextMQEvent = getNextMQConfirmation(fakeMessageQueue);
// Act
fakeMessageQueue.pushMessageToQueue('deleted-user', { id: addedOrderId });
// Assert
const eventFromMessageQueue = await getNextMQEvent;
// Check here that the user's orders were deleted
expect(eventFromMessageQueue).toEqual([{ event: 'message-acknowledged' }]);
});โช๏ธ 4. Test processing of messages batch
intermediate
โ Code Examples
# docker-compose file
version: "3.6"
services:
db:
image: postgres:11
command: postgres
environment:
- POSTGRES_USER=myuser
- POSTGRES_PASSWORD=myuserpassword
- POSTGRES_DB=shop
ports:
- "5432:5432"โช๏ธ 5. Test for 'poisoned' messages
๐ทย Tags: #intermediate
โ Code Examples
# docker-compose file
version: "3.6"
services:
db:
image: postgres:11
command: postgres
environment:
- POSTGRES_USER=myuser
- POSTGRES_PASSWORD=myuserpassword
- POSTGRES_DB=shop
ports:
- "5432:5432"โช๏ธ 6. Test for idempotency
๐ทย Tags: #intermediate
โ Code Examples
# docker-compose file
version: "3.6"
services:
db:
image: postgres:11
command: postgres
environment:
- POSTGRES_USER=myuser
- POSTGRES_PASSWORD=myuserpassword
- POSTGRES_DB=shop
ports:
- "5432:5432"โช๏ธ 7. Avoid a zombie process by testing connection failures
#advanced, #strategic
โ Code Examples
# docker-compose file
version: "3.6"
services:
db:
image: postgres:11
command: postgres
environment:
- POSTGRES_USER=myuser
- POSTGRES_PASSWORD=myuserpassword
- POSTGRES_DB=shop
ports:
- "5432:5432"โช๏ธ 8. top of development testing, write a few E2E tests
#intermediate
โ Code Examples
# docker-compose file
version: "3.6"
services:
db:
image: postgres:11
command: postgres
environment:
- POSTGRES_USER=myuser
- POSTGRES_PASSWORD=myuserpassword
- POSTGRES_DB=shop
ports:
- "5432:5432"Section 7: Development Workflow
โช๏ธ 1. Always START with integration/component tests
#strategic
โ Do: Regardless of the exact timing, the first set of tests to be written is component tests. Once a new sprint or feature is kicked off, the first details known to the developer are about the outcome of the component. At first, a developer can tell what the API/MQ might receive and what (roughly) type of information is returned. Naturally, testing this outer layer, the public interface and outcome, should come first. By doing so, developers are pushed to work with the end in mind - Define the goals before the implementation. Testing the inner functions with unit tests before the overall outcome is specified and understood does not make any sense. Surprisingly, even classic TDD books mention this workflow, see the double verification loop model. What about E2E tests? These usually focus on a broader problem than needed at first - Consequently, it should also get deferred.
โช๏ธ 2. Run few E2E, selectively consider unit tests
โ Do: Always write few E2E tests on top of component tests. Based on the specific nature of the component, some unit tests might be needed as well. Though E2E means different things to different testers, in the context of a backend they represent tests that are done with live collaborators (i.e., external services) on a real infrastructure. Therefore, they cover risks that are not covered by components tests - configuration issues, misunderstanding with 3rd party services, infrastructural issues and more. When then unit tests are needed? in the presence of none-trivial logic and algorithms. When having a single module with remarkable complexity, it's easier to avoid the distraction coming from other parts by isolating the challenging unit. This article greatly outlines when unit tests shine.
โช๏ธ 3. Cover features, not functions
Mutation tests is also an increasing technique that can be combined in the verification suite of tools. That said, it can not serve as the primary technique since it is slow by nature and shows poor performance in tests that involve DB and IO.
โช๏ธ 4. Write the tests before or during the code, but not after the fact
#strategic
โช๏ธ 5. Run the tests frequenly, if possible run continously in watch mode
๐ทย Tags: ``
โ Do: Run the tests very frequently, not longer than every few minutes during coding. If possible let it happen automatically, even continuously, while a developer is coding. The more frequent the tests run, the sooner they will discover issues. When they run automatically, the developer won't even need to remember to do anything - The tests are just there, watching her back like a robot assistant. When a component's size is relatively small, the tests can get executed in watch mode, so every code change will trigger a new run. Try this with our example app (includes live DB) - The test will show feedback in 3 seconds. Concerned with noise coming from the testing terminal? Put it in the background: Some test runner will show pop-up when the tests suddenly fail (e.g., Jest notify). There are also silent test runners like mocha-silent and jest-silent. You may also try our experimental watch mode extension that will run the tests every 30 seconds automatically in the background. Interested? Just open an issue
โช๏ธ 6. [Repeated Bullet] Consider testing the 5 known outcomes
#strategic
(This section also appear at the begining and is repeated here as it also integral part of the testing workflow)
โข Response - The test invokes an action (e.g., via API) and gets a response. It's now concerned with checking the response data correctness, schema, and HTTP status
โข A new state - After invoking an action, some data is probably modified. For example, when updating a user - It might be that the new data was not saved. Commonly and mistakenly, testers check only the response and not whether the data is updated correctly. Testing data and databases raises multiple interesting challenges that are greatly covered below in the ๐ section 'Dealing with data'
โข External calls - After invoking an action, the app might call an external component via HTTP or any other transport. For example, a call to send SMS, email or charge a credit card. Anything that goes outside and might affect the user - Should be tested. Testing integrations is a broad topic which is discussed in the
โข Message queues - The outcome of a flow might be a message in a queue. In our example application, once a new order was saved the app puts a message in some MQ product. Now other components can consume this message and continue the flow. This is very similar to testing integrations only working with message queues is different technically and tricky. The
โข Observability - Some things must be monitored, like errors or remarkable business events. When a transaction fails, not only we expect the right response but also correct error handling and proper logging/metrics. This information goes directly to a very important user - The ops user (i.e., production SRE/admin). Testing error handler is not very straighforward - Many types of errors might get thrown, some errors should lead to process crash, and there are many other corners to cover. We plan to write the
๐ Example application
In this folder you may find a complete example of real-world like application, a tiny Orders component (e.g. e-commerce ordering), including tests. We recommend skimming through this examples before or during reading the best practices. Note that we intentionally kept the app small enough to ease the reader experience. On top of it, a 'various-recipes' folder exists with additional patterns and practices - This is your next step in the learning journey
๐ช Recipes
More use cases and platforms. Each lives in its own folders:
- Nest.js
- Fastify (coming soon ๐ )
- Mocha
- Authentication
- Message Queue
- Testing OpenAPI (Swagger)
- Consumer-driven contract tests (coming soon ๐ )
- Data isolation patterns
- Optimized DB for testing
- Error handling
- Performance
The Team
The people who spent almost 1000 hours cumulatively to bring this content together
Yoni Goldberg
Independent Node.js consultant who works with customers in the USA, Europe, and Israel on building large-scale Node.js applications. Author of Node.js best practices. Holds testing workshops online, onsite and also a recorded course.
Michael Solomon
Started to program accidentally and fell in love. Strive for readable code. Chasing after perfection. Knowledge freak. Nothing is obvious. Backend developer.
Daniel Gluskin
Enthusiastic Node.js and javscript developer. Always eager to learn and explore new technologies.