• Stars
    star
    574
  • Rank 77,739 (Top 2 %)
  • Language
    Ruby
  • License
    MIT License
  • Created over 11 years ago
  • Updated 11 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

parallel test runner for CI environments

test-queue

Gem Version CI

Yet another parallel test runner, built using a centralized queue to ensure optimal distribution of tests between workers.

Specifically optimized for CI environments: build statistics from each run are stored locally and used to sort the queue at the beginning of the next run.

Usage

test-queue bundles testunit-queue, minitest-queue, and rspec-queue binaries which can be used directly:

$ minitest-queue $(find test/ -name \*_test.rb)
$ rspec-queue --format progress spec

But the underlying TestQueue::Runner::TestUnit, TestQueue::Runner::Minitest, and TestQueue::Runner::RSpec are built to be subclassed by your application. I recommend checking a new executable into your project using one of these superclasses.

$ vim script/test-queue
$ chmod +x script/test-queue
$ git add script/test-queue

Since test-queue uses fork(2) to spawn off workers, you must ensure each worker runs in an isolated environment. Use the after_fork hook with a custom runner to reset any global state.

#!/usr/bin/env ruby

class MyAppTestRunner < TestQueue::Runner::Minitest
  def after_fork(num)
    # Use separate mysql database (we assume it exists and has the right schema already)
    ActiveRecord::Base.configurations.configs_for(env_name: 'test', name: 'primary').database << num.to_s
    ActiveRecord::Base.establish_connection(:test)

    # Use separate redis database
    $redis.client.db = num
    $redis.client.reconnect
  end

  def prepare(concurrency)
    # Create mysql databases exists with correct schema
    concurrency.times do |i|
      # ...
    end

    # If this is a remote master, tell the central master something about us
    @remote_master_message = "Output for remote master 123: http://myhost.com/build/123"
  end

  def around_filter(suite)
    $stats.timing("test.#{suite}.runtime") do
      yield
    end
  end
end

MyAppTestRunner.new.execute

Environment variables

  • TEST_QUEUE_WORKERS: Number of workers to use per master (default: all available cores)
  • TEST_QUEUE_VERBOSE: Show results as they are available (default: 0)
  • TEST_QUEUE_SOCKET: Unix socket path (or TCP address:port pair) used for communication (default: /tmp/test_queue_XXXXX.sock)
  • TEST_QUEUE_RELAY: Relay results back to a central master, specified as TCP address:port
  • TEST_QUEUE_STATS: path to cache build stats in-build CI runs (default: .test_queue_stats)
  • TEST_QUEUE_FORCE: Comma separated list of suites to run
  • TEST_QUEUE_RELAY_TIMEOUT: When using distributed builds, the amount of time a remote master will try to reconnect to start work
  • TEST_QUEUE_RELAY_TOKEN: When using distributed builds, this must be the same on remote masters and the central master for remote masters to be able to connect.
  • TEST_QUEUE_REMOTE_MASTER_MESSAGE: When using distributed builds, set this on a remote master and it will appear in that master's connection message on the central master.
  • TEST_QUEUE_SPLIT_GROUPS: Split tests up by example rather than example group. Faster for tests with short setup time such as selenium. RSpec only. Add the :no_split tag to ExampleGroups you don't want split.

Design

test-queue uses a simple master + pre-fork worker model. The master exposes a Unix domain socket server which workers use to grab tests off the queue.

─┬─ 21232 minitest-queue master
 ├─── 21571 minitest-queue worker [3] - AuthenticationTest
 ├─── 21568 minitest-queue worker [2] - ApiTest
 ├─── 21565 minitest-queue worker [1] - UsersControllerTest
 └─── 21562 minitest-queue worker [0] - UserTest

test-queue also has a distributed mode, where additional masters can share the workload and relay results back to a central master.

Distributed mode

To use distributed mode, the central master must listen on a TCP port. Additional masters can be booted in relay mode to connect to the central master. Remote masters must provide a TEST_QUEUE_RELAY_TOKEN to match the central master's.

$ TEST_QUEUE_RELAY_TOKEN=123 TEST_QUEUE_SOCKET=0.0.0.0:12345 bundle exec minitest-queue ./test/example_test.rb
$ TEST_QUEUE_RELAY_TOKEN=123 TEST_QUEUE_RELAY=0.0.0.0:12345  bundle exec minitest-queue ./test/example_test.rb
$ TEST_QUEUE_RELAY_TOKEN=123 ./test-multi.sh

See the Parameterized Trigger Plugin for a simple way to do this with Jenkins.

See also

More Repositories

1

stackprof

a sampling call-stack profiler for ruby 2.2+
Ruby
2,084
star
2

rbtrace

like strace, but for ruby code
Ruby
1,691
star
3

perftools.rb

gperftools for ruby code
C
1,027
star
4

rblineprof

line-profiler for ruby
C
771
star
5

emoji-extractor

extracts high-resolution emoji pngs from /System/Library/Fonts/Apple Color Emoji.ttf
Ruby
548
star
6

ripper-tags

fast, accurate ctags generator for ruby source code using Ripper
Ruby
544
star
7

gctools

profiler/logger/oobgc for rgengc in ruby 2.1
C
258
star
8

hotspots

a graphical view into your rails app's performance
JavaScript
199
star
9

brew2deb

homebrew + fpm = debian packages
Ruby
108
star
10

sinbook

simple sinatra facebook extension in 300 lines of ruby
Ruby
84
star
11

graphite

git-bzr mirror of graphite trunk
Python
81
star
12

ruby-hacking-guide

ruby hacking guide english translation
Ruby
77
star
13

em-mysql

Async MySQL driver for Ruby/Eventmachine
Ruby
76
star
14

flyapp-mastodon

mastodon on fly.io
Dockerfile
71
star
15

jssocket

generic javascript socket API
JavaScript
57
star
16

em-spec

Simple BDD API for testing asynchronous Ruby/EventMachine code
Ruby
56
star
17

xmpp4em

EventMachine based XMPP client
Ruby
54
star
18

taxsim.app

US Income Tax simulator, built on NBER TAXSIM and taxsim.js
JavaScript
45
star
19

rmongo

Ruby/Eventmachine driver for 10gen's object database Mongo
Ruby
36
star
20

cycript

my personal git fork of http://svn.saurik.com/repos/cycript/trunk/
C++
27
star
21

taxsim.js

JS/WebAssembly version of NBER TAXSIM
HTML
21
star
22

taxes.cue

tax organizer and calculator in #cuelang
CUE
18
star
23

fiber18

API compatible Thread based Fiber implementation for Ruby 1.8
Ruby
18
star
24

ruby_posix_mq

Unofficial ruby_posix_mq Mirror. Updated semi-regularly.
Ruby
18
star
25

mailfactory

Create MIME email messages with multiple body parts and attachments in Ruby
Ruby
17
star
26

ruby187

fork of ruby 1.8.7p72 with branches for various gc and thread related patches
Ruby
16
star
27

apthunter

greasemonkey-style chrome extension to add keyboard navigation, stars, notes and inline images/maps to craigslist apartment listings
JavaScript
16
star
28

raindrops

my personal fork of http://repo.or.cz/w/raindrops.git
Ruby
14
star
29

nginxr

Nginxr is a ruby wrapper for nginx config file by oleganza
Ruby
12
star
30

memprof.com

memprof.com
JavaScript
11
star
31

ruby-collectd

Send collectd statistics from your Ruby script
Ruby
8
star
32

rvm.deb

debian package for rvm
8
star
33

ctags.rb

ruby wrapper for exuberant-ctags
Ruby
7
star
34

macvim_vimshell

macvim + vimshell patch
C
6
star
35

lingo

5
star
36

tablo-for-channels

M3U generator optimized for Channels' custom channels.
Go
5
star
37

async_sequel

asynchronous sequel datasets and models
5
star
38

debugproxy

TCP man-in-the-middle proxy
Ruby
5
star
39

nzbget

my git-svn fork of nzbget
C++
5
star
40

haproxy

my copy of haproxy-1.4.git
C
4
star
41

matzruby

A Git mirror of http://svn.ruby-lang.org/repos/ruby.
4
star
42

libmemcached.rb

simple, clean ruby wrapper for libmemcached 1.0
C
3
star
43

tmm1.github.com

tmm1.net
CSS
2
star
44

node-fflush

fflush(3)
C++
2
star
45

TomatoTent

A discreet Home Grow Box
C++
1
star
46

renderapp-mastodon

mastodon on render.com
Dockerfile
1
star
47

haproxy-dev

my copy of haproxy.git (1.5-dev)
C
1
star
48

androidhdmi-for-channels

androidhdmi-for-channels
Go
1
star