• Stars
    star
    264
  • Rank 154,483 (Top 4 %)
  • Language
    Groovy
  • License
    Apache License 2.0
  • Created about 11 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

This project creates a global extension to Spock to create test reports.

Spock Reports Extension

by Renato Athaydes

Check out the latest news about this project!

Actions Status Maven Central

What it is

This project is a global extension for Spock to create test (or, in Spock terms, Specifications) reports.

By default, the report creator generates a HTML report for each Specification, as well as a summary of all Specifications that have been run (index.html).

If you prefer to have your own template to generate reports from, you can use the TemplateReportCreator. This allows you to generate reports in any text format. See the "Using template reports" section below.

Support for Geb tests: if you use Geb for web testing, check out the geb-spock-reports extension which adds screenshots to Spock reports.

Where to find demo reports

I am using CodePen to design the HTML feature report, which contains detailed information about each Specification run by Spock, including the examples given (Where block) and their results, if any, and the summary report, which summarizes the results of all Specification runs. Click on the links to see the reports used for testing.

If you don't like the styles, you can use your own css stylesheets (see the customization section below). I welcome feedback on how to improve the report looks!

How to use it

To enable this Spock extension, you only need to declare a dependency to it (if using Maven, Ivy, Gradle etc) or, just add the jar to the project's classpath.

Spock-reports is available on Maven Central.

Version compatibility:

Java Groovy Spock spock-reports
8+ 4.0+ 2.3-groovy-4.0 2.5.0-groovy-4.0
8+ 3.0+ 2.3-groovy-3.0 2.5.0-groovy-3.0
8+ 2.5+ 2.0-groovy-2.5 2.0-groovy-2.5
7, 8 2.4+ 1.3+ 1.8.0

If you are using Maven

Add spock-reports to your <dependencies>:

<dependency>
  <groupId>com.athaydes</groupId>
  <artifactId>spock-reports</artifactId>
  <version>$spockReportsVersion</version>
  <scope>test</scope>
  <!-- this avoids affecting your version of Groovy/Spock -->
  <exclusions>
    <exclusion>
      <groupId>*</groupId>
      <artifactId>*</artifactId>
    </exclusion>
  </exclusions>
</dependency>

<!-- // if you don't already have slf4j-api and an implementation of it in the classpath, add this! -->
<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-api</artifactId>
  <version>1.7.36</version>
  <scope>test</scope>
</dependency>
<dependency>
  <groupId>org.slf4j</groupId>
  <artifactId>slf4j-simple</artifactId>
  <version>1.7.36</version>
  <scope>test</scope>
</dependency>

If you are using Gradle

repositories {
    mavenCentral()
}

test {
    useJUnitPlatform()
}

dependencies {
    // you can use testRuntimeClasspath if you don't want to use spock-report-specific features in your Specs
    testImplementation( "com.athaydes:spock-reports:$spockReportsVersion" ) {
        transitive = false // this avoids affecting your version of Groovy/Spock
    }
    // if you don't already have slf4j-api and an implementation of it in the classpath, add this!
    testImplementation 'org.slf4j:slf4j-api:1.7.36'
    testRuntimeClasspath 'org.slf4j:slf4j-simple:1.7.36'
}

If you prefer, you can just download the jar directly from Maven Central.

The only dependencies this project has are Groovy (only the groovy, groovy-templates, groovy-xml and groovy-json modules are required) and Spock, of course.

Adding information to reports programmatically

If you want to add information to your Spock-reports programmatically, since version 1.4.0, you can use the following Specification class' extension methods which are added by Spock Reports:

  • void reportHeader( arg ) - dynamically insert data into the top of the Specification report.
  • void reportInfo( arg ) - add data to the feature's section.

These methods are added as Groovy extensions, so your IDE should be able to show them in auto-completion!

For example, you could do something like this within your Specification:

def setupSpec() {
    reportHeader "<h2>Browser: ${driver.browser.name}</h2>"
}

def "My feature"() {
    expect:
    reportInfo "Some information I want to show in the report"
}

Customizing spock-reports logging

spock-reports uses the slf4j-api for logging, so you can get logging information to investigate any issues you may face with your Spock tests.

If your application does not have a slf4j implementation framework in the classpath, you may get this warning when running your tests:

SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

To get rid of the warning, add a dependency on a logging framework that implements the slf4j-api.

For example, to use slf4j-simple, add this line to your Gradle dependencies (or the equivalent XML in your Maven pom):

testCompile group: 'org.slf4j', name: 'slf4j-simple', version: '1.7.30'

To configure the logging framework itself, please check the documentation of the framework you decide to use. If you're using slf4j-simple, check this.

Most logging messages emitted by spock-reports use the DEBUG level, except when errors happen, in which case the WARN level is used.

The base spock-reports's logger name is com.athaydes.spockframework.report.

Customizing the reports

Spock-reports can be configured via the Spock config file, a dedicated properties file or system properties.

All properties listed below are supported either way.

Using SpockConfig.groovy

supported since version 1.5.0

All properties for spock-reports should be added in the spockReports block. To set properties, use the following syntax:

spockReports {
    set 'com.athaydes.spockframework.report.showCodeBlocks': true
    set 'com.athaydes.spockframework.report.outputDir': 'target/spock-reports'
}

Alternatively:

spockReports {
    // set all properties at once
    set( [ 'com.athaydes.spockframework.report.showCodeBlocks': true,
           'com.athaydes.spockframework.report.outputDir': 'target/spock-reports' ] )
}

Using a properties file

If you prefer to use a properties file, the file should be located at the following location (relative to the classpath):

META-INF/services/com.athaydes.spockframework.report.IReportCreator.properties

If you use Grails, the above location will not work... the correct location depends on the Grails version you're using. See the following blog posts by @rdmueller for instructions:

Using system properties

If you use Gradle and prefer system properties, they should be configured on the test task, e.g.:

task('functionalTest', type: Test) {
  systemProperty 'com.athaydes.spockframework.report.outputDir', 'build/reports/spock'
}

If you use Maven and prefer system properties, they should be configured as systemPropertyVariables in the configuration section of the failsafe and surefire plugins.

Default properties' values

# Name of the implementation class(es) of report creator(s) to enable (separate multiple entries with commas)
# Currently supported classes are:
#   1. com.athaydes.spockframework.report.internal.HtmlReportCreator
#   2. com.athaydes.spockframework.report.template.TemplateReportCreator
com.athaydes.spockframework.report.IReportCreator=com.athaydes.spockframework.report.internal.HtmlReportCreator

# Set properties of the report creator
# For the HtmlReportCreator, the only properties available are
# (the location of the css files is relative to the classpath):
com.athaydes.spockframework.report.internal.HtmlReportCreator.featureReportCss=spock-feature-report.css
com.athaydes.spockframework.report.internal.HtmlReportCreator.summaryReportCss=spock-summary-report.css
com.athaydes.spockframework.report.internal.HtmlReportCreator.printThrowableStackTrace=false
com.athaydes.spockframework.report.internal.HtmlReportCreator.inlineCss=true
com.athaydes.spockframework.report.internal.HtmlReportCreator.enabled=true
# options are: "class_name_and_title", "class_name", "title"
com.athaydes.spockframework.report.internal.HtmlReportCreator.specSummaryNameOption=class_name_and_title

# exclude Specs Table of Contents
com.athaydes.spockframework.report.internal.HtmlReportCreator.excludeToc=false

# Output directory (where the spock reports will be created) - relative to working directory
com.athaydes.spockframework.report.outputDir=build/spock-reports

# Output directory where to store the aggregated JSON report (used to support parallel builds)
com.athaydes.spockframework.report.aggregatedJsonReportDir=

# If set to true, hides blocks which do not have any description
com.athaydes.spockframework.report.hideEmptyBlocks=false

# Set the name of the project under test so it can be displayed in the report
com.athaydes.spockframework.report.projectName=

# Set the version of the project under test so it can be displayed in the report
com.athaydes.spockframework.report.projectVersion=Unknown

# Show the source code for each block
com.athaydes.spockframework.report.showCodeBlocks=false

# Set the root location of the Spock test source code (only used if showCodeBlocks is 'true')
com.athaydes.spockframework.report.testSourceRoots=src/test/groovy

# Set properties specific to the TemplateReportCreator
com.athaydes.spockframework.report.template.TemplateReportCreator.specTemplateFile=/templateReportCreator/spec-template.md
com.athaydes.spockframework.report.template.TemplateReportCreator.reportFileExtension=md
com.athaydes.spockframework.report.template.TemplateReportCreator.summaryTemplateFile=/templateReportCreator/summary-template.md
com.athaydes.spockframework.report.template.TemplateReportCreator.summaryFileName=summary.md
com.athaydes.spockframework.report.template.TemplateReportCreator.enabled=true

Notes on outputDir

Be aware that the outputDir property is relative to the working directory.

For Maven projects which use the defaults, you might want to change the outputDir to target/spock-reports.

If your build system can cache build outputs (to, for example, skip unnecessary build steps when in/out do not change), register the outputDir as an output of the test task.

In Gradle, for example, this can be accomplished with the following declaration in your build.gradle file:

test {
    // set to the same value as 'com.athaydes.spockframework.report.outputDir'
    outputs.dir "$buildDir/spock-reports"
}

Customizing the report stylesheets

The CSS properties above can be set to either of the following kinds of values:

  • a classpath resource.
  • a URL (the value will be used to call Java's new URL(value).

If the value does not match a full URL starting with a protocol (eg. file:///usr/local/css/report.css), the value will be treated as an absolute path to a classpath resource.

For example, if you set the value of a CSS property to my-css/test-report.css, the resource /my-css/test-report.css will be looked up in all Jars and directories which are part of the classpath.

If you set the value to http://myhost.com/css/test-report.css, the resource at this URL will be read.

Disabling CSS inlining

By default, the CSS resource will be inlined in the HTML report.

If you set the inlineCss property to false, then the CSS resource will be copied to the outputDir directory, together with the HTML reports, with the following names:

  • feature-report.css (for the featureReportCss property).
  • summary-report.css (for the summaryReportCss property).

A link to the CSS resources with the above names will be added to the HTML file instead of inlining the CSS.

Using template reports

If you don't like the looks of the HTML report, or want your reports in a different text format, you can use the TemplateReportCreator to do that.

All you need to do to get started is provide a config file, or system properties, as explained above that set the IReportCreator to com.athaydes.spockframework.report.template.TemplateReportCreator:

com.athaydes.spockframework.report.IReportCreator=com.athaydes.spockframework.report.template.TemplateReportCreator

# Set properties of the report creator
com.athaydes.spockframework.report.template.TemplateReportCreator.specTemplateFile=/templateReportCreator/spec-template.md
com.athaydes.spockframework.report.template.TemplateReportCreator.reportFileExtension=md
com.athaydes.spockframework.report.template.TemplateReportCreator.summaryTemplateFile=/templateReportCreator/summary-template.md
com.athaydes.spockframework.report.template.TemplateReportCreator.summaryFileName=summary.md

# Output directory (where the spock reports will be created) - relative to working directory
com.athaydes.spockframework.report.outputDir=build/spock-reports

# If set to true, hides blocks which do not have any description
com.athaydes.spockframework.report.hideEmptyBlocks=false

Just copy the above contents to a file at META-INF/services/com.athaydes.spockframework.report.IReportCreator.properties relative to the classpath (eg. in src/test/resources for Maven users) and spock-reports will create a MD (mark-down) report for your tests.

To provide your own template, change the location of the template files, the file extension you wish your reports to have, and the name for the summary report file, using the config file.

To get started with your own template, check the existing spec template file and the summary template.

You can see an example report created with the default spec template file here (this is actually used in the spock-reports tests).

How templates work

The template report creator uses Groovy's GStringTemplateEngine to create reports based on a template file.

This template mechanism is very simple to use, but also very powerful, as you can write any code you want in the template file.

There are two templates you should provide:

  • Spec report template: report for the run of a single Specification.
  • Summary template: contains a summary of all Specifications that have been run during a JVM lifetime.

Spec report template

Here's the most basic Spec template you could imagine, which simply outputs the name of the Specification that ran:

This is a Report for ${utils.getSpecClassName(data)}

As you can see, you can use ${variable} to run actual code whose result will be printed in the report. Another way to do this, is to use <% code %> blocks, as in the following example, which prints the name and result of all features in a Specification:

<%
    features.eachFeature { name, result, blocks, iterations, params ->
%>
Feature Name: $name
Result: $result
<%
    }
%>

NOTE: before version 1.2.6, eachFeature used to be called forEach. This had to be changed to avoid conflict with Java 8's method of the same name.

You probably noticed the use of some predefined variables in the template file code example. The available variables are:

  • data: an instance of SpecData containing the result of running a Specification.
  • reportCreator: the TemplateReportCreator instance.
  • fmt: an instance of StringFormatHelper. It provides methods such as String toTimeDuration( timeInMs ) and String escapeXml( String str ).
  • utils: the Utils class offers many useful methods like Map stats( SpecData data ), which returns statistics about the given Specification.
  • features: as shown above, is an Object whose eachFeature method allows you to iterate over all the features of a Specification. When inside the eachFeature closure, you can access directly all members of the current feature (an instance of FeatureInfo). So, for example, to get the Title annotation of a feature, you can call utils.specAnnotation( data, spock.lang.Title ).

As the default template file shows, you can get statistics for the Specification easily with this code snippet:

<% def stats = utils.stats( data ) %>
Report statistics: $stats

The variable stats is a Map containing the following keys:

failures, errors, skipped, totalRuns, totalFeatures, passed, successRate, time

So, you can use it in your template like this:

Features:           :  ${stats.totalFeatures}
Total number of runs:  ${stats.totalRuns}
Passed              :  ${stats.passed}
Number of failures..:  ${stats.failures}
Number of errors....:  ${stats.errors}
Number of ignored...:  ${stats.skipped}
Success rate........:  ${stats.successRate}
Total time (ms).....:  ${stats.time}

Created on ${new Date()} by ${System.properties['user.name']}

Summary template

The summary template has access to a single variable called data, which is a Map containing all the available data for all Specifications that were run.

For example, after running two Specifications named test.FirstSpec and test.SecondSpec, the data Map could look like this:

[
 'test.FirstSpec' : [ failures: 0, errors: 0, skipped: 2, totalRuns: 3, totalFeatures: 4, passed: 5, successRate: 0.1, time: 1000 ],
 'test.SecondSpec': [ failures: 0, errors: 1, skipped: 3, totalRuns: 4, totalFeatures: 6, passed: 6, successRate: 0.2, time: 2000 ],
]

You can then iterate over each Spec's data as follows:

<% data.each { name, map ->
      def s = map.stats
 %>| $name | ${s.totalRuns} | ${s.failures} | ${s.errors} | ${s.skipped} | ${s.successRate} | ${s.time} |
<% }
 %>

Check the default summary template for a full example.

Submitting pull requests

Please submit pull requests with bug fixes at any time!!

But if your Pull Request is about a new feature, please make sure to create an issue first so that we can all discuss whether it's a good idea and what's the best way to go about it.

Also, please notice that the master branch is supposed to contain only releases... the development branch is called dev, so all PRs should be submitted against dev, not master.

Thank you.

More Repositories

1

LogFX

LogFX is a simple Log reader supporting color highlighting and able to handle giant files.
Java
191
star
2

rawhttp

HTTP library to make it easy to deal with raw HTTP.
Java
190
star
3

go-hash

Small utility to store secret information like passwords.
Go
104
star
4

Automaton

Simple framework which allows the testing of Swing and JavaFX2 applications.
Groovy
96
star
5

kunion

Union types for Kotlin
Kotlin
83
star
6

osgi-run

Osgi-Run - A Gradle plugin to make the development of modular applications using OSGi completely painless
Groovy
53
star
7

actors

Actor Model library for Dart.
Dart
47
star
8

mako-smarthome

Mako Lua Server for managing SmartHome devices based on the deCONZ API
Lua
34
star
9

prechelt-phone-number-encoding

Comparison between Java and Common Lisp solutions to a phone-encoding problem described by Prechelt
Java
30
star
10

jvm-alternatives-to-js

Repository comparing JVM alternatives to JS: CheerpJ, GWT, JSweet, TeaVM, Vaadin Flow, bck2brwsr (bonus: React, Dart)
Java
30
star
11

kotlin-hidden-costs-benchmark

A JMH benchmark of Kotlin hidden costs.
Java
29
star
12

wasm-on-jvm

A Gradle Plugin to compile WASM to JVM easily
Kotlin
24
star
13

ceylon-gradle-plugin

A simple Gradle plugin to manage Ceylon projects.
Groovy
22
star
14

jersey-guice-app

Application demonstrating a web app which provides a RESTful API through Jersey. It uses Guice as its Dependency Injection framework.
Java
18
star
15

osgiaas

OSGiaaS - OSGi as a Service
Java
17
star
16

jgrab

Runs Java code without a build system, grabbing dependencies declared in the Java file itself.
Java
17
star
17

magnanimous

The simplest and fastest static website generator in the world!!
Go
16
star
18

structured_async

Structured concurrency for the Dart Programming Language.
Dart
15
star
19

zig-common-tasks

Zig common tasks (code samples)
Zig
15
star
20

dartle

A simple build system written in Dart.
Dart
14
star
21

javanna

A Java library to create and introspect annotations at runtime.
Java
14
star
22

jbuild

JBuild is a intentionally simple, small build tool for Java.
Java
14
star
23

CeylonFX

Ceylon interface for JavaFX
Ceylon
13
star
24

wasmin

A programming language that is a thin layer over pure WebAssembly (WASM).
Dart
13
star
25

specks

Specks enables a different way to check that your Ceylon code works
Ceylon
13
star
26

Grasmin

Groovy AST Transformation to allow writing Jasmin code (JVM bytecode) directly on groovy files
Groovy
13
star
27

raw-jse

Vanilla Java: Using Java SE as a Framework
Java
12
star
28

parcey

A combinator parser for Ceylon
Ceylon
11
star
29

pony-gradle-plugin

A Gradle plugin to build Pony projects
Groovy
9
star
30

gohash_mobile_app

go-hash official mobile app (Android and iOS)
Dart
8
star
31

eventory

An event sourcing-inspired library targeting offline-first use, with automatic synchronization with remote instances when connection is available.
Dart
8
star
32

emacs.d

Personal emacs.d configuration
Emacs Lisp
7
star
33

gohash_mobile

go-hash Flutter Plugin
Objective-C
7
star
34

jb

The JBuild CLI.
Dart
6
star
35

faster-command-line-tools-kotlin

Faster command line applications with Kotlin (blog post backing repo)
Kotlin
6
star
36

ConcurrenCey

ConcurrenCey is a concurrency framework for the Ceylon language
Ceylon
6
star
37

isolate_current_directory

Support for changing the current working directory only within the scope of a function.
Dart
6
star
38

h_view

A wrk2 histogram viewer
Dart
5
star
39

ansi-color

A Racket library to make it easy to colorize terminal output
Racket
5
star
40

OsgiMonitor

This Project provides a set of bundles to allow users to monitor and control an OSGi instance.
Groovy
4
star
41

flattery

Flattery is a library for building HTML elements using Widgets.
Dart
4
star
42

zig-build-c

Zig Demo building and testing C code
Zig
4
star
43

kanvas

Kotlin and Groovy Canvas DSL based on JavaFX
Kotlin
4
star
44

wasmin-lang

A minimal WASM-focused programming language.
Rust
4
star
45

open_url

Dart lib to open a URL on the user's default application.
Dart
3
star
46

vinegar

A collection of functions and macros to help testing Rust code.
Rust
3
star
47

functions4j

Functions4J attempts to provide high-level functional constructs to Java 8+.
Java
3
star
48

vscode-ponylang

VSCode plugin for the Pony programming language.
TypeScript
3
star
49

spark-ws

Spark-WS is a Java Websockets library inspired by Spark for Java (which is inspired by Sinatra for Ruby).
Java
2
star
50

kootlin

Pure Kotlin OOP
Kotlin
2
star
51

rawhttp-tutorial

Source code for RawHTTP blog post.
Kotlin
2
star
52

kyang

A Yang IntelliJ Plugin
Kotlin
2
star
53

swing-selectors

Swing selectors allows declaratively selecting Swing components anywhere in a component tree - a breakoff from the Automaton testing framework
Groovy
2
star
54

dzipper

A CLI utility and library to extract zip file metadata.
D
2
star
55

mergequick

A web app to help manage and merge Pull Ruquests on popular code hosts.
Dart
1
star
56

simple-fx-app

Very simple JavaFX Application. Used for testing Automaton.
Java
1
star
57

MachineLearning

In this project, I implement some machine learning / data mining algorithms. The preferred language is Groovy. Email: [email protected]
Groovy
1
star
58

rpc-examples

Examples of implementations of services based on RPC frameworks.
Java
1
star
59

pathtrie

A Trie implementation specifically designed for paths
Java
1
star
60

ts-parcel

Basic TypeScript Browser App packaged with Parcel
HTML
1
star
61

dart-bf

brainfuck implementation in Dart
Dart
1
star
62

apps-bouncer

Application that keeps track of running processes, snitching bad behaving ones to the user, who can choose to kill them.
Dart
1
star
63

dart-sass-bulma-example

Example Dart Web Project that uses Sass and Bulma
HTML
1
star
64

checker-maven-demo

This project demonstrates the use of the checker framework with Maven.
Java
1
star
65

FxCodeEditor

A source code editor
Java
1
star
66

easy-jetty

Makes it really easy to embed a Jetty Web Server
Java
1
star
67

keepup

Tiny Java app self-updater designed to work with Java 11+ jlink distributions.
Java
1
star
68

protobuf-tcp-rsa-provider

TCP/Protobuffer implementation of Aries RSA DistributionProvider.
Java
1
star
69

ansi-escapes.zig

ANSI Escapes (colors, styles) for Zig Terminal-based applications
Zig
1
star
70

gradle-multi-java-modules-sample

This project shows how to organize Java modules in a Gradle project.
Java
1
star
71

test_report_parser.dart

Dart Model representing the events emitted by the Dart Tests JSON reporter
Dart
1
star
72

ceylon-medical-web-app

Example application in Ceylon using the type system to create reliable, extensible software
Ceylon
1
star