• Stars
    star
    598
  • Rank 71,890 (Top 2 %)
  • Language
    Groovy
  • License
    Apache License 2.0
  • Created almost 10 years ago
  • Updated 8 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Integrates the JMH benchmarking framework with Gradle

JMH Gradle Plugin

Build Status Coverage Status (coveralls) Download Apache License 2

This plugin integrates the JMH micro-benchmarking framework with Gradle.

Usage

build.gradle
plugins {
  id "me.champeau.jmh" version "0.7.1"
}
Warning
Versions of the plugin prior to 0.6.0 used the me.champeau.gradle.jmh plugin id.

Samples can be found in the samples folder.

What plugin version to use?

Version 0.6+ requires Gradle 6.8+.

Gradle Minimal plugin version

8.0

0.7.0

7.0

0.5.3

5.5

0.5.0

5.1

0.4.8

4.9

0.4.7 (to benefit from lazy tasks API)

4.8

0.4.5

4.7

0.4.5

4.6

0.4.5

4.5

0.4.5

4.4

0.4.5

4.3

0.4.5

4.2

0.4.4

4.1

0.4.4

Configuration

The plugin makes it easy to integrate into an existing project thanks to a specific configuration. In particular, benchmark source files are expected to be found in the src/jmh directory:

src/jmh
     |- java       : java sources for benchmarks
     |- resources  : resources for benchmarks

The plugin creates a jmh configuration that you should use if your benchmark files depend on a 3rd party library. For example, if you want to use commons-io, you can add the dependency like this:

build.gradle
dependencies {
    jmh 'commons-io:commons-io:2.7'
}

The plugin uses JMH 1.36. You can upgrade the version just by changing the version in the dependencies block:

build.gradle
dependencies {
    jmh 'org.openjdk.jmh:jmh-core:0.9'
    jmh 'org.openjdk.jmh:jmh-generator-annprocess:0.9'
}

Tasks

The project will add several tasks:

  • jmhClasses : compiles raw benchmark code

  • jmhRunBytecodeGenerator : runs bytecode generator over raw benchmark code and generates actual benchmarks

  • jmhCompileGeneratedClasses : compiles generated benchmarks

  • jmhJar : builds the JMH jar containing the JMH runtime and your compiled benchmark classes

  • jmh : executes the benchmarks

The jmh task is the main task and depends on the others so it is in general sufficient to execute this task:

gradle jmh

Configuration options

By default, all benchmarks will be executed, and the results will be generated into $buildDir/reports/jmh. But you can change various options thanks to the jmh configuration block. All configurations variables apart from includes are unset, implying that they fall back to the default JMH values:

build.gradle
jmh {
   includes = ['some regular expression'] // include pattern (regular expression) for benchmarks to be executed
   excludes = ['some regular expression'] // exclude pattern (regular expression) for benchmarks to be executed
   iterations = 10 // Number of measurement iterations to do.
   benchmarkMode = ['thrpt','ss'] // Benchmark mode. Available modes are: [Throughput/thrpt, AverageTime/avgt, SampleTime/sample, SingleShotTime/ss, All/all]
   batchSize = 1 // Batch size: number of benchmark method calls per operation. (some benchmark modes can ignore this setting)
   fork = 2 // How many times to forks a single benchmark. Use 0 to disable forking altogether
   failOnError = false // Should JMH fail immediately if any benchmark had experienced the unrecoverable error?
   forceGC = false // Should JMH force GC between iterations?
   jvm = 'myjvm' // Custom JVM to use when forking.
   jvmArgs = ['Custom JVM args to use when forking.']
   jvmArgsAppend = ['Custom JVM args to use when forking (append these)']
   jvmArgsPrepend =[ 'Custom JVM args to use when forking (prepend these)']
   humanOutputFile = project.file("${project.buildDir}/reports/jmh/human.txt") // human-readable output file
   resultsFile = project.file("${project.buildDir}/reports/jmh/results.txt") // results file
   operationsPerInvocation = 10 // Operations per invocation.
   benchmarkParameters =  [:] // Benchmark parameters.
   profilers = [] // Use profilers to collect additional data. Supported profilers: [cl, comp, gc, stack, perf, perfnorm, perfasm, xperf, xperfasm, hs_cl, hs_comp, hs_gc, hs_rt, hs_thr, async]
   timeOnIteration = '1s' // Time to spend at each measurement iteration.
   resultFormat = 'CSV' // Result format type (one of CSV, JSON, NONE, SCSV, TEXT)
   synchronizeIterations = false // Synchronize iterations?
   threads = 4 // Number of worker threads to run with.
   threadGroups = [2,3,4] //Override thread group distribution for asymmetric benchmarks.
   timeout = '1s' // Timeout for benchmark iteration.
   timeUnit = 'ms' // Output time unit. Available time units are: [m, s, ms, us, ns].
   verbosity = 'NORMAL' // Verbosity mode. Available modes are: [SILENT, NORMAL, EXTRA]
   warmup = '1s' // Time to spend at each warmup iteration.
   warmupBatchSize = 10 // Warmup batch size: number of benchmark method calls per operation.
   warmupForks = 0 // How many warmup forks to make for a single benchmark. 0 to disable warmup forks.
   warmupIterations = 1 // Number of warmup iterations to do.
   warmupMode = 'INDI' // Warmup mode for warming up selected benchmarks. Warmup modes are: [INDI, BULK, BULK_INDI].
   warmupBenchmarks = ['.*Warmup'] // Warmup benchmarks to include in the run in addition to already selected. JMH will not measure these benchmarks, but only use them for the warmup.

   zip64 = true // Use ZIP64 format for bigger archives
   jmhVersion = '1.36' // Specifies JMH version
   includeTests = true // Allows to include test sources into generate JMH jar, i.e. use it when benchmarks depend on the test classes.
   duplicateClassesStrategy = DuplicatesStrategy.FAIL // Strategy to apply when encountring duplicate classes during creation of the fat jar (i.e. while executing jmhJar task)
}

JMH Options Mapping

The following table describes the mappings between JMH’s command line options and the plugin’s extension properties.

JMH Option Extension Property

-bm <mode>

benchmarkMode

-bs <int>

batchSize

-e <regexp+>

exclude

-f <int>

fork

-foe <bool>

failOnError

-gc <bool>

forceGC

-i <int>

iterations

-jvm <string>

jvm

-jvmArgs <string>

jvmArgs

-jvmArgsAppend <string>

jvmArgsAppend

-jvmArgsPrepend <string>

jvmArgsPrepend

-o <filename>

humanOutputFile

-opi <int>

operationsPerInvocation

-p <param={v,}*>

benchmarkParameters?

-prof <profiler>

profilers

-r <time>

timeOnIteration

-rf <type>

resultFormat

-rff <filename>

resultsFile

-si <bool>

synchronizeIterations

-t <int>

threads

-tg <int+>

threadGroups

-to <time>

timeout

-tu <TU>

timeUnit

-v <mode>

verbosity

-w <time>

warmup

-wbs <int>

warmupBatchSize

-wf <int>

warmupForks

-wi <int>

warmupIterations

-wm <mode>

warmupMode

-wmb <regexp+>

warmupBenchmarks

Dependency on project files

The jmh plugin makes it easy to test existing sources without having to create a separate project for this. This is the reason why you must put your benchmark source files into src/jmh/java instead of src/main/java. This means that by default, the jmh (benchmarks) task depends on your main (production) source set.

It is possible a dependency on the test source set by setting property includeTests to true inside jmh block.

Using JMH Gradle Plugin with Shadow Plugin

Optionally it is possible to use Shadow Plugin to do actual JMH jar creation. The configuration of Shadow Plugin for JMH jar is done via jmhJar block. For example:

build.gradle
jmhJar {
  append('META-INF/spring.handlers')
  append('META-INF/spring.schemas')
  exclude 'LICENSE'
}

Duplicate dependencies and classes

This plugin will merge all dependencies that are defined as part of jmh, runtime and optionally testRuntime configurations into a single set from which fat jar will be created when executing jmhJar task. This is done to ensure that no duplicate dependencies will be added the generated jar.

In addition plugin applies DuplicatesStrategy defined via duplicateClassesStrategy extension property to every class while creating fat jar. By default this property is set to DuplicatesStrategy.FAIL which means that upon detection of duplicate classes the task will fail.

It is possible to change this behavior by configuring duplicateClassesStrategy property via jmh block, e.g.:

build.gradle
jmh {
  duplicateClassesStrategy = DuplicatesStrategy.WARN
}

However if you do encounter problem with defaut value it means that the classpath or sources in your project do contain duplicate classes which means that it is not possible to predict which one will be used when fat jar will generated.

To deal with duplicate files other than classes use Shadow Plugin capabilities, see Using JMH Gradle Plugin with Shadow Plugin.

More Repositories

1

deck2pdf

Converts various HTML5 slide decks to PDF
Java
241
star
2

japicmp-gradle-plugin

A Gradle plugin for JApicmp
Java
134
star
3

jdoctor

A Java library for designing good error messages
Java
133
star
4

gr8confagenda

Source code for the GR8Conf Agenda Android application
Groovy
118
star
5

includegit-gradle-plugin

A Gradle plugin to include Git repositories
Java
95
star
6

asm-bytecode-intellij

An ASM (http://asm.ow2.org/) plugin for IntelliJ IDEA
Java
69
star
7

grooidshell-example

An example of usage of dynamic classes at runtime using Groovy on Android
Java
45
star
8

groovy-bytecode-ast

A Groovy AST transformation which allows writing the body of a method as bytecode instructions.
Groovy
45
star
9

gradle-buildscan-recipes

A Gradle plugin for Build Scans
Groovy
43
star
10

antlr4-gradle-plugin

Antlr4 plugin for Gradle
Groovy
40
star
11

ast-workshop

Unleashing the power of AST transformations
Shell
38
star
12

jmh-gradle-example

Sample project showcasing the JMH gradle plugin
Java
38
star
13

jlangdetect

A language detection library for the JVM
Java
33
star
14

mrjar-gradle

Example of how to produce a MRJAR with Gradle
Groovy
21
star
15

speakertime

A demo project for Groovy and Android, as seen in Devoxx
Groovy
20
star
16

astro4j

Astronomy libraries for Java
Java
18
star
17

mrjar-gradle-plugin

A multi-release JAR plugin for Gradle
Java
18
star
18

springboot-groovytemplates

A demo app that makes use of Spring Boot, Groovy templates and GORM
Groovy
15
star
19

gradle-presentation-template

A presentation template using Asciidoctor+Reveal.js and Gradle
Kotlin
14
star
20

gradle-6-whats-new

What's new in Gradle 6?
Kotlin
13
star
21

gradle-cf-plugin

Gradle plugin for CloudFoundry
Groovy
13
star
22

gradle-tapi-demo-artifacts

A demonstration of the Gradle Tooling API: inspecting Gradle builds to get some information about them
Java
12
star
23

teamcity-groovy-buildstep

A TeamCity plugin which adds the ability to write a build step using Groovy
JavaScript
10
star
24

openbeans

Automatically exported from code.google.com/p/openbeans
Java
9
star
25

codenarc-idea

Java
9
star
26

blog

My blog
CSS
8
star
27

gradle-wasm-plugin

A POC of writing Gradle tasks using wasm as implementation
Java
7
star
28

micronaut-test-resources-demo

Java
7
star
29

lecharny-challenge

The Lecharny Challenge
Java
7
star
30

gradle-maven-exec

Embedding Maven build in Gradle
Kotlin
6
star
31

graal-simple-httpserver

A simple HTTP server using Kotlin, Groovy and Gradle
Kotlin
6
star
32

version-catalog-poc

Kotlin
5
star
33

grithub

A Groovy API for GitHub Java API
Groovy
5
star
34

robovm-groovy-test

A test project for Groovy on RoboVM
Groovy
4
star
35

spring-groovymarkup

Support classes for using MarkupTemplateEngine from Spring
Java
4
star
36

maven-repository-injection

A POC which demonstrates repository injection with Apache Maven
Java
4
star
37

staticbuilder

Statically compiled markup builder for Groovy
Java
4
star
38

javaone2016-juggling-jigsaw

Groovy
3
star
39

advisories-gradle-plugin

A plugin which scans your dependencies for vulnerabilities looking at GitHub advisories database
Kotlin
3
star
40

groovy-guava

A Groovier API for Google Guava
Groovy
3
star
41

javaone-groovy-dsls

HTML
3
star
42

paseq-gradle-plugin-poc

Proof of concept of paseq plugin for Gradle
Java
3
star
43

poc-gradle-git-demo

Demonstrates generation of a class from Git info
Java
3
star
44

gradle-variant-aware-dependency-resolution

Java
2
star
45

devoxxbe2015-groovy-dsls

HTML
2
star
46

finistjug-fast-builds

HTML
2
star
47

gradlesummit2016-jigsaw-gradle

Java
2
star
48

gradle-compile-avoidance-current-model

Groovy
2
star
49

gradle-repl

Kotlin
2
star
50

kotlin-mpp-demo

Kotlin
2
star
51

s2gx-groovy-dsls

Slides for S2GX 2015 "Building modern DSLs in Groovy"
HTML
2
star
52

bdxjug2015

Talk at BordeauxJUG, May 2015
Groovy
2
star
53

gradle-download-helper

Java
2
star
54

rivieradev-composite-builds

Talk about composite builds at RivieraDev 2017
HTML
2
star
55

benchmark-groovy-gstring

Benchmark for String concatenation in Groovy
Groovy
2
star
56

gr8conf2016-intro-software-model

Introduction to the new Java software model in Gradle
Groovy
2
star
57

gradle-javac-dependencies-plugin

Experimental javac plugin
Java
2
star
58

jroller-export

Exports from JRoller.com to JBake compatible format
Shell
2
star
59

micronaut-license-report

An aggregating project to generate a license report
Kotlin
1
star
60

s2gx-deepdive-groovy-compiler

Slides for S2GX 2015 "Deep dive into the Groovy compiler"
Groovy
1
star
61

micronaut-aot-demo

Java
1
star
62

gradle-dependencies-dump

Playground for dumping gradle dependencies
Java
1
star
63

s2gx-2013

SpringOne2GX 2013 slideware
JavaScript
1
star
64

gr8conf2016-deepdive-groovy-compiler

Deep dive into the Groovy compiler - GR8Conf Europe 2016
Groovy
1
star
65

devoxxfr2013

Slides and source code for Devoxx France 2013 conferences
Shell
1
star
66

ggx2013

Slides for my Groovy & Grails eXchange 2013 talk
JavaScript
1
star
67

CodeStory

Concours CodeStory
Groovy
1
star
68

webinar-dep-mgmt-part-2

Kotlin
1
star
69

micronaut-uber-build

Java
1
star
70

gr8conf2016-variant-aware-dep-mgmt

Slides for "Variant aware dependency management with Gradle"
HTML
1
star
71

talks

Slides for various talks
JavaScript
1
star
72

gradle-global-config-consistency-plugin

Kotlin
1
star
73

virtualjug-fast-builds

HTML
1
star
74

paris-android-ug-fast-builds

Slides for my Paris Android User Group talk
HTML
1
star
75

micronaut-svelte-demo

Java
1
star
76

javaone-2017-max-incremental

Slides for my JavaOne 2017 talk : minutes to seconds - maximizing incrementality
HTML
1
star
77

javaone-2017-jigsaw

Slides for my JavaOne 2017 talk : building and testing Java 9 applications with Gradle
HTML
1
star
78

groovy-script-extension-example

Java
1
star
79

sphero-playground

Go
1
star