We have some tests in our build that are flaky. We should consider making use of Gradle's flaky test mitigation to help us to identify these tests. It would allow us to decide whether or not we want flaky tests to fail the build, while also making it easier for us to identify them via the test reports and build scans. We could continue to make code changes to main code where the flakiness is our fault and to test code where the flakiness is caused by third-party code and we need to work around it.
Comment From: wilkinsona
We're going to add the plugin and configure it to fail the build when there is a flaky test.
Comment From: mikesmithson
Can a PR be submitted for this issue? If so, do you want this plugin configured at the root for all modules or module-specific ? My thinking is at the root build.gradle file, configure the plugin like so:
plugins {
id "org.gradle.test-retry" version "1.1.3"
}
...
...
test {
retry {
maxRetries = 1
maxFailures = 1
failOnPassedAfterRetry = true
}
}
Comment From: wilkinsona
Thanks for the offer, @mikesmithson. A pull request would be most welcome.
Rather than changing the root build.gradle
, I think this should be handled in buildSrc
. I'd imagined a dependency on org.gradle:test-retry-gradle-plugin:1.1.3
add then some code in our ConventionsPlugin
that reacts to the Java plugin being applied by also applying and configuring the test retry plugin.
Comment From: mikesmithson
@wilkinsona - made the requested changes and pushed to my forked branch. Thanks...