How to run Gradle test when all tests are UP-TO-DATE?

153
votes

I have my grade script set up. When I execute the Gradle build, everything is working and it runs the jUnit tests.

After that when I run the Gradle test I get the following:

C:\Users\..\..\Project>gradle test
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test UP-TO-DATE

When I perform gradle clean, then Gradle build works, of course... I want to be able to reset only the tests, not build the whole project: how should I do this?

9
javagradlebuild.gradle
This seems unnecessary, based on the given information. If neither the application code nor the test code have changed, why do you need to re-run the tests? – Jolta
@Jolta Some of the tests in my code are related on 3-party inputs, I'm running my tests not only to make sure i did not put any bug inside the code, also to check if something change on the 3-party inputs that i'm getting – USer22999299
Sorry to be nit-picky, but I don't really think this is the correct way of thinking about this: if you have variable 3-party inputs isn't the correct way of dealing with this to mock these inputs in some way? Testing should actually be about testing the code you're writing. Aren't you in fairly obvious danger of getting false-positives if you're relying on 3-party input to be unacceptable? Shouldn't the strategy be to cater for problem input as part of your app code? – mike rodent
@mikerodent consider testing your code against a 3rd party online service. You would want to monitor possible changes in the service API to be able to respond with deployed fixes ASAP. Isn't CI tests a good way of doing that? Using a mock will only tell you your own code doesn't have regressions, but the dependencies still might have changes. using the real service would indicate that your product can actually perform the expected operations in the current environment. – Elist
This is also valid from an integration testing point of view where the point of the test is to validate the integration of your code with other bits of code, where it would be not appropriate to mock in dependancies – 1800 INFORMATION

9 Answers

198
votes

One option would be using the --rerun-tasks flag in the Forcing tasks to execute section. This would rerun all the the test task and all the tasks it depends on.

If you're only interested in rerunning the tests then another option would be to make gradle clean the tests results before executing the tests. This can be done using the cleanTest task.

Some background - the Java plugin defines a clean tasks to each of the other tasks. According to the Tasks documentation:

cleanTaskName - Deletes files created by specified task. cleanJar will delete the JAR file created by the jar task, and cleanTest will delete the test results created by the test task.

Therefore, all you need in order to re-run your tests is to also run the cleanTest task, i.e.:
gradle cleanTest test

60
votes

Other option would be to add following in your build.gradle:

test.outputs.upToDateWhen {false}
27
votes

gradle test --rerun-tasks

Specifies that any task optimization is ignored.

Source: https://gradle.org/docs/current/userguide/gradle_command_line.html

24
votes

This was recently the topic on Gradle's blog post Stop rerunning your tests. The author shows an example using outputs.upToDateWhen { false } and explains why it is wrong:

This doesn’t actually force reruns

What the author of this snippet probably wanted to say is “Always rerun my tests”. That’s not what this snippet does though. It will only mark the task out-of-date, forcing Gradle to recreate the output. But here’s the thing, if the build cache is enabled, Gradle doesn’t need to run the task to recreate the output. It will find an entry in the cache and unpack the result into the test’s output directory.

The same is true for this snippet:

test.dependsOn cleanTest

Gradle will unpack the test results from the build cache after the output has been cleaned, so nothing will be rerun. In short, these snippets are creating a very expensive no-op.

If you’re now thinking “Okay, I’ll deactivate the cache too”, let me tell you why you shouldn’t.

Then, the author goes on to explain why rerunning some tests is a waste of time:

The vast majority of your tests should be deterministic, i.e. given the same inputs they should produce the same result.

In the few cases where you do want to rerun tests where the code has not changed, you should model them as an input. Here are both examples from the blog post that show adding an input so the task will use it during its up-to-date checks.

task randomizedTest(type: Test) {
  systemProperty "random.testing.seed", new Random().nextInt()
}

task systemIntegrationTest(type: Test) {
  inputs.property "integration.date", LocalDate.now()
}

I recommend reading the entire blog post.

16
votes

--rerun-tasks works, but is inefficient as it reruns all tasks.

cleanTest by itself may not suffice due to build cache.

so, best way to accomplish this is:

./gradlew --no-build-cache cleanTest test
11
votes

Here's a solution using the "build.gradle" file, in case you don't want to modify your command line:

test {
    dependsOn 'cleanTest'
    //Your previous task details (if any)
}

And here's the output. Notice 2 changes from your previous output:

1) A new 'cleanTest' task appears in the output.

2) 'test' is always cleaned (i.e. never 'UP-TO-DATE') so it gets executed every time:

$ gradle build
:compileJava UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:findMainClass
:jar
:bootRepackage
:assemble
:cleanTest
:compileTestJava UP-TO-DATE
:processTestResources UP-TO-DATE
:testClasses UP-TO-DATE
:test
:check
:build
0
votes

Also, having to add --rerun-tasks is really redundant. Never happens. Create a --no-rerun-tasks and make --rerun-tasks default when cleanTask

0
votes

None of the above methods worked for me.
What worked for me was simply removing all items from the cache directory in /Users/<username>/.gradle/caches/build-cache-1/.

-1
votes

TL;DR

test.dependsOn cleanTest