Gradle compiles but does not run TestNG tests

Question

We just started using Gradle and TestNG on our project, so I am checking whether failing any test actually fails the build. I was quite surprised to see it does not. The tests are picked up and compiled correctly, so I see the class files. Also I do get a report of the run, but it says 0 tests (expected 2). Running gradle clean test -i gives me the following:

:contentplatform-service:compileTestJava (Thread[Daemon worker Thread 7,5,main])
 started.
:contentplatform-service:compileTestJava
Executing task ':contentplatform-service:compileTestJava' (up-to-date check took
 0.08 secs) due to:
  Output file D:\Dev\contentplatform-service\build\classes\test has changed.
  Output file D:\Dev\contentplatform-service\build\dependency-cache has changed.

  Output file D:\Dev\contentplatform-service\build\classes\test\nl\xillio\conten
tplatform\service\SuperSimpleTest.class has been removed.
All input files are considered out-of-date for incremental task ':contentplatfor
m-service:compileTestJava'.
Compiling with JDK Java compiler API.
:contentplatform-service:compileTestJava (Thread[Daemon worker Thread 7,5,main])
 completed. Took 0.229 secs.
:contentplatform-service:processTestResources (Thread[Daemon worker Thread 7,5,m
ain]) started.
:contentplatform-service:processTestResources
Skipping task ':contentplatform-service:processTestResources' as it has no sourc
e files.
:contentplatform-service:processTestResources UP-TO-DATE
:contentplatform-service:processTestResources (Thread[Daemon worker Thread 7,5,m
ain]) completed. Took 0.001 secs.
:contentplatform-service:testClasses (Thread[Daemon worker Thread 7,5,main]) sta
rted.
:contentplatform-service:testClasses
Skipping task ':contentplatform-service:testClasses' as it has no actions.
:contentplatform-service:testClasses (Thread[Daemon worker Thread 7,5,main]) com
pleted. Took 0.001 secs.
:contentplatform-service:test (Thread[Daemon worker Thread 7,5,main]) started.
:contentplatform-service:test
Executing task ':contentplatform-service:test' (up-to-date check took 0.049 secs
) due to:
  Output file D:\Dev\contentplatform-service\build\test-results\binary\test has
changed.
  Output file D:\Dev\contentplatform-service\build\test-results has changed.
  Output file D:\Dev\contentplatform-service\build\reports\tests has changed.
Finished generating test XML results (0.0 secs) into: D:\Dev\contentplatform-ser
vice\build\test-results
Generating HTML test report...
Finished generating test html results (0.014 secs) into: D:\Dev\contentplatform-
service\build\reports\tests
:contentplatform-service:test (Thread[Daemon worker Thread 7,5,main]) completed.
 Took 0.194 secs.

SuperSimpleTest.java:

package nl.xillio.contentplatform.service;

import org.testng.Assert;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.Test;

@Test
public class SuperSimpleTest {

    @BeforeClass
    public void setUp() {
        // code that will be invoked when this test is instantiated
    }

    @Test
    public void testTest() {
        Assert.assertEquals(true, true);
    }
}

build.gradle contains:

test {
    // enable TestNG support (default is JUnit)
    useTestNG()
    scanForTestClasses = false  
    include '**/*'

    testLogging {
        showStandardStreams = true

        // log results to "build/test-results" directory
        exceptionFormat "full"
        events "started", "passed", "skipped", "failed", "standardOut", "standardError"
    }
}

I already took a look at other questions about this topic, and there I found the hint to use scanForTestClasses = false as a workaround (see https://issues.gradle.org/browse/GRADLE-1682). However, this problem seems unrelated. Am I making some other noob mistake here? How can I get SuperSimpleTest to just execute?

UPDATE: I tried forcing gradle to run a specific test with an interesting error as a result: Gradle complains about JUnit version on TestNG task

Answer 1

It was a stupid noob mistake indeed. We are using a multi-project setup with a master project and several folders on the same level, which contain the code and the tests. The build.gradle I was using accidentally only configured the master project to use TestNG. The solution is to include the test target within the allprojects or subprojects closure. Which is explained here: What is the difference between allprojects and subprojects

So here is the working build.gradle:

def mainClassName = 'nl.xillio.contentplatform.view.Run'


// Load settings for all projects including master and subprojects
allprojects {
    apply plugin: 'java'
    apply plugin: 'eclipse'

    version '0.1'

    repositories {
        mavenCentral()
        maven {
            url 'http://mvnrepository.com/maven2'
        }
        maven {
            url 'http://download.java.net/maven/2'
        }
    }
}

// Load the dependencies for all subprojects (layers)
subprojects {
    dependencies {
    //to do: move these to the correct subprojects
        compile 'javax.inject:javax.inject:1'
        compile 'org.springframework:spring-context:4.1.4.RELEASE'
        compile 'org.springframework:spring-core:4.1.4.RELEASE'
        compile 'org.springframework:spring-beans:4.1.4.RELEASE'
        compile 'commons-logging:commons-logging:1.2'
        testCompile 'org.springframework:spring-test:4.1.4.RELEASE'
        testCompile 'org.testng:testng:6.1.1'
    }

    test {
        // enable TestNG support (default is JUnit)
        useTestNG() 
    }
}

// Resolve the dependencies between the layers in the individual project's build.gradle files
// add ONLY specific per project behaviour of the GLOBAL build here:

project(':contentplatform-web') {
}

project(':contentplatform-service') {
}

project(':contentplatform-dao') {
}

// Return a list of all the external libraries
def getLibraries() {
    return configurations.runtime.filter{!it.name.startsWith('contentplatform')}
}

// Copy all the libraries to a libs folder
task copyLibraries(type: Copy) {
    group 'Content Platform'
    description 'Copy all the external libraries to the /libs folder.'

    destinationDir file('./build/')

    into('libs/') {
        from getLibraries()
    }
}

// Perform the build task before building the big jar
jar {
    group 'Content Platform'
    description 'Package all the layers and dependencies into a big jar.'

    // The libraries are required to build
    dependsOn(copyLibraries)

    // The final big jar needs all the layers
    dependencies {
        compile project(':contentplatform-web'), project(':contentplatform-dao'),
            project(':contentplatform-service'), project(':contentplatform-model')
    }
    // Create a MANIFEST.MF file, the main class file is in the web layer
    manifest {
        attributes 'Implementation-Title': 'Content Platform',  
            'Implementation-Version': version,
            'Built-By': System.getProperty('user.name'),
            'Built-Date': new Date(),
            'Built-JDK': System.getProperty('java.version'),
            'Class-Path': getLibraries().collect{'../libs/' + it.getName()}.join(' '),
            'Main-Class': mainClassName
    }

    // Include the layers in the fat jar
    from(configurations.compile.filter{it.name.startsWith('contentplatform')}.collect{it.isDirectory() ? it : zipTree(it) }) {
        exclude "META-INF/*.SF"
        exclude "META-INF/*.DSA"
        exclude "META-INF/*.RSA"
    }

    // Save the fat jar in the root of the folder instead of in build/libs
    destinationDir file('.')
}