ant junit task - running all tests from mutiple test folders, irrespective of failure

Question

My build script has multiple test folders configured, each containing hundreds of test cases. junit task is executed on each of these folders.'haltonfailure' and 'haltonerror' is set to false so that execution continues in case of failure.

However, once the execution of all test cases from that folder is done, the build stops (due to some tests failing from that folder) and does not continue execution of tests from remaining folders. So what I am looking for is that irrespective of which folder the test failure occurs, test cases from remaining folders should also get executed.

How can I do that (assuming I will have to retain multiple test folder structure)?

Here is the relevant ant script I have:

<target name="test">

	<junit_tests />
    
    <antcall target="generatetxtreport"/>
    <antcall target="generatexmlreport"/>
    <fail message="folder1 test(s) failed" if="junit.folder1.failed"/>
    <fail message="folder2 test(s) failed" if="junit.folder2.failed"/>
    <fail message="folder3 test(s) failed" if="junit.folder3.failed"/>
    <fail message="folder4 test(s) failed" if="junit.folder4.failed"/>

    <fail message="folder1 test(s) error" if="junit.folder1.error"/>
    <fail message="folder2 test(s) error" if="junit.folder2.error"/>
    <fail message="folder3 test(s) error" if="junit.folder3.error"/>
    <fail message="folder4 test(s) error" if="junit.folder4.error"/>
</target>


<macrodef name="junit_tests">
<sequential>
	<junit_test testfolder="folder1"/>
	<junit_test testfolder="folder2"/>
	<junit_test testfolder="folder3"/>
	<junit_test testfolder="folder4"/>
</sequential>
</macrodef>


<macrodef name="junit_test">
<attribute name="testfolder"/>
<attribute name="testMode" default="once"/>
<sequential>
  <junit printsummary="true"
		 haltonfailure="false"
		 haltonerror="false"
		 fork="on"
		 forkmode="@{testMode}"
		 failureproperty="junit.@{testfolder}.failed"
		 errorproperty="junit.@{testfolder}.error"
		 showoutput="false"
		 outputtoformatters="true">
	<formatter type="brief" usefile="false"/>
	<formatter type="xml"/>
	<classpath>
	  <path refid="some.classpath"/>
	</classpath>
	<batchtest todir="some.dir">
	  <fileset dir="some.root\@{testfolder}\">
		<include name="**/Test*.class"/>
	  </fileset>
	</batchtest>
  </junit>
</sequential>
</macrodef>

Answer 1

I think all your tests are running, but the build is exiting at your first explicit fail, i.e.

<fail message="folder1 test(s) failed" if="junit.folder1.failed"/>

Ant does not continue past a fail, so you cannot use those statements to print all the results.

Here is an example of one way in which you could achieve what I think you want, using ant:if (Ant 1.9.3+) and changing your fail blocks into conditional echo blocks.

<project name="test" default="test" xmlns:if="ant:if" xmlns:unless="ant:unless">

    <target name="test">

        <junit_tests />

        <condition property="junit.overall.failed">
            <or>
                <istrue value="${junit.folder1.failed}"/>
                <istrue value="${junit.folder2.failed}"/>
            </or>
        </condition>

        <condition property="junit.overall.error">
            <or>
                <istrue value="${junit.folder1.error}"/>
                <istrue value="${junit.folder2.error}"/>
            </or>
        </condition>

        <echo message="folder folder1 test(s) failed" if:true="${junit.folder1.failed}"/>
        <echo message="folder folder2 test(s) failed" if:true="${junit.folder2.failed}"/>
        <echo message="folder folder1 test(s) error" if:true="${junit.folder1.error}"/>
        <echo message="folder folder2 test(s) error" if:true="${junit.folder2.error}"/>

        <fail message="some tests had errors" if="junit.overall.error"/>
        <fail message="some tests failed" if="junit.overall.failed"/>

    </target>