JUnit tests pass in Eclipse but fail in Maven Surefire

112
votes

I have written some JUnit tests using JUnit 4 and spring-test libraries. When I run the tests inside Eclipse then run fine and pass. But when I run them using Maven (during the build process), they fail giving a spring related error. I am not sure what is causing the problem, JUnit, Surefire or Spring. Here is my test code, spring configuration and the exception that I get from Maven:

PersonServiceTest.java

package com.xyz.person.test;

import static com.xyz.person.util.FjUtil.toFjList;
import static junit.framework.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;

import java.util.List;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.AbstractTransactionalJUnit4SpringContextTests;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import org.springframework.test.context.transaction.TransactionConfiguration;
import org.springframework.transaction.annotation.Transactional;

import com.xyz.person.bo.Person;
import com.xyz.person.bs.PersonService;

import fj.Effect;

@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration(locations = { "classpath*:personservice-test.xml" })
@TransactionConfiguration(transactionManager = "transactionManager", defaultRollback = false)
public class PersonServiceTest {

    @Autowired
    private PersonService service;

    @Test
    @Transactional
    public void testCreatePerson() {
        Person person = new Person();
        person.setName("abhinav");
        service.createPerson(person);

        assertNotNull(person.getId());
    }

    @Test
    @Transactional
    public void testFindPersons() {
        Person person = new Person();
        person.setName("abhinav");
        service.createPerson(person);

        List<Person> persons = service.findPersons("abhinav");
        toFjList(persons).foreach(new Effect<Person>() {
            public void e(final Person p) {
                assertEquals("abhinav", p.getName());
            }});
    }

}

personservice-test.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:tx="http://www.springframework.org/schema/tx"
    xmlns:aop="http://www.springframework.org/schema/aop"
    xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="http://www.springframework.org/schema/beans
      http://www.springframework.org/schema/beans/spring-beans.xsd
      http://www.springframework.org/schema/tx
      http://www.springframework.org/schema/tx/spring-tx.xsd
      http://www.springframework.org/schema/aop
      http://www.springframework.org/schema/aop/spring-aop-2.5.xsd
      http://www.springframework.org/schema/context
      http://www.springframework.org/schema/context/spring-context-2.5.xsd">

    <import resource="classpath:/personservice.xml" />

    <bean id="datasource"
        class="org.springframework.jdbc.datasource.DriverManagerDataSource"
        lazy-init="true">
        <property name="driverClassName" value="org.apache.derby.jdbc.EmbeddedDriver" />
        <property name="url" value="jdbc:derby:InMemoryDatabase;create=true" />
    </bean>

    <bean id="entityManagerFactory"
        class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
        <property name="dataSource" ref="datasource" />
        <property name="persistenceUnitName" value="PersonService" />
        <property name="jpaVendorAdapter">
            <bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
                <property name="databasePlatform" value="org.hibernate.dialect.DerbyDialect" />
                <property name="showSql" value="true" />
                <property name="generateDdl" value="true" />
            </bean>
        </property>
        <property name="jpaPropertyMap">
            <map>
                <entry key="hibernate.validator.autoregister_listeners" value="false" />
                <entry key="javax.persistence.transactionType" value="RESOURCE_LOCAL" />
            </map>
        </property>
    </bean>

    <bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
        <property name="entityManagerFactory" ref="entityManagerFactory" />
        <property name="dataSource" ref="datasource" />
    </bean>

    <tx:annotation-driven transaction-manager="transactionManager"
        proxy-target-class="false" />

    <bean id="beanMapper" class="org.dozer.DozerBeanMapper">
        <property name="mappingFiles">
            <list>
                <value>personservice-mappings.xml</value>
            </list>
        </property>
    </bean>

</beans>

Exception in Maven

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.xyz.person.test.PersonServiceTest
23:18:51,250  WARN JDBCExceptionReporter:77 - SQL Warning: 10000, SQLState: 01J01
23:18:51,281  WARN JDBCExceptionReporter:78 - Database 'InMemoryDatabase' not created, connection made to existing database instead.
23:18:52,937  WARN JDBCExceptionReporter:77 - SQL Warning: 10000, SQLState: 01J01
23:18:52,937  WARN JDBCExceptionReporter:78 - Database 'InMemoryDatabase' not created, connection made to existing database instead.
23:18:52,953  WARN TestContextManager:429 - Caught exception while allowing TestExecutionListener [org.springframework.test.context.transaction.TransactionalTestExecutionListener@359a359a] to process 'after' execution for test: method [public void com.xyz.person.test.PersonServiceTest.testCreatePerson()], instance [com.xyz.person.test.PersonServiceTest@1bc81bc8], exception [org.springframework.transaction.IllegalTransactionStateException: Pre-bound JDBC Connection found! JpaTransactionManager does not support running within DataSourceTransactionManager if told to manage the DataSource itself. It is recommended to use a single JpaTransactionManager for all transactions on a single DataSource, no matter whether JPA or JDBC access.]
java.lang.IllegalStateException: No value for key [org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean@3f563f56] bound to thread [main]
        at org.springframework.transaction.support.TransactionSynchronizationManager.unbindResource(TransactionSynchronizationManager.java:199)
        at org.springframework.orm.jpa.JpaTransactionManager.doCleanupAfterCompletion(JpaTransactionManager.java:489)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.cleanupAfterCompletion(AbstractPlatformTransactionManager.java:1011)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:804)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:723)
        at org.springframework.test.context.transaction.TransactionalTestExecutionListener$TransactionContext.endTransaction(TransactionalTestExecutionListener.java:515)
        at org.springframework.test.context.transaction.TransactionalTestExecutionListener.endTransaction(TransactionalTestExecutionListener.java:290)
        at org.springframework.test.context.transaction.TransactionalTestExecutionListener.afterTestMethod(TransactionalTestExecutionListener.java:183)
        at org.springframework.test.context.TestContextManager.afterTestMethod(TestContextManager.java:426)
        at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:90)
        at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
        at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:240)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
        at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
        at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
        at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:180)
        at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
        at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:115)
        at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:102)
        at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:599)
        at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
        at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
23:18:53,078  WARN TestContextManager:377 - Caught exception while allowing TestExecutionListener [org.springframework.test.context.transaction.TransactionalTestExecutionListener@359a359a] to process 'before' execution of test method [public void com.xyz.person.test.PersonServiceTest.testFindPersons()] for test instance [com.xyz.person.test.PersonServiceTest@79f279f2]
org.springframework.transaction.IllegalTransactionStateException: Pre-bound JDBC Connection found! JpaTransactionManager does not support running within DataSourceTransactionManager if told to manage the DataSource itself. It is recommended to use a single JpaTransactionManager for all transactions on a single DataSource, no matter whether JPA or JDBC access.
        at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:304)
        at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:371)
        at org.springframework.test.context.transaction.TransactionalTestExecutionListener$TransactionContext.startTransaction(TransactionalTestExecutionListener.java:507)
        at org.springframework.test.context.transaction.TransactionalTestExecutionListener.startNewTransaction(TransactionalTestExecutionListener.java:269)
        at org.springframework.test.context.transaction.TransactionalTestExecutionListener.beforeTestMethod(TransactionalTestExecutionListener.java:162)
        at org.springframework.test.context.TestContextManager.beforeTestMethod(TestContextManager.java:374)
        at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:73)
        at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:82)
        at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:72)
        at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.runChild(SpringJUnit4ClassRunner.java:240)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
        at org.springframework.test.context.junit4.statements.RunBeforeTestClassCallbacks.evaluate(RunBeforeTestClassCallbacks.java:61)
        at org.springframework.test.context.junit4.statements.RunAfterTestClassCallbacks.evaluate(RunAfterTestClassCallbacks.java:70)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
        at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:180)
        at org.apache.maven.surefire.junit4.JUnit4TestSet.execute(JUnit4TestSet.java:59)
        at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.executeTestSet(AbstractDirectoryTestSuite.java:115)
        at org.apache.maven.surefire.suite.AbstractDirectoryTestSuite.execute(AbstractDirectoryTestSuite.java:102)
        at org.apache.maven.surefire.Surefire.run(Surefire.java:180)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:599)
        at org.apache.maven.surefire.booter.SurefireBooter.runSuitesInProcess(SurefireBooter.java:350)
        at org.apache.maven.surefire.booter.SurefireBooter.main(SurefireBooter.java:1021)
Tests run: 3, Failures: 0, Errors: 3, Skipped: 0, Time elapsed: 15.625 sec <<< FAILURE!

Results :

Tests in error:
  testCreatePerson(com.xyz.person.test.PersonServiceTest)
  testCreatePerson(com.xyz.person.test.PersonServiceTest)
  testFindPersons(com.xyz.person.test.PersonServiceTest)

Tests run: 3, Failures: 0, Errors: 3, Skipped: 0
18
javaspringmaven-2surefirespring-test
do you have any special configuration of the surefire plugin in your POM? – matt b
@matt I don't have any configuration for surefire in my pom – Abhinav Sarkar
I came to this article because I had the same problem, but in my case I used another solution. After enabling DEBUG logs on my tests, I found out that Spring Framework was looking at an old MongoDB database name, and this name was set in an old version of a jar created by another project on my workspace (although it was built several times with the new name). Some Maven Clen + deleting the libraries on my .m2 followed by Maven Install of all those projects solved the problem. Although there was no reason for the project to look at an old jar (it was cached somewhere, unfortunately) – Cotta

18 Answers

122
votes

I had the same problem (JUnit tests failed in Maven Surefire but passed in Eclipse) and managed to solve it by setting forkMode to always in the maven surefire configuration in pom.xml:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.12</version>
    <configuration>
        <forkMode>always</forkMode>
    </configuration>
</plugin>

Surefire parameters: http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html

Edit (January 2014):

As Peter Perháč pointed out, the forkMode parameter is deprecated since Surefire 2.14. Beginning from Surefire 2.14 use this instead:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.16</version>
    <configuration>
        <reuseForks>false</reuseForks>
        <forkCount>1</forkCount>
    </configuration>
</plugin>

For more information see Fork Options and Parallel Test Execution

8
votes

I suddenly experienced this error, and the solution for me was to disable to run tests in parallel.

Your milage may vary, since I could lower number of failing tests by configuring surefire to run parallel tests by ´classes´.:

            <plugin>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.22.2</version>
                <configuration>
                    <parallel>classes</parallel>
                    <threadCount>10</threadCount>
                </configuration>
            </plugin>

As I wrote first, this was not enough for my test suite, so I completely disabled parallel by removing the <configuration> section.

7
votes

I had a similar problem, the annotation @Autowired in the test code did not work under using the Maven command line while it worked fine in Eclipse. I just update my JUnit version from 4.4 to 4.9 and the problem was solved.

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.13.1</version>
</dependency>
6
votes

Test execution result different from JUnit run and from maven install seems to be symptom for several problems.

Disabling thread reusing test execution did also get rid of the symptom in our case, but the impression that the code was not thread-safe was still strong.

In our case the difference was due to the presence of a bean that modified the test behaviour. Running just the JUnit test would result fine, but running the project install target would result in a failed test case. Since it was the test case under development, it was immediately suspicious.

It resulted that another test case was instantiating a bean through Spring that would survive until the execution of the new test case. The bean presence was modifying the behaviour of some classes and producing the failed result.

The solution in our case was getting rid of the bean, which was not needed in the first place (yet another prize from the copy+paste gun).

I suggest everybody with this symptom to investigate what the root cause is. Disabling thread reuse in test execution might only hide it.

5
votes

This doesn't exactly apply to your situation, but I had the same thing -- tests that would pass in Eclipse failed when the test goal from Maven was run.

It turned out to be a test earlier in my suite, in a different package. This took me a week to solve!

An earlier test was testing some Logback classes, and created a Logback context from a config file.

The later test was testing a subclass of Spring's SimpleRestTemplate, and somehow, the earlier Logback context was held, with DEBUG on. This caused extra calls to be made in RestTemplate to log HttpStatus, etc.

It's another thing to check if one ever gets into this situation. I fixed my problem by injecting some Mocks into my Logback test class, so that no real Logback contexts were created.

5
votes

I have the similar problem, but with IntelliJ IDEA + Maven + TestNG + spring-test. (spring-test is essential of course :) ) It was fixed when I've change config of maven-surefire-plugin to disable run tests in parallel. Like this:

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.9</version>
    <configuration>
        <skipTests>${maven.test.skip}</skipTests>
        <trimStackTrace>false</trimStackTrace>
        <!--<parallel>methods</parallel>-->
        <!-- to skip integration tests -->
        <excludes>
            <exclude>**/IT*Test.java</exclude>
            <exclude>**/integration/*Test.java</exclude>
        </excludes>
    </configuration>
    <executions>
        <execution>
            <id>integration-test</id>
            <phase>integration-test</phase>
            <goals>
                <goal>test</goal>
            </goals>
            <configuration>
                <skipTests>${maven.integration-test.skip}</skipTests>
                <!-- Make sure to include this part, since otherwise it is excluding Integration tests -->
                <excludes>
                    <exclude>none</exclude>
                </excludes>
                <includes>
                    <include>**/IT*Test.java</include>
                    <include>**/integration/*Test.java</include>
                </includes>
            </configuration>
        </execution>
    </executions>
</plugin>
3
votes

I had the same issue, but the problem for me was that Java assertions (e.g. assert(num > 0)) were not enabled for Eclipse, but were enabled when running maven.

Therefore running the jUnit tests from Eclipse did not catch trigger the assertion error.

This is made clear when using jUnit 4.11 (as opposed to the older version I was using) because it prints out the assertion error, e.g.

java.lang.AssertionError: null
    at com.company.sdk.components.schema.views.impl.InputViewHandler.<init>(InputViewHandler.java:26)
    at test.com.company.sdk.util.TestSchemaExtractor$MockInputViewHandler.<init>(TestSchemaExtractor.java:31)
    at test.com.company.sdk.util.TestSchemaExtractor.testCreateViewToFieldsMap(TestSchemaExtractor.java:48)
3
votes

I had a similar problem with a different cause and therefore different solution. In my case, I actually had an error where a singleton object was having a member variable modified in a non-threadsafe way. In this case, following the accepted answers and circumventing the parallel testing would only hide the error that was actually revealed by the test. My solution, of course, is to fix the design so that I don't have this bad behavior in my code.

2
votes

[I am not sure that this is an answer to the original question, since the stacktrace here looks slightly different, but it may be useful to others.]

You can get tests failing in Surefire when you are also running Cobertura (to get code coverage reports). This is because Cobertura requires proxies (to measure code use) and there is some kind of conflict between those and Spring proxies. This only occurs when Spring uses cglib2, which would be the case if, for example, you have proxy-target-class="true", or if you have an object that is being proxied that does not implement interfaces.

The normal fix to this is to add an interface. So, for example, DAOs should be interfaces, implemented by a DAOImpl class. If you autowire on the interface, everything will work fine (because cglib2 is no longer required; a simpler JDK proxy to the interface can be used instead and Cobertura works fine with this).

However, you cannot use interfaces with annotated controllers (you will get a runtime error when trying to use the controller in a servlet) - I do not have a solution for Cobertura + Spring tests that autowire controllers.

2
votes

I had a similar problem: JUnit tests failed in Maven Surefire but passed in Eclipse when I used JUnit library version 4.11.0 from SpringSource Bundle Repository. Particulary:

<dependency>
    <groupId>org.junit</groupId>
    <artifactId>com.springsource.org.junit</artifactId>
    <version>4.11.0</version>
</dependency>

Then I replaced it with following JUnit library version 4.11 and everything works fine.

<dependency>
    <groupId>junit</groupId>
    <artifactId>junit</artifactId>
    <version>4.11</version>
</dependency>
1
votes

I had this problem today testing a method that converted an object that contained a Map to a JSON string. I assume Eclipse and the Maven surefire plugin were using different JREs which had different implementations of HashMap ordering or something, which caused the tests run through Eclipse to pass and the tests run through surefire to fail (assertEquals failed). The easiest solution was to use an implementation of Map that had reliable ordering.

1
votes

In my case the reason was a bug in the code. The test relied on a certain order of elements in a HashSet, which turned out to be different when run either in Eclipse or in Maven Surefire.

1
votes

I had a similar problem, I ran my tests disabling the reuse of forks like this

mvn clean test -DreuseForks=false

and the problem disappeared. The downside is that the overall test execution time will be longer, that's why you may want to do this from the command line only if necessary

0
votes

You don't need to inject a DataSource in the JpaTransactionManager since the EntityManagerFactory already has a datasource. Try the following:

<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
   <property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
0
votes

Usually when tests pass in eclipse and fail with maven it is a classpath issue because it is the main difference between the two.

So you can check the classpath with maven -X test and check the classpath of eclipse via the menus or in the .classpath file in the root of your project.

Are you sure for example that personservice-test.xml is in the classpath ?

0
votes

This has helped me in troubleshooting my problem. I had a similar symptoms in that maven would fail however running junit tests runs fine.

As it turns out my parent pom.xml contains the following definition:

    <plugin>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>2.9</version>
      <configuration>
        <forkMode>pertest</forkMode>
        <argLine>-Xverify:none</argLine>
      </configuration>
    </plugin>

And in my project I override it to remove the argLine:

    <plugin>
       <artifactId>maven-surefire-plugin</artifactId>
       <configuration>
            <forkMode>pertest</forkMode>
            <argLine combine.self="override"></argLine>
          </configuration>
    </plugin>

Hopefully this will help someone in troubleshooting surefire plugin.

0
votes

I had the same problem, and the solution for me was to allow Maven to handle all dependencies, including to local jars. I used Maven for online dependencies, and configured build path manually for local dependencies. Thus, Maven was not aware of the dependencies I configured manually.

I used this solution to install the local jar dependencies into Maven:

How to add local jar files in maven project?

-2
votes

It is most likely that your configuration files are in src/main/resources, while they must be under src/test/resources to work properly under maven.

https://cwiki.apache.org/UIMA/differences-between-running-unit-tests-in-eclipse-and-in-maven.html

I'm replying this after two years 'cause I couldn't find this answer here and I think it is the right one.