JMeter Maven Plugin does not provide test results when using remote host configuration

1
votes

When starting a JMeter test with the jmeter-maven-plugin with the following configuration...

<project>
...
    <build>
        <plugins>
            <plugin>
                <groupId>com.lazerycode.jmeter</groupId>
                <artifactId>jmeter-maven-plugin</artifactId>
                <executions>
                    <execution>
                        <id>jmeter-tests</id>
                        <phase>integration-test</phase>
                        <goals>
                            <goal>jmeter</goal>
                        </goals>
                    </execution>
                </executions>                       
                <configuration>
                    <customPropertiesFile>${basedir}/src/test/jmeter/jmeter.properties</customPropertiesFile>
                    <testFilesDirectory>${basedir}/src/test/jmeter</testFilesDirectory>
                    <remoteConfig>
                        <startServersBeforeTests>true</startServersBeforeTests>
                        <serverList>server01,server02</serverList>
                        <stopServersAfterTests>true</stopServersAfterTests>
                    </remoteConfig>
                    <testResultsTimestamp>false</testResultsTimestamp>
                </configuration>
            </plugin>
        </plugins>
    </build>            
...
</project>

...then the tests are executed properly on the given remote servers "server01" and "server02". The Maven task does not wait regarding the remote host's process but states that everything is fine:

[INFO] -------------------------------------------------------
[INFO]  P E R F O R M A N C E    T E S T S
[INFO] -------------------------------------------------------
[INFO]  
[INFO]  
[INFO] Proxy server is not being used.
[debug] JMeter is called with the following command line arguments: -n -t /home/me/demo/src/test/jmeter/test.jmx -l /home/me/demo/target/jmeter/results/20130925-test.jtl -d /home/me/demo/target/jmeter -q /home/me/demo/src/test/jmeter/jmeter.properties -r -R server01,server02 -X
[info] Executing test: test.jmx
[info] Completed Test: test.jmx
[INFO]  
[INFO] Test Results:
[INFO]  
[INFO] Tests Run: 1, Failures: 0
[INFO]  
[INFO] 
[INFO] --- maven-failsafe-plugin:2.15:verify (verify) 
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 9.598s
[INFO] Finished at: Wed Sep 25 16:43:00 CEST 2013
[INFO] Final Memory: 21M/349M
[INFO] ------------------------------------------------------------------------

But there aren't any test results collected. If you check the appropriate JMeter result's file then you'll realize an empty one:

cat /home/me/demo/target/jmeter/results/20130925-test.jtl

#empty

So the Maven process is finished before the remote servers have finalized the test execution. Am I wrong to expect that the jmeter-maven-plugin should collect the test results of the remote hosts instead of writing a 0 byte file (20130925-test.jtl)?

If I start the same test without using remote JMeter agents then everything works fine. The jmeter-maven-plugin executes the test and writes the result in the according "jtl" file. So what could be wrong with the above configuration regarding the "remoteConfig" element?

UPDATE 1: I used the jmeter-maven-plugin version 1.8.1 and Maven 3.0.5

UPDATE 2: The example above executed just 1 JMeter test. If there are more than 1 JMeter test in the directory ${basedir}/src/test/jmeter then the Maven build fails. In this case the log file of the first JMeter test has no errors but all other test logs (e.g. ./target/jmeter/logs/test2.log) state the following message:

Error in NonGUIDriver java.lang.IllegalStateException: Engine is busy - please try later

So the 1st test was executed on the remote agent (also without collecting any results in the according "jtl" file on the client site) but the other tests have been denied by the remote hosts. So it seems that the maven-jmeter-plugin tries to parallelize the execution of the tests but IMHO they should be executed in a serialized way.

2
mavenjmeterjmeter-maven-plugin
I had the same problem. I even tried the 1.8.2-SNAPSHOT version of the Maven plugin, different error. The test run fine from cmd line. Then in a panic I tried Ant with its JMeter plugin, worked like a charm. Stayed with panic solution - for now. – PålOliver
I haven't found any satisfying solution for me. So at the end I wrote a small Shell script (Bash) which is called by the Continuous Integration service (Jenkins) and does the same. So I was also able to make the best of it and enhanced the test execution to be able to assert the throughput (requests/seconds) of a test plan :-) – Achim Tromm
It's a known bug (github.com/Ronnie76er/jmeter-maven-plugin/issues/64) that should be fixed in the current snapshot (1.9.0-SNAPSHOT). It would be useful to know if you are still having problems. – Ardesco

2 Answers

0
votes

using the snapshot version of the plugin and specifying the plugin repo, fixed the issue for me.

I know it's not ideal to have snapshot versioin in poms, but if you're using jmeter in a test project and not prod code it should not be too much of a big deal.

my 2 cents

<pluginRepositories>
    <pluginRepository>
        <id>sonatype-snapshots</id>
        <url>https://oss.sonatype.org/content/repositories/snapshots/</url>
    </pluginRepository>
</pluginRepositories>

<build>
         <plugins>
             <plugin>
                 <groupId>com.lazerycode.jmeter</groupId>
                 <artifactId>jmeter-maven-plugin</artifactId>
                 <version>1.8.2-SNAPSHOT</version>
                 <executions>
                     <execution>
                         <id>jmeter-tests</id>
                         <phase>verify</phase>
                         <goals>
                             <goal>jmeter</goal>
                         </goals>
                     </execution>
                 </executions>
                     <configuration>
                         <remoteConfig>
                             <startServersBeforeTests>true</startServersBeforeTests>
                             <!--<stopServersAfterTests>true</stopServersAfterTests>-->
                             <serverList>192.168.1.27</serverList>
                         </remoteConfig>
                     </configuration>
             </plugin>
         </plugins>
 </build>
0
votes

You dint specify where the output has to go, so all the results are getting dropped

change config as specified below adding 'java.rmi.server.hostname'.

<remoteConfig>                  
    <startServersBeforeTests>true</startServersBeforeTests>                 
        <serverList>server01,server02</serverList>
    <stopServersAfterTests>false</stopServersAfterTests>                                        
</remoteConfig>
<propertiesSystem>
<java.rmi.server.hostname>**Master.invoking.machine.com**</java.rmi.server.hostname>
</propertiesSystem>

If the build fails with any error or error in your Test plan.jmx files , it is expected that you wont get any results in JTL or any output format. example : suppose you have 2 testplans , first executed properly , second failed because of some error in testplan [like some dependency jars for your plan is missing] . If this case happens , since you are using same jtl file for both plans , your jtl file will be imbalanced and it will be null.

I would recommend

1.Create 2 sample test plans and place them in ${basedir}/src/test/jmeter and execute them , check if you still see build failures. If it is successfull check the results in jtl file.

  1. Always restart the slave/remote jmeter process before even starting the tests.