Having trouble getting the selenium command line to run HTML tests and record output, then exit w/ status that reflects test result

Question

I am having trouble running selenium on my Mac so that it runs a simple suite with one simple test (see sections 'TEST CASE' and 'TEST SUITE', below) using the following command line:

java -jar lib/selenium-server.jar -timeout 10 -log server.log -browserSideLog browser.log -debug -htmlSuite '*firefox' http://google.com `pwd`/suite3 `pwd`/selenium.html

My problem is that i don't get a non-zero return code as expected when tests fail, and also, i don't see any reports.

I have tried the following variants:

Using the given suite and testcase files either:

specify -timeout option (ends drastically with no reports being written), or

do not specify the -timeout option. without the time out option the selenium server never shuts down... even though the help documentation says:

Run a single HTML Selenese (Selenium Core) suite and then exit
immediately, using the specified browser (e.g. "*firefox") on the
specified URL (e.g. "http://www.google.com").

To get around the hang i added an extra selenium command to the test case to force shut down of the server, as follows:

....  previous commands ...
          <tr>
            <td>open</td>
            <td>http://localhost:4444/selenium-server/driver/?cmd=shutDownSeleniumServer</td>
            <td></td>
          </tr>
        </tbody>
     </table>
  </body>
</html>

This did eliminate the hang. The browser shut down and the process terminated.. but i got no test results, and my exit code was (incorrectly) zero... not non-zero as i would have expected.

TEST SUITE (this needs to be saved with the name 'suite3' )

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head profile="http://selenium-ide.openqa.org/profiles/test-case">
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <link rel="selenium.base" href="http://google.com" />
    <title>testcase3</title>
  </head>
  <body>
    <table cellpadding="1" cellspacing="1" border="1">
      <thead>
        <tr><td rowspan="1" colspan="3">testcase3</td></tr>
      </thead>
      <tbody>
        <tr>
          <td>open</td>
          <td>/</td>
          <td></td>
        </tr>
        <tr>
          <td>verifyTextPresent</td>
          <td>xbout Google</td>
          <td></td>
        </tr>
      </tbody>
    </table>
  </body>
</html>

TEST CASE

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
  <head profile="http://selenium-ide.openqa.org/profiles/test-case">
    <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <link rel="selenium.base" href="http://google.com" />
    <title>testcase3</title>
  </head>
  <body>
    <table cellpadding="1" cellspacing="1" border="1">
      <thead>
        <tr><td rowspan="1" colspan="3">testcase3</td></tr>
      </thead>
      <tbody>
        <tr>
          <td>open</td>
          <td>/</td>
          <td></td>
        </tr>
        <tr>
          <td>verifyTextPresent</td>
          <td>xbout Google</td>
          <td></td>
        </tr>
      </tbody>
    </table>
  </body>
</html>

Thanks in advance to whoever can give me a clue to solve this ! - chris

Chris Bedford Chris Bedford · Accepted Answer · 2012-02-25T00:16:22

Our colleague in Beijing, Chenyun Xiao, figured out the trick here. If you rename both the suite and testcase file so they have .html extensions (and u make sure that the ref from suite to testcase is updated to reflect the new name for the test case file) then everything will work as expected.. Thanks, Chenyun – Chris Bedford 1 min ago

Having trouble getting the selenium command line to run HTML tests and record output, then exit w/ status that reflects test result

3 Answers