0
votes

When running a JMeter script defined as this (3 assertions), regardless there was an error or not in either the first or the second assertion, the only assertion that will count to reflect in the JTL log file is the last assertion. So, s="true" or s="false" will only be set if either Assertion Duration fails, not when Status Code or Content fails.

  • HTTP Request
    • Assertion Status Code
    • Assertion Content
    • Assertion Duration


<httpSample s="true"> <assertionResult> <name>Assertion Status Code</name> <failure>true</failure> <error>false</error> </assertionResult> <assertionResult> <name>Duration Assertion</name> <failure>true</failure> <error>false</error> </assertionResult>

Am I missing anything? Is there a way to set the status code when either or one of the assertion fails?

2

2 Answers

1
votes

All assertions are independent.

As per JMeter Manual

Assertions are used to perform additional checks on samplers, and are processed after every sampler in the same scope. To ensure that an Assertion is applied only to a particular sampler, add it as a child of the sampler.

It looks like that your sampler is:

  • returning response code which doesn't match assertion's expectations
  • fails to finish for expected time

I've simulated your situation using a Beanshell Sampler with following code:

Thread.sleep(2000);
ResponseCode="200";
ResponseMessage="test";
return "test";

Which returns 200 Response Code, test Response Message and lasts for 2 seconds (and a little bit more).

And I applied 3 assertions to it:

  1. Check for Response Code to be "200";
  2. Check for Response Message to be "test";
  3. Check for sampler duration not more than "1000 ms"

and got next response:

<?xml version="1.0" encoding="UTF-8"?>
<testResults version="1.2">
<sample t="2033" lt="0" ts="1391084548919" s="false" lb="BeanShell Sampler" rc="200"     rm="test" tn="Thread Group 1-1" dt="text" by="4">
  <assertionResult>
    <name>Duration Assertion</name>
    <failure>true</failure>
    <error>false</error>
    <failureMessage>The operation lasted too long: It took 2,033 milliseconds, but should not have lasted longer than 1,000 milliseconds.</failureMessage>
  </assertionResult>
  <assertionResult>
    <name>Code Assertion</name>
    <failure>false</failure>
    <error>false</error>
  </assertionResult>
  <assertionResult>
    <name>Text Assertion</name>
    <failure>false</failure>
    <error>false</error>
  </assertionResult>
</sample>

As you can see, "duration" assertion failed, other 2 are fine. Is there any chance that you using sub-samples, applying assertion to samplers tree or to Transaction Controller? If so - please re-consider assertion scope and make sure that you place assertions as children of samplers, you want to check.

0
votes

Thanks Dmitri for the elaborated answer. I finally found the root cause. The assertions from above were incorrectly set, by having both response assertions "Ignore Status" checkbox checked. The flags seems counterintuitive to me. However, this does the trick. The following change made the trick:

  • HTTP Request - Wrong

    • Assertion Status Code (Ignore Status checked)
    • Assertion Content (Ignore Status checked)
    • Assertion Duration
  • HTTP Request - Right

    • Assertion Status Code (Ignore Status checked)
    • Assertion Content (Ignore Status UNchecked)
    • Assertion Duration

Also see this one. JMeter how to NOT fail 500 Internal Server Errors