Stop JMeter test execution only after n assertion errors

3
votes

Problem

I am modelling stress tests in JMeter 2.13. My idea of it is to stop the test after certain response time cap is reached, which I test with Duration Assertion node.

I do not want, however, to stop the test execution after first such fail - it could be a single event in otherwise stable situation. I would like the execution to fail after n such assertion errors, so I can be relatively sure the system got stressed and the average response should be around what I defined as a cap, which is where I want to stop the whole thing.

What I tried

I am using Stepping Thread Group from JMeter plugins. There I could use a checkbox to stop the test after an error, but it does that on first occasion. I found no other node in documentation that could model it, so I'm guessing there's a workaround I'm not seeing right now.

2
jmeterjmeter-plugins
May be this can help! stackoverflow.com/questions/15288200/… and stackoverflow.com/questions/12580368/… - Anil

2 Answers

1
votes

Close but not exactly what you're asking for: The Auto-Stop Jmeter plugin. See the documentation here. You can configure it to stop your test if there are n% failures in a certain amount of time.

If you want a specific number of errors, you can use a test-action sampler, combined with an if-controller - if (errorCount = n) test-action stop test

enter image description here

4
votes

I would recommend switching to Beanshell Assertion as it is more flexible and allows you to put some custom code in there.

For instance you have 3 User Defined Variables:

  • threshold - maximum sampler execution time. Any value exceeding the threshold will be counted
  • maxErrors - maximum amount of errors, test will be stopped if reached and/or exceeded
  • failures - variable holding assertion failure count. Should be zero in the beginning.

User Defined Variables

Example Assertion code:

long elapsed = SampleResult.getTime();

long threshold = Long.parseLong(vars.get("threshold"));

if (elapsed > threshold) {

    int failureCount = Integer.parseInt(vars.get("failures"));
    failureCount++;

    int maxErrors = Integer.parseInt(vars.get("maxErrors"));

    if (failureCount >= maxErrors) {
        SampleResult.setSuccessful(false);
        SampleResult.setResponseMessage(failureCount + " requests failed to finish in " + threshold + " ms");
        SampleResult.setStopTest(true);
    } else {

        vars.put("failures", String.valueOf(failureCount));
    }
}

Example assertion work:

Beanshell Assertion work

See How to use BeanShell: JMeter's favorite built-in component guide to learn more about extending your JMeter tests with scripting.