Jmeter Run Thread Groups Consecutively vs No of Threads

Question

What is the difference between following Jmeter test architectures?

--Test Plan(Run Thread Groups Consecutively not checked)
    --Thread Group 1(No of Threads(users) : 1, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 2(No of Threads(users) : 1, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 3(No of Threads(users) : 1, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 4(No of Threads(users) : 1, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 5(No of Threads(users) : 1, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 6(No of Threads(users) : 1, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 7(No of Threads(users) : 1, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 8(No of Threads(users) : 1, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 9(No of Threads(users) : 1, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 10(No of Threads(users) : 1, Ramp up Period(in Seconds) : 1, Loop count: 1)

--Test Plan(Run Thread Groups Consecutively not checked)
    --Thread Group 1(No of Threads(users) : 10, Ramp up Period(in Seconds) : 1, Loop count: 1)

--Test Plan(Run Thread Groups Consecutively not checked)
    --Thread Group 1(No of Threads(users) : 2, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 2(No of Threads(users) : 2, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 3(No of Threads(users) : 2, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 4(No of Threads(users) : 2, Ramp up Period(in Seconds) : 1, Loop count: 1)
    --Thread Group 5(No of Threads(users) : 2, Ramp up Period(in Seconds) : 1, Loop count: 1)

In my scenario, I want to do load test of my solution where I have different test cases executed by different users. Here users have a very little chance to do same operation at a time. So I use different thread groups for my test cases. And with my scenario increase No of Threads(users) in thread group doesn't mean logical to me. So I unchecked Run Thread Groups Consecutively in my Test plan but not sure about does it really makes a concurrency test or not.

Answer 1

If you check Run Thread Groups Consecutively then thread groups will fire up consecutively.That means JMeter will start Thread Group 01 first, then Thread Group 02 ,......, and so on. It's an option here to instruct JMeter to run the Thread Groups serially rather than in parallel.

So if you unchecked Run Thread Groups Consecutively in your Test plan, it will make a concurrency in your test.

Answer 2

In case 1 you are guaranteed to have a concurrency of 10 threads right from the start of execution.

In case 2 the concurrency of 10 threads is reached after 1 second, with rate of 1 thread per 100ms (provided no threads finished their execution within that time).

Case 3 mixes both options: 5 threads will start immediately, and 5 more after 1 sec, so the concurrency of 10 threads is reached after 1 second.

Note that when I say "concurrency reached" I mean on JMeter side. Depending on the setup of the script, concurrency for the server may be reached later.

What's also important is that you have "Loop count: 1". If all users run that single loop iteration for a long time (minutes/hours), or there's a loop controller inside the thread group, the difference between 3 options is insignificant for statistics. But if that iteration is short (seconds to a few minutes), or each operation within each thread is unique, then in case of options 2 and 3 (especially 2), your statistics will be flawed, since many operations on threads that started earlier will be performed before 10 thread concurrency is reached and vice versa.

Answer 3

JMeter acts as follows:

• For each Thread Group it starts threads during "ramp-up" period
• Threads start executing samplers upside-down or according to Logic Controllers
• When thread doesn't have a sampler to execute or loop to iterate - it's being shut down

So you can run into a situation where 1st thread has already finished everything and was shut down and 10th thread hasn't been started yet.

If you want to have guaranteed concurrency - provide enough loops at Thread Group level. You can also use i.e. Ultimate Thread Group which has some extra features to flexibly define users arrival rate and visualise anticipated concurrency.

By the way, you can get different virtual users executing different scenarios even in the same Thread Group using Throughput Controller. See Running JMeter Samplers with Defined Percentage Probability article for detailed explanation of the possible approaches.