Jmeter vs Gatling for performance testing 2020

Question

I am trying to compare Gatling to Jmeter for performance testing.

My requirements is:

1. I want compatibility with CI/CD and automation.
2. Performance should be good (I know Gatling is a bit better performance in regard resource hogging).
3. Dynamic -> i.e. able to run a test, change some settings in the environment, and then run the test again.
4. Use on Cloud (I feel like because Gatling is better for performance, it would be better for the cloud... But I don't know if there really is a big difference here).
5. I am testing a Monolith system, so I wont be able to split up the system as well (as microservices), and using mocks(mockito) will be harder to be able to get more exact performance on for example a request for an endpoint, excluding database time, etc. So this kind of means the performance test I suppose I will be doing is more of a stress test and testing general performance of system, rather then getting measurements of isolated parts of system.

Any help would be great. I ask because a lot of information seems to be out of date on this...

Edit: I amn't afraid of Gatling being programmed in Scala and gatling dsl. Im more afraid of limitations and gotchas that i amnt seeing!

Answer 1

Disclaimer: Gatling creator here

IMHO, first criterion should actually be: which tool would you feel the most comfortable with?

Gatling is code oriented, integrates nicely with IDEs such as IntelliJ or vscode. JMeter is GUI oriented.

If you know about programming, you might prefer Gatling so you can use your usual dev environment and practices (refactoring, peer review, etc). You don't need to be a dev expert, you'll be mostly using Gatling's DSL (make sure to read the official documentation and go through the tutorials).

On the contrary, if you can't code (and don't want to learn the basics such as methods, parameters, IDE), you shouldn't go with Gatling and go with JMeter instead.

Note: if you're looking for a cloud based platform for Gatling, we provide Gatling FrontLine.

Answer 2

Both tools support all your requirements so it's rather a matter of personal preference/experience.

1. Gatling supports less protocols out of the box (limited to HTTP, JMS and MQTT) than JMeter, however it is possible to extend both and add support of the protocol(s) of your choice
2. You will need to do some extra manual work in order to set up Gatling distributed testing while JMeter aggregates data from the slaves automatically in Distributed Mode
3. Gatling is more CI friendly as it assumes writing tests in Scala while supported way of creating a JMeter test is using GUI and JMeter test plan is XML file and adding a single test element will result in 10+ lines of XML code which is not very readable.