Performance / Stress Testing Java EE applications

Question

It's difficult to find all bottlenecks, deadlocks, and memory leaks in a Java application using unit tests alone.

I'd like to add some level of stress testing for my application. I want to test the limits of the application and determine how it reacts under high load.

I'd like to gauge the following:

• Availablity under high load
• Performance under high load
• Memory / CPU / Disk Usage under high load
• Does it crash under high load or react gracefully

It would also be interesting to measure and contrast such characteristics under normal load.

Are their well known, standard techniques to address stress testing. I am looking for help / direction in setting up such an environment. Ideally, I would like to run these tests regularly, so that wecan determine if recent deliveries impact performance.

Answer 1

I am a big fan of JMeter. You can set up calls directly against the server just as users would access it. You can control the number of user (concurrent threads) and accesses. It can follow a workflow, scraping pertinent information page to page. It takes 1 to 2 days to learn it well enough to be productive. (You can do the basics within an hour of downloading!)

As for seeing how all that affects the server, that is a tougher question. I have used professional tools from CA and IBM. (I am drawing a blank on specific tool names - maybe due to PTSD!) I have used out-of-the-box JVM profilers. I have used native linux and windows tools. If you are not too concerned about profiling what parts of your application causes issues, then you can just use the native tools for your OS to monitor CPU/Memory/IO.

Answer 2

One of our standard techniques is running stepped-ramp load tests to measure scalability.

Answer 3

There are mainly two approaches for performance on an application:

Performance test and System Test

How do they differ? Well it's easy, it's based on their scope, Performance tests' scope is limited and are highly unrealistic. Example: Test the IncomingMessage handler on some App X, for this you would setup a test which sends meesages to this handler on a X,Y,Z basis. This approach will help you pin down problems and measure performance of individual and limited zones on your application.

So this should now take you to the question, so am I to benchmark and performance test each one of the components in my app individually? Yes if you believe the component's behavior is critical and changes on newer versions are likely to induce performance penalties. But, if you want to get a feel on your application as a whole, the bunch of components interacting with each other and see how performance comes out, then you need a system test.

A system test will always, try to replicate as close as possible any customer production environment. Here you can observe what a real world feel of your app's performance is like and act accordingly to correct it.

So as conclusion,setup a system test on your app and measure what you were saying you wanted to measure. Then Stress the system as a whole and see how it reacts, you will be surprised on the outcome.

Finally, Performance test individually any critical components you have identified or would like to keep tracked on your app.

As a general guideline, when doing performance you should always: 1.- Get a baseline for the system on an idle state. 2.- Get a baseline for the system under normal expected load. 3.- Get a baseline for the system under stress conditions.

Keep in mind that Normal load results should be extrapolated to stress conditions, and a nice system will always be that one which scales linearly.

Hope this helps.

P.S. Tests, envirnoment setup and even data collection should be as fully automated as possible, this will help you run this on a basis and spend time diagnosing performance problems and not setting up the test.

Answer 4

As mentioned by others; tools like JMeter (Commercial tools like LoadRunner and more) can help you generate concurrent test load. Many monitoring tools (some provided within JDK like MissionControl, some other open source/ free tools like java Melody and many commercial one's) can help you do generic monitoring of various system (memory, CPU, network bandwidth) and JVM resources (Heap, CPU, GC overheads etc).
But to really identify bottlenecks within your code as well as other dependencies of your applications (like external services invoked, DB queries/updates etc) in a very quick and easy way; I recommend considering a good APM i.e. Application Performance Monitoring Tools like AppDynamics/ DynaTrace and more. They can help you pinpoint bottlenecks for specific request level, highlight slower parts of apps, generate percentile metrics at individual service end point or component / method level etc. They can be immensely useful , if one is dealing with very high concurrent users and stringent response time NFR's. They help uncover many bottlenecks across the layers of your application. Many even configure these tools in production (expected to cause 2-3% overheads; but worth it per me for the benefits they provide) - as production logging is not at debug level by default; so once some errors or slowness is observed; it's often extremely difficult to reproduce in lower environments or debug in absence of debug level logs from specific past duration.

Answer 5

There's no one tool to tackle this as far as I know. So build you own environment

• Load Injecting & Scripting: JMeter, SOAP UI, LoadUI
• Scheduling Tests & Automation: Jenkins, Rundeck
• Analytics on transaction data, resources, application performance logs: AppDynamics, ElasticSearch, Splunk
• Profiling: AppDynamics, YouKit, Java Mission Control, VisualVm