JMeter environment specific configuration

Question

I have several JMeter test plans which should be executed in different environments, say Dev, Test, UAT, Live. In each test plan I would like to have a simple way to specify which environment to use. Each environment has a lot of configuration such as hostname, port, ssl-cert, user name, password, account numbers and other test data.

One thing I'm trying to achieve is the ease of switching environments while using JMeter GUI or running scenarios from build scripts.

One of my ideas is to use the "Include Controller" to include another jmx file which has list of User Defined Variables and other config elements. However, JMeter does not support variables in the included file name, so I cannot parametrise the scenario by an environment name. Include Controller supports JMeter parameter "includecontroller.prefix", but it is not very flexible, e.g. I cannot change it from JMeter GUI, I should change JMeter config files and restart it.

I've tried to use Switch Controller, but no luck, it doesn't switch configuration elements, only samplers.

What is the best practice to externalise environment specific configuration from test scenarios and share it between several scenarios?

Answer 1

I would suggest to substitute all environment-specific variables or values with JMeter Properties. See following functions for reference:

• __property()
• __P()

For example you can define a property called hostname in either jmeter.properties file or as JMeter command line argument as follows

jmeter -Jhostname=169.140.130.120 -n -t yourscript.jmx -l yourscriptresults.jtl

and refer to in inside your script as:

• ${__P(hostname,)} or
• ${__property(hostname,,)}

See Apache JMeter Properties Customization Guide for more details.

Answer 2

Like the Manish Sapariya mentioned, Parametrized Controller is quite useful to prepare configuration for more than one environment. I used it in the previous place I worked and started the configuration now in new place. It is a bit of work at the beginning, but later it is easy in maintenance. There is a bit of tutorial in the link that is provided above, but it won't take in consideration that you want to run just one env at a time. I will describe it a bit below, maybe it will be useful. So, slowly step by step:

1. First of all - you need two thread groups - one for environment profiles (no 1 on the first screenshot - Env Profiler) and one for your test cases, included test plans etc. (no 2 - API Requests). The latter has to be disabled as it is container that should not be executed straight from here (right click -> disable or Ctrl+T)
2. Then you need your User Defined Variables elements (no 3) - I'm using three of them:
   • first for defining which environment will be executed (environmentType var) and logins/passwords, tokens etc.
   • second with IDs for items needed for tests
   • third with IPs, ports, path prefixes and so on.

The most important thing here is that I have them separated at this moment by prefixes in variable names, so in one UDV element I have variables like dev.serverIP, dev.serverPort, preprod.serverIP and so on (second screenshot) populated with values relevant for that environment. Additionally in one of this UDVs I have environmentType variable (mentioned earlier) with default value 'dev' (which you can change manually here or provide different value when launching through command line/CI or whatever)

3. Now in the Env Profiler I have If Controllers(no 4 on the first screenshot). For dev env I have (no 5 on the first screen):

   "${environmentType}" == "dev"

For each env (if controller) you have to provide proper condition like this above.

4. Each IfController contains that "jp@gc - Parametrized Controller" mentioned before (that you can download as part of Extras Set here by the way). In each Param Controller you assign to variables that you use in test plans variables specific for that environment, e.g. name: serverIP, value: ${dev.serverIP} for dev env (third screenshot)

5. And now the last thing - tests and plans you want to execute.
   • In that disabled Thread Group (API Requests) you add Simple Controllers that contains your tests or Include Controllers that import some tests from other files.
   • When you have those tests, for each one that you want to run in that particular environment you have to add Module cotroller inside Parametrized Controller with path to that test (screenshot below)

And that it is pretty much it. Now maintaining:

• to add new variable you have to add it in UDV with prefixes (dev.newVar, preprod.newVar) and fill the relevant values, then add proper entry in Parametrized controllers (those newVar = ${dev.newVar}) and that's it
• to add new test from other Test Plan - add Include controller with path to that file and add Module controller in each Paramterized controller directing them to that Include Controller
• to add new environment just copy the one you already have, change its If contr., Parametrized controller and fill values in UDVs The nice thing here is that if you want, lets say, dev env with all tests and the other with just some somke tests you can prepare a copy, change If controller to take some other value of env variable (like dev for all tests, devsmoke for smoke tests) and add or delete some of the module controllers in that new profile. Of course you can build it up a bit and you can ues different threads for different parts of the system for easier maintanance, just do not forget to disable those threads working as containers.

I know it's a lot to do when you start, but it is not so bad later, when just adding some stuff - probably the easiest way to do it anyway.

Answer 3

As the current solution I'm using JSR223 sampler with custom JavaScript code to set up variables from external properties-files. Something like that:

var file = new java.io.File(args[0]);
var props = new java.util.Properties();
var is = new java.io.FileInputStream(file);
props.load(is);
is.close();
for(var it = props.entrySet().iterator(); it.hasNext();)
{
    var entry = it.next();
    vars.put(entry.getKey(), entry.getValue());
}

Now I just need to add this code as the very first sampler in a test plan and pass environment specific filepath as the sampler parameter args[0] it will load variables from the file and put them as JMeter variables.

Answer 4

I have not used myself, but this jmeter-pluing may help you. Here is snippet from documentation

Parameterized Controller since 0.1.0

When your JMeter test plan tree becomes like a sequoia or a banyan, 
you start feeling yourself like a monkey in a jungle, jumping from 
branch to branch, trying to support this important test consistent. 
You really need some way to have parameterized subroutines, to reuse 
parts of test plan like regular programming language functions and 
procedures.

JMeter have out-of-box Module controller, but it has no parameters 
to pass to, so if you need to call repeating sequence of the same 
action with different parameters, your reflection in a mirror starts 
morphing into monkey. Parameterized Controller helps you stay human 
and sane.

Answer 5

If your are looking to make your life easy in the GUI only, an easy way is to duplicate your User Defined Variables for each environment and disable all others except the environment you are executing your tests against, something like the following:

Answer 6

I have the same problem. My current approach is to have several user defined variable (UDV) elements, e.g., DevVariables, TestVariables, etc. Each of these have the same variables defined (hostname, port, etc.). Then I manually disable those UDV elements that aren't in use.

Edit:

The JMeter User Manual states - "The User Defined Variables element lets you define an initial set of variables, just as in the Test Plan. Note that all the UDV elements in a test plan - no matter where they are - are processed at the start." I'm not sure what event constitutes the "start", but it makes it sound like you can't conditionally include UDVs, at least not via controllers.

You may be able to have a single UDV per test plan, where you set the various variables (host, etc.). Maybe you can set each variable's value using an ugly JavaScript function that keys off of a passed-in property value.

You might try asking the question on the Apache JMeter User mailing list. There must be a way to do what you want.

Answer 7

I use a JSR223 sampler with Groovy to register new parameters. Make sure to put the JSR223 sampler in a Only Once Controller, so it is ran only once (per thread).

I have in the test plan all variables registered for all environments like this:

TEST.user = 123
TEST.url = test.mysite.com
LOAD.user = 435
LOAD.url = load.mysite.com

Then I specify which environment I want to select with a command line parameter at startup like: -Denvironment=TEST

My JSR223 script will add new variables to the test, without the TEST. prefix. In my scripts I just reference ${user}

This is the script to use:

JMeterVariables jmv = new JMeterVariables();

for (Map.Entry item : vars.entrySet()) {
    if (item.getKey().startsWith(env)) {
        String keyname = item.getKey().substring(env.length()+1);
        String origval = vars.get(keyname);
        if (origval==null || (origval.equals("") && !item.getValue().equals(""))) {
            jmv.put(keyname, item.getValue());
        }
    }
}
if (!jmv.entrySet().isEmpty()) {
    vars.putAll(jmv);
}

Answer 8

I realize this question is old, but if you are using the GUI and running tests manually, one very simple approach is to have a User Defined Variables element per environment, and just have one of these enabled when running the test. But I still don't understand why jmeter does not support setting UDV:s conditionally.