Jmeter ForEach Controller failing to write variables to file in order retrieved

Question

Jmeter ForEach Controller failing to write variables in original order correctly

I am executing a http request retrieving a json payload with an array of employees. For each record (employee) I need to parse the record for specific fields e.g. firstName, lastName, PersonId and write to a single csv file, incrementing a new row per record.

Unfortunately, the file created has two issues. The PersonId never gets written and secondly the sequence of the values is not consistent with the returned original values. Sometimes I get the same record for lastName with the wrong firstName and vice versa. Not sure if the two issues are related, I suspect my regular expression extract is wrong for a number.

Jmeter setup. (5.2.1)

jmeter setUp

Thread group
+ HTTP Request
++ JSON JMESPath Extractor

+ ForEach Controller
++ Regular Expression Extractor: PersonId
++ Regular Expression Extractor: firstName
++ Regular Expression Extractor: lastName
++ BeanShell PostProcessor

getWorker returns the following payload

jsonPayload

JSON JMESPath Extractor to handle the payload.

{
  "items" : [
    {
      "PersonId" : 398378,
      "firstName" : "Sam",
      "lastName" : "Shed"
    },
    {
      "PersonId" : 398379,
      "firstName" : "Bob",
      "lastName" : "House"
    }
  ],
  "count" : 2,
  "hasMore" : true,
  "limit" : 2,
  "offset" : 0,
  "links" : [
    {
      "rel" : "self",
      "href" : "https://a.site.on.the.internet.com/employees",
      "name" : "employees",
      "kind" : "collection"
    }
  ]
}

JSON JMESPath Extractor Configuration

Name of created variables: items
JMESPath expressions: items
Match No. -1
Default Values: Not Found

ForEach Controller

ForEach Controller Configuration

Input variable prefix: items
Start Index: Empty
End Index: Empty
Output variable name: items
Add "_"? Checked

Each of the Regular Expression Extracts follow the same pattern as below.

Extract PersonId with Regular Expression

Apply to: Main Sample Only
Field to check: Body
Name of created variable: PersonId
Regular Expression: "PersonId":"(.+?)"
Template: $1$
Match No. Empty
Default Value: PersonId

The final step in the thread is where I write out the parsed results.

BeanShell PostProcessor

PersonNumber = vars.get("PersonNumber");
DisplayName = vars.get("DisplayName");

f = new FileOutputStream("/Applications/apache-jmeter-5.2.1/bin/scripts/getWorker/responses/myText.csv", true);
p = new PrintStream(f);
this.interpreter.setOut(p);

print(PersonId+", "+ PersonNumber+ ", " + DisplayName);
f.close();

I am new to this and looking either for someone to tell me where I screwed up or direct me to a place I can read up on the appropriate topics. (Both are fine). Thank you.

M Navneet Krishna M Navneet Krishna · Accepted Answer · 2020-06-07T05:46:29

For Each Controller doesn't know the structure of items variable since it is in JSON format. It is capable of just understanding an array and traverses through them. I would suggest to move away from For Each Controller in your case and use the JSON extractor itself for all the values like below

Person ID

First Name

Last Name

Beanshell Sampler Code

import java.io.FileWriter;   // Import the FileWriter class

int matchNr = Integer.parseInt(vars.get("personId_C_matchNr"));
log.info("Match number is "+matchNr);


f = new FileOutputStream("myText.csv", true);
p = new PrintStream(f);

for (int i=1; i<=matchNr; i++){
    PersonId = vars.get("personId_C_"+i);
    FirstName = vars.get("firstName_C_"+i);
    LastName = vars.get("lastName_C_"+i);   
    log.info("Iteration is "+i);
    log.info("Person ID is "+PersonId);
    log.info("First Name is "+FirstName);
    log.info("Last Name is "+LastName); 
    p.println(PersonId+", "+FirstName+", "+LastName);
}

p.close();
f.close();

Output File

HOW THE ABOVE ACTUALLY WORKS

When you extract values using the matchNr, it goes in a sequential order in which the response has arrived. For example, in your case, Sam & Shed appear as first occurrences and Bob & House appear as subsequent occurrences. Hence JMeter captures them with the corresponding match and stores them as 1st First Name = Sam, 2nd First Name = Bob and so on.

GENERIC STUFF

The regex expression for capturing Person ID which you have used seems to be inaccurate. The appropriate one would be

"PersonId" :(.+?),

and not

"PersonId":"(.+?)"

Move to JSR223 processors instead of Beanshell as they are more performant. Source: Which one is efficient : Java Request, JSR223 or BeanShell Sampler for my script. The migration is pretty simple. Just copy the code that you have in Beanshell and paste it in JSR223.
Close any stream or writer that is open appropriately else it might cause issues when other users are trying to write to the file during load test
In case you are planning to use this file as a subsequent input within JMeter, please note that there is a space between comma and the next element. For example, it is "Sam, Shed" and not "Sam,Shed".JMeter by default does not trim any spaces and will use the value just like that. Hence you might want to take a judicious call regarding that space

Hope this helps!

Jmeter ForEach Controller failing to write variables to file in order retrieved

2 Answers