Currently I am trying to use Jmeter for functional tests, but I am currently stuck on how to best implement flexible test validation.
The ultimate goal would be to have a csv file for data input and validation, so it is easy for other people to add or remove test cases.
Case:
- Login
- Execute bulk job (involving a variable nr of objects and creating a variable nr of objects)
- Validate the result via datase sql statements. (multiple SQL requests and response assertions)
- Logout
Some statements I would like to execute:
- Nr of invoiced BI (per contract ID)
- Nr of distinct invoice ID's for BI with invoice ID (per contract ID)
*Get list of Invoice ID's to use in following SQL statement:
- For list of invoice ID's invoice header must be equal to....
- For list of invoice ID's invoice lines must be equal to....
For each sql I would use a response assertion to validate the sql result. Response assertion with for example 10 lines with a variable inside each line and OR statement to match/equal at least one of the lines. Data is generated, so ID's can be different each run, I am only sure that the data for an object should match one of x cases.
Everything is very dynamic and the nr of check varies for each test case that is being executed from the csv file. So I imagine I would need a foreach controller for each sql statement/check/assertion
CSV file would look something like this:
- Bulk Job / Contracts / Contract Validation / Invoice header validation /
- 1234 / 12345 / 2 / 456
- 1234 / 12435 / 5 / 968
- 4256 / 89754 / 1 / 987465
- 4256 / 78597 / 4 / 654
- 4256 / 87596 / 2 / 852
Or like this:
- Bulk Job / Contracts / Contract Validation / Invoice header validation /
- 1234 , 12345:12435 , 2:5 , 456:968
- 4256 , 89754:78597:87596 , 1:4:2 , 987465:654:852
What would be the best way to store and setup something like this? CSV file or xml or ...? Maybe use multiple csv files and try to keep everything aligned?
