How to perform ETL process performace testing

1
votes

I have been asked to perform performance testing of ETL process. I use Jmeter for performace testing of web applications, but this is the first time I have been asked to do it for ETL process. After researching I think what I want to achieve are the below steps but do not know how to achieve them.

The steps are

  • Verify that data loads and queries are executed within anticipated time frames
  • Verify that maximum anticipated volume of data is loaded within an acceptable time frame
  • Verify load times with various amounts of data to predict scalability

Note: Our Company is not using any tool to for testing ETL process and they are doing it using java, so for performance testing also, I will not be provided with any paid tool, I will have to use open source technologies. My skill sets includes java, Jmeter.

2
javaperformancejmeterperformance-testingetl

2 Answers

0
votes

I think there is no any specific tool for ETL testing. Still you can refer logs files generated by underlying ETL tool which would have CPU usage / Execution/Process time of data etc.

0
votes

I believe there is no particular tool to do the performance tests on the ETL process.

But still we can use the ETL tool to monitor the load process by generating the logs for each process, in my case we use Informatica Powercenter monitor. we calculate the parameters such as load time, throughput manually through the logs retrieved