In jmeter using http requests i'm posting some json bundles and from the responses i'm using jsr223 post processor to extract data and store it inside csv files, each entry in each line. now for 10 post requests i'm getting duplicate data into the csv file. Is there a way to read back csv files and remove duplicate lines using jmeter. The number of lines in csv files can be almost 200,000.
eg:csv file be like csvFile1.csv: line1 line2 duplicateline ...........so on