How to read from file in chunks to upload in Jmeter?

Question

I am writing a Jmeter Test Plan to upload files to server in chunks. I've little knowledge of Java.

I've used while controller on HTTP Request Sampler with Bean Shell Pre-Processor. I have written a short script to get bytes from file, now the problem I am facing is: The HTTP Request Sampler takes file path in File Upload Section. Is there any way to create a file in memory in Bean Shell Pre Processor and then use that memory file variable in File Path field.

What I think is theoretically, It is possible. Because whenever we upload a file, we first take it into memory and then send to server. So, can we just create a file in memory from bytes (chunks of 1 MB) and then send it as file upload. Here is the code I've written in Bean Shell Pre-Processor

Integer maxChunkSize    = new Integer(vars.get("FILE_MAX_SIZE"));
String uploadFilePath   = vars.get("UPLOAD_FILE");
uploadFileSize      = new File(uploadFilePath).length();
InputStream uploadFile  = new BufferedInputStream(new FileInputStream(uploadFilePath));
int offset      = whileCounter * maxChunkSize;
ByteArrayOutputStream bos = new ByteArrayOutputStream();
byte[] bytes;
int size;

/* creating bytes array to hold file data*/
if (offset < uploadFileSize) {
    if (((int) offset + (int) maxChunkSize) < uploadFileSize) {
        bytes = new byte[ (int) maxChunkSize];
        size = maxChunkSize;
    } else {
        size = (int) (uploadFileSize - offset);
        bytes = new byte[ (int) size];
        vars.put("WHILE_LOOP", "0");
    }
}

/* printing results for debugging */
/*
log.info(" ============================================================== ");
log.info("While counter " + whileCounter.toString() );
log.info("While loop " + vars.get("WHILE_LOOP").toString() );
log.info("The file to upload is : " + uploadFilePath);
log.info("Maximum Chunk size is : " + maxChunkSize.toString());
log.info("Current Offset is : " + offset.toString());
log.info("The file size is " + uploadFileSize.toString());
log.info(" ============================================================== ");
*/


/* here it is giving method invocation on loop counter 2, so skip method is used */
uploadFile.skip(offset);
int bytesRead = uploadFile.read(bytes, 0, size);


/* write to byte output stream to read as a file */
bos.write(bytes, 0, bytesRead); 


/* params for next iteration */
uploadFile.close();
whileCounter++;
vars.put("WHILE_COUNTER", whileCounter.toString() );

Expected: An alternative to upload file in chunks via JMeter OR Create a memory variable that will act as file for file upload path in JMeter HTTP Request Sampler -> Files Upload Section.

Dmitri T Dmitri T · Accepted Answer · 2019-01-31T05:33:40

I don't think you can achieve this using HTTP Request sampler, the options I would consider are in:

Implement chunked file upload using Java code without relying onto JMeter's HTTP Request sampler, something like

import org.apache.http.client.methods.HttpPost
import org.apache.http.entity.InputStreamEntity
import org.apache.http.impl.client.CloseableHttpClient
import org.apache.http.impl.client.HttpClients

CloseableHttpClient client = HttpClients.createDefault();

HttpPost post = new HttpPost("http://example.com");
File file = new File("/path/to/your/file");

InputStreamEntity reqEntity = new InputStreamEntity(new FileInputStream(file), -1, "Your file mime type");

//Alternatively:
//FileEntity reqEntity = new FileEntity(file, "Your file mime type");

reqEntity.setChunked(true);

client.execute(post)

Go for HTTP Raw Request sampler which simply sends chunked data over the wire (the size of the chunk is controllable via kg.apc.jmeter.samplers.FileReadChunkSize property) so you can just record the associated request body data using a sniffer tool like Wireshark and later replay it with the desired chunk size. Check out The JMeter HTTP Raw Request Sampler - When and How to Use It article for more information if needed.
Also be aware that since JMeter 3.1 it's recommended to use JSR223 Test Elements and Groovy language for scripting so consider migrating to Groovy on next available opportunity.

How to read from file in chunks to upload in Jmeter?

1 Answers