Jmeter protobuf testing. Could not read Protobuf message

Question

I am testing one project via protobuf protocol and using HTTP Request Sampler. The target appserver is also written on Java. There is a problem with errors in reponses:

"Could not read Protobuf message: Protocol message contained an invalid tag (zero).; nested exception is com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero)"

The case is that it is happening not in 100% requests. When i used HttpClient4 it was about 30-40% of failed requests. After i changed it to HttpClient3.1 error rate decreased down to ~10% which is also not a good deal.

To send a protobuf message I am using variable ${data} in Bodydata tab of HttpSampler. And in BeanShell preProcessor i do the next:

(import and non-necessary stuff were ommited)
MapViewport mv = MapRequest.MapViewport.newBuilder().setMaxX(mc.getX()+15).setMaxY(mc.getY()+15).setMinX(mc.getX()-15).setMinY(mc.getY()-15).build();

byte[] data = mv.toByteArray();
vars.put("data", new String(data));

Also I tryed to to use different encoding like a new String(data,"UTF-8") anso on.

If to look on Request tab if View Result Tree I can say that all failed messages contain "?" symbol:

Seems like some weird symbols should not be sent, but ~10% of requests after saving a byte array to String contain them.

So you're converting a binary stream to String to save as a var - which can be fraught with issues depending on the encoding. It is also system dependent and behavior can change. Moreover when you're ${data}, you're sending a string, which for some binary data is inserting ? The first workaround that comes to mind is to write the binary stream to file and use the file path as variable in bodydata. But that would cause a lot of File I/O for high volume. Hmm.. this is a bit of a thinker. — RaGe
Alternatively you can assemble your own HTTP request and send it from a beanshell sampler instead of using a HTTP sampler. Wonder if the HTTP Raw Request from jmeter-plugins would be useful. — RaGe

RaGe RaGe · Accepted Answer · 2015-11-18T15:19:52

I'm fairly convinced your issue is that you're losing some non-printable characters while converting from binary stream to String and then back. I'm thinking of two possible workarounds:

Write the binary data to a file instead of saving to string, and then use the filename as variable in the HTTP sampler, in the body from file section
Use a beanshell sampler, and construct your own HTTPClient object and POST request, with the binary data in body and fire it yourself instead of using the HTTP sampler

I don't like the first option because of all the additional file I/O. I do not like the second option because measuring response time will now include all the request assembly you're doing in beanshell - so you'll have to pick the one that bothers you less I guess.

Let me know if you want me to write up some code examples for either case.

Edit: For beanshell HTTP call using HttpClient 4:

import org.apache.http.HttpEntity;
import org.apache.http.HttpHeaders;
import org.apache.http.HttpResponse;
import org.apache.http.client.ClientProtocolException;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ByteArrayEntity;
import org.apache.http.impl.client.DefaultHttpClient;

byte[] data = null;
//...assign protobuf binary buffer to data...

HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost("http://127.0.0.1");
HttpEntity entity = new ByteArrayEntity(data);
post.setEntity(entity);
post.setHeader(HttpHeaders.CONTENT_TYPE, "application/octet-stream");
HttpResponse response=null;
try {
    response = client.execute(post);
} catch (ClientProtocolException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
} catch (IOException e) {
    // TODO Auto-generated catch block
    e.printStackTrace();
}

ResponseCode = response.getStatusLine().getStatusCode().toString();
//if some assert is true then
Issuccess = true;
ResponseMessage="Some Response Message";

This is untested against a protobuf end point, let me know how it works out for you.

Jmeter protobuf testing. Could not read Protobuf message

2 Answers