Decode protobuf binary -- getting "Failed to parse input"

4
votes

I tried to decode a protobuf binary message using protoc and I am getting "Failed to parse input" error. I tried both protoc --decode_raw < proto.bin and protoc --decode package.MessageType MySchema.proto < proto.bin.

I read two threads in SO, one of which suggests that protoc --decode is not 100% reliable and another one which suggests that message length data at the beginning needs to be stripped manually. My question is

  • How to use protoc tool for decoding proto binaries?
  • Is it true that the tool is not 100% reliable to be capable of decoding (even if I supply type) or messages need some hacking before they can be deserialized? (which begs the question, what is the purpose of protoc --decode option?

P.S: I am using protoc 3.6.1 and messages are created in a Java program (syntax=proto2)

1
protocol-buffers

1 Answers

2
votes

It would have helped if you had linked the threads you talk about.

But in general, yes, you need to strip the length prefix before decoding with protoc. There is no standard about how to prefix length to protobuf messages, some use fixed32 prefix and others varint, some have a longer header. Thus the protoc tool cannot parse the length header.

And I haven't seen any case where protoc --decode wouldn't decode a valid message. But it has very little tolerance for errors, such as truncated data - it always rejects the whole message even if only one field is corrupted.

For better debugging, you can use e.g. this tool, which decodes byte by byte and shows the first error that occurs: https://protogen.marcgravell.com/decode