Protobuf-net sirialization/deserialization c# vs Linux c++

1
votes

I'm passing messages between Windows C# client and Linux C++ server via TCP sockets. C# code uses protobuf-net v2, Linux Google's version of protobuf. Small test object that I pass has 6 fields ( Enum, Int, String). I need help with two problems:

  1. C# portion unable to deserialize data sent from Linux, unless the Memory stream used as the storage for the data is initialized with the binary array in the constructor. Array can not be larger than the data sent from Linux ( 9 bytes in my case ). Code sample - byte[] data = new byte[9], copy data from the socket into the array. MemoryStream myStream = new MemoryStream(data), pass myStream to Serializer.Deserialize... If I initialize MemoryStream without bynary buffer or with array of 1024 bytes, Deserialize will create empty object, without processing data.

  2. When I try to serialize the same object with the same values as Linux same in C# the size of the data is 11 bytes vs 9 on Linux. I checked the byte array in the debugger, C# version has the same 9 fields as Linux data in the indexes 2-11 of the array. Index 0 is 8 and index 1 is 9. I can try to get around the problem, by modifying Linux deserialization code, just need to know, if I always have to deal with two extra fields at the beginning of the message. Also, I can add two extra fields to messages generated on Linux if it going to fix my deserialization in C#, just need to know how to generate values for these two fields.

Thanks. Alex.

1
c#c++linuxserializationprotobuf-net
Hi @Marc. Thanks for your prompt respond. I was able to identify the source of the difference between C++ and C# data. C++ didn't populate enum field with default value. I'm more concern with the deserialization in C#. I was planning to reuse the same stream with preallocated buffer, but it doesn't work. Now I'll have to allocate new byte array to match the size of the data in the message and pass it to the MemoryStream construction. I expect high rate of messages, the solution I described will lead to the frequent garbage collections. Can I reuse ProtoReader to avoid extra allocations? – AlexF

1 Answers

0
votes
  1. Protobuf data is not self-terminating, simply. You can, however, create either a MemoryStream or ProtoReader that takes a larger payload, but is limited to a virtual length. If you are sending multiple messages, you will need to know the length of the payload - that is inescapable. This is often achieved via a length prefix. I would expect this to throw random errors - most likely "invalid field header 0" - and I wonder if you are swallowing that exception
  2. impossible to comment without the specific example; most likely something to do with the default value of field number 1 (header 8 === "field 1 varint")