4
votes

As described in this post, we can use ProtoInclude attribute to manage class hierarchy serialization. And if we use just protobuf-net, it works quite fine both directions. But problem occurs when we try to deserialize messages, serialized by "external" legacy Protocol Buffers implementations, ex. Java, etc.

As mentioned in that post above, Protobuf-net recognizes class hierarchy by "reversed" bytes sequences, when child class fields serialized before parent's. But legacy code serialize them in "proper" order and protobuf-net throws an "Unable to cast object of type 'A' to type 'B'." exception during deserialization. In opposite direction it works fine, legacy code can deserialize "hierarchic" messages produced by protobuf-net library.

I can not affect byte serialization order on opposite side of the pipe. How can I correctly deserialize this type of messages on .NET protobuf-net side?

Update: code examples

On our end of the line we have original protobuf-net hierarchic classes:

[ProtoContract, ProtoInclude(10, typeof(B))]
public class A
{
    [ProtoMember(1)]
    public int Age;
}

public class B : A
{
    [ProtoMember(2)]
    public int Balls;
}

On the other end of the line classes are generated using .proto file:

message B {
    optional int32 balls = 2;   
}

message A {
    optional int32 age = 1; 
    optional B b = 10;
}

Generated classes example, we could use protobuf-net generator to create them for .NET:

[ProtoContract]
public class A_generated
{
    [ProtoMember(1)]
    public int Age;

    [ProtoMember(10)]
    public B b;
}

[ProtoContract]
public class B_generated 
{
    [ProtoMember(2)]
    public int Balls;
}

So now, let's serialize and deserialize class B:

  • Serialize and deserialize back original classes - OK
  • Serialize and deserialize back generated classes - OK
  • Serialize original and deserialize as generated - OK
  • Serialize generated and deserialize as original - FAIL, "Unable to cast object of type 'A' to type 'B'." exception

I've investigated resulting bytes and discovered a difference - bytes order.

Example: let Age=10 and Balls=23. Then:

  • original B serialized: [82, 2, 16, 23, 8, 10], could be deserialized using both as original as generated classes;
  • generated B serialized: [8, 10, 82, 2, 16, 23], can NOT be deserialized using protobuf-net original classes above.

I hope now it's clear enough, and wish to get positive answer: yes, there is a way to use ProtoInclude and deserialize generic classes.

1
Can you be more specific about the scenario here? I was under the impression that regular protobuf did not include inheritance. As such it was frequently shimmed via encapsulation. Additionally, you could just use the existing .proto to generate the model...? Happy to help, but I want to ensure I am answering correctly.Marc Gravell♦
Ah, right. I understand the context now, thank you. I understand what is happening, and why. I will investigate later today to see if there is an easy fix.Marc Gravell♦
very minor, but should A_generated.b be typed as B_generated ? this doesn't fix things - I just like to be precise in my test...Marc Gravell♦
I've spent a few hours looking at this. I agree it is desirable to support (the encoding spec is fairly clear that implementations should handle fields in any order), but the implementation is... not obvious. It is going to take quite a bit of effort to make this work.Marc Gravell♦
Thanks, waiting for your answer with great hope to the fix. :)Anton Krupnov

1 Answers

2
votes

Edit: this should be supported in v2 from r616 onwards.


To quote from the protobuf / java tutorial:

Don't go looking for facilities similar to class inheritance, though – protocol buffers don't do that.

So: whatever you have used locally to spoof inheritance locally, I would advise: use that here too. You could run your existing .proto through protogen for example.

If you can be very specific about the layout on both sides (a sample .proto for example), I might be able to advise further.