protobuf-net: Cannot serialize type data, How can I define type data with protocol buffers?

Question

I am trying to create a simple in-memory grid that can be serialized using protocol buffers.

The idea is the user can create/define columns of any type (primitives or user defined as long as they're protocol buffers tagged).

My problem is you can't serialize Type data with protocol buffers so how can i achieve this?

Code is below showing how i'd hoped to code the Columns of the grid.

Thanks a lot.

[TestClass]
public class UnitTest1
{
    [TestMethod]
    public void TestMethod1()
    {
        Column colOrderId = new Column("OrderId", typeof(uint));
        Column colOrderDesc = new Column("OrderDesc", typeof(string));
        Column colPrice = new Column("Price", typeof(Price));
        Column colOrderDateTime = new Column("OrderDateTime", typeof(DateTime));

        var s = colOrderId.ToBArray();
    }
}

[ProtoContract, Serializable]
public sealed class Column
{
    public Column(string name, Type type)
    {
        Name = name;
        Type = type;
    }

    [ProtoMember(1)]
    public string Name { get; private set; }

    [ProtoMember(2)]
    public Type Type { get; private set; }
}

[ProtoContract, Serializable]
public sealed class Price
{
    public Price(double value, string currency)
    {
        Value = value;
        Currency = currency;
    }

    [ProtoMember(1)]
    public double Value { get; private set; }

    [ProtoMember(2)]
    public string Currency { get; private set; }
}

public static class ProtoBufEx
{
    public static byte[] ToBArray<T>(this T o)
    {
        using (MemoryStream ms = new MemoryStream())
        {
            ProtoBuf.Serializer.Serialize(ms, o);
            return ms.ToArray();
        }
    }
}

I guess that in protobuf v2 you are able to add a userdefined serializer for Type that serializes to the name of that type. — CodesInChaos
Another potential problem is that your types are immutable. I'm not sure if protobuf supports that out of the box, or if you need another custom serializer. — CodesInChaos
@CodeInChaos just for completeness, the private sets are fine. In v1 it would also need a parameterless constructor (again, private is fine); in v2 it can optionally skip the constructor, WCF-style. — Marc Gravell
The behavior I expect for such a type is that it will pass the values of the serialized fields into the constructor. No private reflection or bypassing of constructors. — CodesInChaos

Marc Gravell Marc Gravell · Accepted Answer · 2011-06-27T20:32:34

There's a couple of issues in this approach; yes, in standard protocol buffers any kind of Type metadata is a bit naughty, as it isn't really interoperable, but I play a little loose with that in v2 on an opt-in basis - allowing you to send object etc as long as internally it still knows what you mean.

This, however, will get expensive for cell-by-cell work, even with the optimisations built in (it only sends the same type meta once, for example).

IMO, though, a far better option is to restrict yourself to a list of known types that are all known ahead of time. If you can do that there are some interesting tricks with inheritance that are pretty efficient and reasonably suited for this scenario. But essentially you'd have something like:

[ProtoContract]
[ProtoInclude(4, typeof(Column<int>))] // etc
abstract class Column {
    [ProtoMember(1)]
    public string Name {get;private set;}

    public abstract object Value {get;private set;}
}
[ProtoContract]
class Column<T> : Column {
    [ProtoMember(1)]
    public T TypedValue { get;private set;}
    override Value {...shim to TypedValue...}
}
// etc

(pseudo-code, incomplete)

I'd happily walk you through a lot of that, but you might also want to look at this blog entry to see if using DataTable (much as I dislike it for data-access) might save some effort here.

protobuf-net: Cannot serialize type data, How can I define type data with protocol buffers?

1 Answers