Protobuf-net dynamic types in a load-balanced system

Question

I'm using protobuf-net in an application that does a lot of binary serialization of objects from (for all intents and purposes) 3rd party dlls. As a result, I can't use the [Proto-] attributes on the contracts themselves, and I'm instead using the RuntimeTypeModel to prepare the serializer at runtime as it encounters new types. Example serializer wrapper:

public static class ProtobufSerializer
{
    public static byte[] Serialize<T>(T obj)
    {
         PrepareSerializer(typeof(T));
         ProtoBuf.Serialize(memoryStream, obj); 
    }

    public static T Deserialize<T>(byte[] bytes)
    {
         PrepareSerializer(typeof(T));
         ProtoBuf.Serialize(memoryStream, obj); 
    }
}

Where we can safely assume that PrepareSerializer is capable of preparing RuntimeTypeModel to serialize any given type. I'm having some issues with the deserialization of objects where I have to leverage DynamicType=true though. For instance, given the following interface:

public interface IFoo
{
    string Name {get;}
}

And the implementation:

public class Foo : IFoo
{
    public string Name {get;set;}
    public Bar Bar {get;set;}

    [OnDeserializing]
    public void OnDeserializing(Type t)
    {
        PrepareSerializer(typeof(Foo));
    }
}

public class Bar
{
    public int Baz {get;set;}
}

The PrepareSerializer method essentially would use a surrogate and generate a model roughly equivalent to:

// registered surrogate for IFoo
[ProtoContract]
public class IFooSurrogate
{
    [ProtoMember(1, DynamicType=true)]
    public object Value

    [OnSerializing]
    public void OnSerializing(Type t)
    {
        PrepareSerializer(this.Value.GetType());
    }
}

Where Value is set by the implicit converters to equal the instance of IFoo. This works fine during serialize (where the event is fired and gives me a chance to prepare the serializer for the specific interface implementation type). It would also work fine in a non-distributed system where I would have to run through a serialize method before ever trying to deserialize that type. During deserialization in a distributed system though, where the current node has never seen Foo before, the ProtoBuf.Serializer throws an InvalidOperationException complaining about a lack of serializer for type Bar before it runs the Foo.OnDeserializing event (giving me a chance to tell it how to deserialize Bar).

Is there any way to attach a hook to ensure my code is given a chance to know about 'Foo' before protobuf-net complains about a lack of serializers?

Marc Gravell Marc Gravell · Accepted Answer · 2018-08-09T21:54:41

I haven't tried exactly this situation, but: in order to allow some flexibility, all Type storage and rehydration goes via the TypeModel.DynamicTypeFormatting event; so, you could in theory hook this event on RuntimeTypeModel.Default, with something like:

RuntimeTypeModel.DynamicTypeFormatting += (sender, args) => {
    if (args.FormattedName != null) { // meaning: rehydrating
        lock(SomeSyncLock) {
            if(NotYetKnown(args.FormattedName))
                Prepare(args.FormattedName);
        }
    }
};

The intent of this API is to allow you to control how types are resolved, but... I guess it would work for this too?

I can, however, get behind the idea of an event that is more deliberately targeted at the first time a new Type is seen, essentially replacing / supplementing the "apply default behaviour" code. I don't think it exists today, though.

Protobuf-net dynamic types in a load-balanced system

1 Answers