Protobuf-net and WebApi, which format to use to return data over the wire?

Question

I have a .Net Core 3.1 WebAPI that:

Serialize my obj with: ProtoBuf.Serializer.Serialize(stream, myobject);
Return it with: Convert.ToBase64String(stream.GetBuffer(), 0, (int)stream.Length);

and a Blazor WASM app that call the WebAPI and

Get the data from the wire: MemoryStream stream = new MemoryStream(Convert.FromBase64String(await response.Content.ReadAsStringAsync()));
and then deserialize it back to my obj: myobj = ProtoBuf.Serializer.Deserialize(stream);

Everything works fine but my doubt is this can be optimized and avoid some conversion to byte[] / base4 string and back.

Is this the correct way to transfer protobuf data over the wire or can this be done in a better way?

Interesting question. I genuinely don't know the answer, but here's hoping... — Marc Gravell
You can send binary data (byte[]) over http - no base64 encoding, i never had trouble doing that. Less bytes over wire and no string conversion (base64) back and forth. — Frank Nielsen
@FrankNielsen could you share some code both API and Client side? — Pietro

Frank Nielsen Frank Nielsen · Accepted Answer · 2020-06-08T19:29:26

How to send raw bytes over the http wire:

var binaryContent = new ByteArrayContent(new byte[] { 1, 2, 3 });
binaryContent.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
content.Add(binaryContent, "content");

var result = await client.PostAsync("api/bytes/incoming", content);

The trick is that the receiver must be aware of "ContentType: application/octet-stream" which is not default for most web servers. So the server side requires a little more effort:

public class RawRequestBodyFormatter : InputFormatter
{
    public RawRequestBodyFormatter()
    {
        SupportedMediaTypes.Add(new MediaTypeHeaderValue("text/plain"));
        SupportedMediaTypes.Add(new MediaTypeHeaderValue("application/octet-stream"));
    }


    /// <summary>
    /// Allow text/plain, application/octet-stream and no content type to
    /// be processed
    /// </summary>
    /// <param name="context"></param>
    /// <returns></returns>
    public override Boolean CanRead(InputFormatterContext context)
    {
        if (context == null) throw new ArgumentNullException(nameof(context));

        var contentType = context.HttpContext.Request.ContentType;
        if (string.IsNullOrEmpty(contentType) || contentType == "text/plain" ||
            contentType == "application/octet-stream")
            return true;

        return false;
    }

    /// <summary>
    /// Handle text/plain or no content type for string results
    /// Handle application/octet-stream for byte[] results
    /// </summary>
    /// <param name="context"></param>
    /// <returns></returns>
    public override async Task<InputFormatterResult> ReadRequestBodyAsync(InputFormatterContext context)
    {
        var request = context.HttpContext.Request;
        var contentType = context.HttpContext.Request.ContentType;


        if (string.IsNullOrEmpty(contentType) || contentType == "text/plain")
        {
            using (var reader = new StreamReader(request.Body))
            {
                var content = await reader.ReadToEndAsync();
                return await InputFormatterResult.SuccessAsync(content);
            }
        }
        if (contentType == "application/octet-stream")
        {
            using (var ms = new MemoryStream(2048))
            {
                await request.Body.CopyToAsync(ms);
                var content = ms.ToArray();
                return await InputFormatterResult.SuccessAsync(content);
            }
        }

        return await InputFormatterResult.FailureAsync();
    }
}

and at Startup.cs

public void ConfigureServices(IServiceCollection services)
{
    services.AddMvc(o => o.InputFormatters.Insert(0, new RawRequestBodyFormatter()));
}

and the controller

[HttpPost]
[Route("api/bytes/incoming")]
public byte[] RawBytesFormatter([FromBody] byte[] rawData)
{
    return rawData;
}

I took the server-side snippets from Rick Strahl excellent blog post about this: https://weblog.west-wind.com/posts/2017/Sep/14/Accepting-Raw-Request-Body-Content-in-ASPNET-Core-API-Controllers?Page=2

all kudos to him :)

Protobuf-net and WebApi, which format to use to return data over the wire?

2 Answers