Why is my protobuf deserialization giving empty payload?

Question

Running protobuf 3.9.0 on Windows with VS2017, I'm trying to serialize and deserialize my protobuf model using CodedOutputStream and CodedInputStream in C++, but the decoder always gives me an empty result. I use length-prefixing and can read the size from the stream on the decoder side. An in-place encode/decode source code is as follows, where the Any message is used

import "google/protobuf/any.proto";

message Pouch {
    google.protobuf.Any msg = 1;
}

bool EncodeDecode(google::protobuf::Message* pMeta, std::string& out_packet) {  
    //
    // Encoding
    //
    // CAUTION:
    // - Must dynamic allocate to avoid protobuf bug <SharedDtor crash #435>.
    std::shared_ptr<Pouch> pPayload = std::make_shared<Pouch>();
    google::protobuf::Any env;
    env.PackFrom(*pMeta);
    pPayload->set_allocated_msg(&env);  
    // Prefix with size
    const int nTipBytes = 4;
    int nBytesMsg = (int)pMeta->ByteSizeLong();
    int nBytesPacket = nBytesMsg + nTipBytes;
    out_packet.assign(nBytesPacket, '\0');
    google::protobuf::io::ArrayOutputStream aos((void*)out_packet.c_str(), nBytesPacket);
    google::protobuf::io::CodedOutputStream cos(&aos);
    cos.WriteVarint32(nBytesMsg);   
    bool res = pMeta->SerializeToCodedStream(&cos);
    printf("payload has message: %d\n", pPayload->has_msg());
    //
    // Decoding
    //
    Pouch out_pouch;
    google::protobuf::io::ArrayInputStream ais(out_packet.c_str(), nBytesPacket);
    google::protobuf::io::CodedInputStream cis(&ais);
    google::protobuf::uint32 nPayloadBytes;
    cis.ReadVarint32(&nPayloadBytes);   
    google::protobuf::io::CodedInputStream::Limit msgLimit = cis.PushLimit(nPayloadBytes);
    res = out_pouch.ParseFromCodedStream(&cis);
    cis.PopLimit(msgLimit);

    printf("decoded payload has message: %d\n", out_pouch.has_msg());

    pPayload->release_msg();
    return res;
}

The above code prints out an empty type URL

payload has message: 1
decoded payload has message: 0

But the size-prefix nPayloadBytes gives the correct number. The results from encoding and decoding are both true.

Where am I wrong?

Arthur Moraes Do Lago Arthur Moraes Do Lago · Accepted Answer · 2019-11-20T21:48:49

You are writing the pMeta package of any type to the CodedOutputStream cos when encoding, and expecting a a message Pouch when decoding. I think you meant to encode the pPayload message instead of pMeta, which will already contain the pMeta message in it's msg field. Full code used for testing:

#include <google/protobuf/io/zero_copy_stream.h>
#include <google/protobuf/io/coded_stream.h>
#include <google/protobuf/util/delimited_message_util.h>
#include "teste.pb.h"

#include <string>

bool EncodeDecode(google::protobuf::Message* pMeta, std::string& out_packet) {  
    //
    // Encoding
    //
    // CAUTION:
    // - Must dynamic allocate to avoid protobuf bug <SharedDtor crash #435>.
    std::shared_ptr<Pouch> pPayload = std::make_shared<Pouch>();
    google::protobuf::Any env;
    env.PackFrom(*pMeta, "bob.bob.bob");
    pPayload->set_allocated_msg(&env);  
    // Prefix with size
    const int nTipBytes = 4;
    int nBytesMsg = (int)pPayload->ByteSizeLong();
    int nBytesPacket = nBytesMsg + nTipBytes;
    out_packet.assign(nBytesPacket, '\0');

    google::protobuf::io::ArrayOutputStream aos((void*)out_packet.c_str(), nBytesPacket);
    google::protobuf::io::CodedOutputStream cos(&aos);
    cos.WriteVarint32(nBytesMsg);   
    bool res = pPayload->SerializeToCodedStream(&cos);
    printf("payload has message: %d\n", pPayload->has_msg());
    //
    // Decoding
    //
    Pouch out_pouch;
    google::protobuf::io::ArrayInputStream ais(out_packet.c_str(), nBytesPacket);
    google::protobuf::io::CodedInputStream cis(&ais);
    google::protobuf::uint32 nPayloadBytes;
    cis.ReadVarint32(&nPayloadBytes);   
    google::protobuf::io::CodedInputStream::Limit msgLimit = cis.PushLimit(nPayloadBytes);
    res = out_pouch.ParseFromCodedStream(&cis);
    cis.PopLimit(msgLimit);

    printf("decoded payload has message: %d\n", out_pouch.has_msg());
    if (out_pouch.has_msg()){
        Sample s;
        out_pouch.msg().UnpackTo(&s);
        printf("value: %d\n", s.number());
    }

    pPayload->release_msg();
    return res;
}

int main(){
    Sample *mymsg = new Sample();
    mymsg->set_number(5123);
    std::string bigstring;
    EncodeDecode(mymsg, bigstring);
}

and the .proto:

syntax = "proto3";
import "google/protobuf/any.proto";

message Pouch {
    google.protobuf.Any msg = 1;
}

message Sample {
    int32 number = 1;
}

Why is my protobuf deserialization giving empty payload?

1 Answers