how to deserialize an int from wire format from socket in C++

Question

I have a protobuf message that contains a header and a body. The header has a member called size (reflecting the size of the entire message, i.e header plus body). Since protobuf (AFAIK) does not provide message boundary, I use the size (first element of header and fixed size of 4 bytes) to know how much to slurp from the socket.

Assuming I have read (or recv(2)ed) 4 bytes from the socket, I now need to convert (decode) this to an int so I can instruct recv(2) to slurp that much more.

How do I convert / decode this wire format to an int in C++ land ?

I have tried ParseFromString() but this function is returning a boolean and the doc does not tell me much.

I will show what I think is relevant...lets see

message PsdAgentMsg { 
    message Header { 
       fixed32 theSize = 1; // size includes header and message 
       uint32 theInstanceId = 2; 
       Type theMsgType = 3; 
    } 
    Header theHeader = 1;
    oneof theMsg { 
       abc = 2; 
       def = 3; 
       ghi = 4; 
       klm = 5; 
       PsdAgentGPCMsg theGPCMsg = 6; 
    } 
} 
message PsdAgentGPCMsg { 
    int32 theCount = 1; 
} 
...few minutes later..

PsdAgentMsg *msg = new PsdAgentMsg();   // psd msg is header + body
...
rc = recv(sock, buf, 4, 0);     // I am simplifying some stuff
std::string sizeString = buf;
...
size_t payloadSize = msg->ParseFromString(sizeString);

I am finding that payloadSize = 0. which is a false being assigned to a size_t. So ParseFromString() does not seem to be the right way to decode the 4 bytes. Again, I need to decode the sizeString to an int. so I can say recv(sock, buf, payloadSize,0)

What exactly is sent? Is it a message with a single int32 field? What's the type of payloadSize? This question needs a minimal reproducible example to be answerable. — Miles Budnek
Sorry I tried to insert some litteral text , but couldn't figure it out. I tried a single back quote, triple back tick, html tags...to no avail — Medi Montaseri
We need to know what the wire format for the size is. There are lots of possible choices and without knowing which choice was made, it's not possible to write the code. — David Schwartz

Miles Budnek Miles Budnek · Accepted Answer · 2019-07-04T03:54:32

Embedded message fields use wire-type 2: length delimited.

That means that it's encoded like this:

<varint tagAndType> <varint messageLength> <theSize> <theInstanceId> <theMsgType>

Where a varint is a base-128 varint as defined in the protobuf encoding specification.

What this means is that theSize isn't at a fixed offset from the beginning of the message. To make things easer, you could send the total size of the message before you send the message itself. i.e. given a message like this:

message PsdAgentMsg { 
    message Header { 
       uint32 theInstanceId = 2; 
       Type theMsgType = 3; 
    }

    Header theHeader = 1;
    oneof theMsg { 
       int32 abc = 2;
       string def = 3;
       double ghi = 4;
       uint32 klm = 5;
       PsdAgentGPCMsg theGPCMsg = 6; 
    } 
}

You could send your data as something like this:

PsdAgentMsg msg;
fillOutMessageFields(msg);

std::string encoded_msg = msg.SerializeAsString();
uint32_t size = msg.size();

// First send the message size as a fixed-size integer
size = htonl(size);
send(sockfd, &size, sizeof(size), 0);

// Then send the message itself
send(sockfd, msg.c_str(), msg.size(), 0);

Then you could read it back at the other end like this:

// Read the size
uint32_t size;
rc = recv(sockfd, &size, sizeof(size), 0);
assert(rc == sizeof(size));  // Or proper error handling
size = ntohl(size);

// Read the actual message
std::string encoded_msg(size, '\0');
rc = recv(sockfd, &encoded_msg[0], size, 0);
assert(rc == size);  // Or proper error handling

// Parse the message
PsdAgentMsg msg;
msg.ParseFromString(encoded_msg);

how to deserialize an int from wire format from socket in C++

1 Answers