multiple different protobuf messages in a socket

Question

I use protobuf with my c++ server. But I have a problem with multiple different protobuf messages in a socket I define many proto messages, such as

message SdkHGetRet {
  required int32 opcode = 1;
  required bytes value = 2;
}

message SdkHPut {
  required bytes table = 1;
  required bytes hname = 2;
  required bytes key = 3;
  required bytes value = 4;
  optional int32 writesrc = 5 [default = 0];
}

message SdkSet {
  required bytes table = 1;
  required bytes key = 2;
  required bytes value = 3;
  optional int32 writesrc = 4 [default = 0];
}

message SdkSetRet {
  required bool status = 1;
  optional string master = 2;
}

message SdkInvalidOperation {
  required int32 what = 1;
  required bytes why = 2;
}
....

So every time I send the message by socket, I will add 8 bytes, 4 for the total socket len, 4 for the opcode, opcode mean the message type.

So in the server side, I receive the message, I read the first 4 byte, and I got the message's length, then I will read another 4 bytes to get the message's type, at last I read the message's length bytes. Then I will use message type to method map ( such as 404 => "sdkset", 405 => "sdksetret") to decode the message.

I find this way work well, but I wonder is there any elegance way to identify the message without the 4 byte message type. I have read the message history, one method is add the entire message in a big message, like this way

message BigMessage
{
enum Type { sdkset = 0, sdksetred = 1}
require Type t = 1,
optional  string key = 2,
...
}

Since I have more than 40 types of message, I think this way may affect performance, and in my opinion, this way looks ugly.

So can you give me any good suggestion..

Kenton Varda Kenton Varda · Accepted Answer · 2015-08-24T07:19:12

Protobuf v2.6.0 introduced the oneof keyword for this. Example:

message BigMessage {
  oneof message {
    SdkHGetRet hgetret = 1;
    SdkHPut hput = 2;
    ...
  }
}

oneof ensures that exactly one of the fields is set, and lets you switch() on which one.

Note that even before Protobuf 2.6.0, the best solution would have been a series of optional fields, perhaps with an enum to specify which one is set:

message BigMessage {
  enum Type { HGETRET = 0, HPUT = 1, ... }
  required Type t = 1;
  optional SdkHGetRet hgetret = 2;
  optional SdkHPut hput = 3;
  ...
}

multiple different protobuf messages in a socket

1 Answers