Storing binary data buffer in protocol buffer message

Question

I have a binary data buffer which i want to store in a protocol buffer.

In the documentation (https://developers.google.com/protocol-buffers/docs/proto#scalar) it says that the bytes type is equivalent to string in C++. I could not believe this so i had to try it and yes, this seems to be the case..

This proto:

message BufferMsg {
  required bytes buffer = 1;
}

gives a message definition containing this:

private:
  ::std::string* buffer_;

The public setter/getter API looks like this:

  // required bytes buffer = 1;
  inline bool has_buffer() const;
  inline void clear_buffer();
  static const int kBufferFieldNumber = 1;
  inline const ::std::string& buffer() const;
  inline void set_buffer(const ::std::string& value);
  inline void set_buffer(const char* value);
  inline void set_buffer(const void* value, size_t size);
  inline ::std::string* mutable_buffer();
  inline ::std::string* release_buffer();
  inline void set_allocated_buffer(::std::string* buffer);

Surely, this cannot be the way to store binary data in a message. How should it be done? In C++ i would typically use an unsigned char array or something like that to store the data.

"Surely, this cannot be the way to store binary data in a message. How should it be done?" What's your concern? I used that and it just works fine. — πάντα ῥεῖ
No there never was an issue with storing '\0' in std::string. Length is tracked independently of that. — πάντα ῥεῖ

Sebastian Redl Sebastian Redl · Accepted Answer · 2017-03-07T10:02:11

std::string, although meant to be used for text data, isn't exactly bound to it. It does no validation, and has in general no restrictions on the contents of its buffer. So you can use it as a std::vector<char> if you have to.

Embedded zeros are only an issue if you use c_str() and C-style string functions to work on the result.

The main problem is that char may be signed on your platform, which makes it less convenient to work with as a byte type than unsigned char. I'm afraid that's just something you have to live with.

Storing binary data buffer in protocol buffer message

1 Answers