How can I convert serialized data in boost::beast to a string so that I could process it in a FIFO manner?

Question

I have an application of a client where I need to receive http "long running requests" from a server. I send a command, and after getting the header of the response, I have to just receive json data separated by \r\n until the connection is terminated.

I managed to adapt boost beast client example to send the message and receive the header and parse it and receive responses from the server. However, I failed at finding a way to serialize the data so that I could process the json messages.

The closest demonstration of the problem can be found in this relay example. In that example (p is a parser, sr is a serializer, input is a socket input stream and output is an socket output stream), after reading the http header, we have a loop that reads continuously from the server:

do
{
    if(! p.is_done())
    {
        // Set up the body for writing into our small buffer
        p.get().body().data = buf;
        p.get().body().size = sizeof(buf);

        // Read as much as we can
        read(input, buffer, p, ec);

        // This error is returned when buffer_body uses up the buffer
        if(ec == error::need_buffer)
            ec = {};
        if(ec)
            return;

        // Set up the body for reading.
        // This is how much was parsed:
        p.get().body().size = sizeof(buf) - p.get().body().size;
        p.get().body().data = buf;
        p.get().body().more = ! p.is_done();
    }
    else
    {
        p.get().body().data = nullptr;
        p.get().body().size = 0;
    }

    // Write everything in the buffer (which might be empty)
    write(output, sr, ec);

    // This error is returned when buffer_body uses up the buffer
    if(ec == error::need_buffer)
        ec = {};
    if(ec)
        return;
}
while(! p.is_done() && ! sr.is_done());

A few things I don't understand here:

We're done reading the header. Why do we need boost beast and not boost asio to read a raw tcp message? When I tried to do that (with both async_read/async_read_some) I got an infinite reads of zero size.
The documentation of parser says (at the end of the page) that a new instance is needed for every message, but I don't see that in the example.
Since tcp message reading is not working, is there a way to convert the parser/serializer data to some kind of string? Even write it to a text file in a FIFO manner, so that I could process it with some json library? I don't want to use another socket like the example.

The function boost::beast::buffers() failed to compile for the parser and the serializer, and for the parser there's no consume function, and the serializer's consume seems to be for particular http parts of the message, which fires an assert if I do it for body().

Besides that, I also failed at getting consistent chunks of data from the parser and the buffer with old-school std::copy. I don't seem to understand how to combine the data together to get the stream of data. Consuming the buffer with .consume() at any point while receiving data leads to need buffer error.

I would really appreciate someone explaining the logic of how all this should work together.

Vinnie Falco Vinnie Falco · Accepted Answer · 2019-04-10T22:10:26

Where is buf? You could read directly into the std::string instead. Call string.resize(N), and set the pointer and size in the buffer_body::value_type to string.data() and string.size().

How can I convert serialized data in boost::beast to a string so that I could process it in a FIFO manner?

1 Answers