How to use asio buffer after async_read_until for consecutive reads

Question

I am reading from a serial device where each message must be specifically requested. E.g. you send a request and get a response with the serialised payload.

Each message contains these parts in order:

PREAMBLE (2 bytes, "$M")
HEADER (3 bytes, containing payload length N)
PAYLOAD+CRC (N+1 bytes)

My approach with asio is to detect the start (PREAMBLE) of a message by using asio::async_read_until and afterwards using asio::async_read for reading the exact amount of bytes for HEADER and PAYLOAD+CRC. Since there is no static pattern at the end of the message, I cannot use async_read_until to read the full message.

After receiving PREAMBLE, the handler for async_read_until gets called and the buffer contains the PREAMBLE bytes and might contain additional bytes from HEADER and PAYLOAD+CRC. The asio documentation for async_read_until says:

After a successful async_read_until operation, the streambuf may contain additional data beyond the delimiter. An application will typically leave that data in the streambuf for a subsequent async_read_until operation to examine.

I interpret this as that you should only consume the requested bytes and leave all remaining bytes in the buffer for further reads. However, all consecutive reads block since the data is already in the buffer and there is nothing left on the device.

The reading is implemented as a small state machine processState, where different handlers are registered depending on which part of the message is to be read. All reading is done with the same buffer (asio::streambuf). processState is called in an infinite loop.

void processState() {
    // register handler for incomming messages
    std::cout << "state: " << parser_state << std::endl;
    switch (parser_state) {
    case READ_PREAMBLE:
        asio::async_read_until(port, buffer, "$M",
            std::bind(&Client::onPreamble, this, std::placeholders::_1, std::placeholders::_2));
        break;
    case READ_HEADER:
        asio::async_read(port, buffer, asio::transfer_exactly(3),
            std::bind(&Client::onHeader, this, std::placeholders::_1, std::placeholders::_2));
        break;
    case READ_PAYLOAD_CRC:
        asio::async_read(port, buffer, asio::transfer_exactly(request_received->length+1),
            std::bind(&Client::onDataCRC, this, std::placeholders::_1, std::placeholders::_2));
        break;
    case PROCESS_PAYLOAD:
        onProcessMessage();
        break;
    case END:
        parser_state = READ_PREAMBLE;
        break;
    }
    // wait for incoming data
    io.run();
    io.reset();
}

The PREAMBLE handler onPreamble is called when receiving the PREAMBLE:

void onPreamble(const asio::error_code& error, const std::size_t bytes_transferred) {
    std::cout << "onPreamble START" << std::endl;
    if(error) { return; }

    std::cout << "buffer: " << buffer.in_avail() << "/" << buffer.size() << std::endl;

    // ignore and remove header bytes
    buffer.consume(bytes_transferred);

    std::cout << "buffer: " << buffer.in_avail() << "/" << buffer.size() << std::endl;

    buffer.commit(buffer.size());

    std::cout << "onPreamble END" << std::endl;
    parser_state = READ_HEADER;
}

After this handler, no other handlers get called since the data is in the buffer and no data is left on the device.

What is the correct way to use asio::streambuf such that the handlers of consecutive async_read get called and I can process bytes in order of the state machine? I don't want to process the remaining bytes in onPreamble since it is not guaranteed that these will contain the full message.

v1bri v1bri · Accepted Answer · 2017-08-23T03:29:13

You don't need the call to buffer.commit() in the onPreamble() handler. Calling buffer.consume() will remove the header bytes as you expect and leave the remaining bytes (if any were received) in the asio::streambuf for the next read. The streambuf's prepare() and commit() calls are used when you are filling it with data to send to the remote party.

I just finished a blog post and codecast about using the asio::streambuf to perform a simple HTTP GET with a few web servers. It might give you a better idea of how to use async_read_until() and async_read().

How to use asio buffer after async_read_until for consecutive reads

1 Answers