Reading from boost asio streambuf keeps bytes in buffer

Question

I have a problem with reading from boost asio's stream. The first invocation of an async_read_until gives me 460 bytes transfered( I checked that with wireshark). After that I use an istream initialized with the streambuf pointer to use std::copy_n with an istreambuf_iterator. That works fine and the target of std::copy_n holds the request up to the delimiter sequence.

The strange thing happens after the next call to async_read_until. It seems that the last character was not read from the streambuf so the next handler call gives me one byte more than the actual size of the request.

Are there any limitations using istream with asio's streambuf?

are you using the commit() and consume() functions correctly? Can you show minimal, complete code that I can copy paste into my IDE to check? — Richard Hodges
I don't use any of the commit or consume function. I just use the streambuffer for the std::istream and use istreambuf_iterators. After a bit experimenting I found the problem: std::copy_n is doing strange things with iterators. Using a while loop and calling the ++ operator works fine. Is there something wrong with not using commit or consume and just making an ostream or istream from the streambuf? — Gustavo
If you use the streambuffer within an istream or ostream, the commit/consume mechanics are done for you. — Richard Hodges
Ok then everything is fine now. I ended up not using std::copy_n because it seems to omit consuming the last character so it still remains in the buffer. That interferes with the next read operation. Don't know if that is intended by std::copy_n. — Gustavo
@Gustavo copy_n intends to copy n characters. Since the source is an input iterator, it should be expected to be incremented (at most) n times. en.cppreference.com/w/cpp/algorithm/copy_n — sehe

Richard Hodges Richard Hodges · Accepted Answer · 2017-10-23T10:06:21

In addition to the comments, here's a little demo program that shows two ways of interacting with an asio::streambuf.

One way wraps the streambuf in i/o streams, the other uses direct access with prepare/commit and data/consume.

#include <boost/asio.hpp>
#include <iostream>
#include <string>
#include <algorithm>
#include <memory>

namespace asio = boost::asio;

void direct_insert(asio::streambuf& sb, std::string const& data)
{
    auto size = data.size();
    auto buffer = sb.prepare(size);
    std::copy(begin(data), end(data), asio::buffer_cast<char*>(buffer));
    sb.commit(size);
}

void stream_insert(asio::streambuf& sb, std::string const& data)
{
    std::ostream strm(std::addressof(sb));
    strm << data;
}

std::string extract_istream(asio::streambuf& sb)
{
    std::istream is(std::addressof(sb));
    std::string line;
    std::getline(is, line);
    return line;
}

std::string extract_direct(asio::streambuf& sb)
{
    auto buffer = sb.data();
    auto first = asio::buffer_cast<const char*>(buffer);
    auto bufsiz = asio::buffer_size(buffer);
    auto last = first + bufsiz;

    auto nlpos = std::find(first, last, '\n');

    auto result = std::string(first, nlpos);

    auto to_consume = std::min(std::size_t(std::distance(first, nlpos) + 1), bufsiz);
    sb.consume(to_consume);
    return result;
}

int main()
{
    asio::streambuf buf;
    direct_insert(buf, "The cat sat on the mat\n");
    stream_insert(buf, "The cat sat on the mat\n");

    auto s1 = extract_direct(buf);
    auto s2 = extract_istream(buf);

    std::cout << s1 << "\n" << s2 << "\n";
}

Reading from boost asio streambuf keeps bytes in buffer

1 Answers