std::string messed up when using it as storage within (boost.)async_read_some

Question

I am using async_read_some to read data from a port that is saved in a char[] called _data. Its buffer size is big enough for every request:

void start() {

    socket_.async_read_some(boost::asio::buffer(data_,BUFFERSIZE),make_custom_alloc_handler(allocator_,boost::bind(&attentionSession::handle_read, shared_from_this(), boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred)));

}

void handle_read(const boost::system::error_code& error, size_t bytes_transferred) {

    string ip = socket_.remote_endpoint().address().to_string();
    log->processData(data_,ip,"memory");
    strcpy(data_,"");

}

processData adds some additional information (like timestamp etc.) to the request by copying it to another newly alloced char*. Then this char[] is sent to writeToMemory(char*) to append that char* to the std::string memoryBuffer:

void writeCacheToFile() {

    // This function writes buffer data to the log file

    char* temp = new char[memoryBuffer.length() + 1];
    strcpy(temp, memoryBuffer.c_str());
    writeToFile(temp);
    delete[] temp;
    memoryBuffer.clear();

}

void writeToMemory(char* data) {

    int maxSize = 1024;

     // Checks if char* data would 'fit' into the pre-defined maxSize

    if ((strlen((const char*)data) + memoryBuffer.length()) >= maxSize) {
        writeCacheToFile(); // If not the cache memoryBuffer is saved first
    }

    memoryBuffer.append((const char*) data);

    cout << memoryBuffer.length() << endl;

}

It works but if there are constantly requests (bombarding it with requests) things get messed up. As you can see above in the writeToMemory() function I'll added a line to print out the current length of memoryBuffer and this is where I think it has somethings to do with thread safety of std::strings:

The length of each (processed by processData()) request is 96 characters. But here the length of memoryBuffer just rises and falls down - some lengths are even bigger than maxSize (1024 chars).

EDIT: Sam pointed out I should add some more code. This is how I start the io_service:

boost::asio::io_service ioService;
boost::scoped_ptr<boost::thread> ioServiceThread; 
server_class server (ioService,PORT); // Create server instance
ioServiceThread.reset (new boost::thread ( boost::bind ( &boost::asio::io_service::run, &ioService  ) ) ); 
// Only one threaded io_service (to receive user inputs in main() function)

And this is the async_acceptor's function after completing a request:

typedef boost::shared_ptr<session_class> session_ptr;

void handleAccept(session_ptr thisSession, const boost::system::error_code& error) {
    if (!error) {
      thisSession->start(); // Calls the start() function above
      thisSession.reset(new session(ioService,LOGGING_CLASS));
      acceptor.async_accept(thisSession->socket(),boost::bind(&server_class::handleAccept, this, thisSession, PLACEHOLDER_ERROR));
    }
 }

The session_class holds the functions start() and handle_read(x,y) mentioned above. LOGGING_CLASS provides the class to write log files (holds the functions writeCacheToFile() and writeToMemory(char*)). log (mentioned above) is a kind of this class.

EOE: END OF EDIT

If I try to fix outsource the caching part (appending received char* to std::string) with boost::threads it ends up with a totally mixed up memoryBuffer

Is it really the thread safety of std::strings or something else I missed?

Thanks for your help in advance! :)

Paul

This topic is also here on StackOverflow link but I don't know whether g++ 4.2.1 supports thread safe std::strings. — Paul
The current standard does not mention threads, so a std::string is not thread safe. What are you trying to do with multiple threads anyhow? — Sam Miller
To outsource those copying processes of the char* to the string memoryBuffer. This failed. But async_read_some is also some kind of threading or am I wrong? — Paul
The async_read_some method is not some kind of threading. The completion handlers can be invoked by multiple threads if you have a pool of them invoking io_service::run(). How are you invoking io_service::run() or io_service::poll()? — Sam Miller

Sam Miller Sam Miller · Accepted Answer · 2011-02-26T16:59:39

I made this same point as a comment but I figured it would be worth expanding into an answer. Posting the snippets of code you have are not terribly helpful, they don't give us the full picture. For example, the following concepts are not clear to me:

You're not checking the bytes_transferred parameter in your async_read_some handler. This is very important because even though you tell it to read n bytes it can return when reading n - x bytes where x <= n. As the documentation states, you should consider using one of the composed operations like the async_read free function.
you're using custom memory allocations for your asynchronous read operations, presumably based on the example provided. Why do you need that?
buffer lifetime. Ex: do your buffers stay in scope until the async_read handler is invoked?
object lifetime. Ex: are you using shared_ptr properly? Is the io_service in scope for the entirety of its event loop?
are you using a single io_service per process or one per thread?
why you need threads? Typically it's easier to understand asynchronous programming in a single threaded context first.

All of these are very important concepts to get right when using Boost.Asio. Part of debugging is boiling down a perceived problem into a smaller reproducer. This is useful both on Stack Overflow and for becomming a good programmer in general. It will help you understand the problem, and also help us help you find it. I strongly suggest you spend some effort on making a smaller reproducible example that we can compile. If that's not possible, consider using multiple threads only after you have proved a single threaded scenario works.

std::string messed up when using it as storage within (boost.)async_read_some

3 Answers