2
votes

My results look like...

POST /post HTTP/1.1
Host: localhost:3003
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0)    
Gecko/20100101 Firefox/62.0
Accept: */*
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: http://localhost:3003/profile
Content-type: multipart/form-data
Content-Length: 14708
Cookie: mycookie=7bdbed41954cd4133a172acb92988e58
Connection: keep-alive

-----------------------------4636945214860352321751082034
...
binary characters...
...
-----------------------------4636945214860352321751082034

Acquired from

boost::asio::async_read(
    socket_,
    strmbuffer_,
    boost::asio::transfer_exactly(bytes_to_transfer),
    strand_.wrap(
    [this, self](boost::system::error_code ec, std::size_t bytes_transferred)
    {
        std::stringstream ss;
        ss << buffer_data;  // from socket_.async_read_some()
        ss << &strmbuffer_; // now stringstream contains everything

        // the character routine which writes the above
        // and which i use for output...

        std::string output_file = "../../upload/test.png";
        std::ofstream outfile(output_file);
        char c;
        unsigned bl = boundary.length();
        bool endfile = false;
        unsigned bufsize = 512;
        if(outfile){
            char buffer[bufsize];
            while(!endfile){
                // here looks like below
                // to find and pass the first boundary
            }   // then stream continues...
            while(!endfile){
                unsigned j = 0;
                unsigned k;
                memset(buffer, 0, bufsize); // redundant
                while(j < bufsize && ss.get(c) && !endfile){
                    buffer[j] = c;
                    k = 0;
                    while(boundary[bl - 1 - k] == buffer[j - k]){
                        if(k >= bl - 1){
                            endfile = true;
                            break;
                        }
                        k++;
                     }
                     j++;
                 }
                 outfile.write(buffer, j);
                 j = 0;
             }
         }
    }
);

...essentially. Thus, the receiving

socket_.async_read_some()

gives me a

boost::array<char, 8192> buffer_;

which gives me the http request info. But in the case of multipart/form-data, it reads passed the first boundary, which means the next read() doesn't see it. Argh! (So does async_read_until().) So, in

boost::asio::async_read()

i convert the

boost::asio::streambuf strmbuffer_;

to a stringstream and add them together for the std::cout results above.

I'm not convinced that I should be using stringstream, at all. But the above routine (using stringstream) works fine in Boost::Beast. It does NOT in Asio. Unfortunately, the string_body type, which receives the http request in Beast, has a restrictive size limit, 1 meg, i believe. No idea how to change that.

I can't find much on this topic, anywhere. Perhaps, the information is too dangerous. If they told me, they'd have to kill me. What should I be using in Asio to write binary data to disk??

2
one possibility for the problem - i've noticed that some functions, such as while() loops, are not being called in a serial manner. The output (cout) is not as clean as i have it, above.forresttales

2 Answers

2
votes

By default, the HTTP request parser starts with a 1 megabyte limit. This is to prevent resource exhaustion attacks where the client sends very large or endless amounts of body data. You can easily change this limit by calling parser::body_limit with the desired maximum. This is described in the documentation:

https://www.boost.org/doc/libs/1_68_0/libs/beast/doc/html/beast/ref/boost__beast__http__parser/body_limit.html https://www.boost.org/doc/libs/1_68_0/libs/beast/doc/html/beast/using_http/buffer_oriented_parsing.html

In order to adjust parser parameters such as the body limit (or header limit) you will need to use the "parser stream operation" interface. This is explained here:

https://www.boost.org/doc/libs/1_68_0/libs/beast/doc/html/beast/using_http/parser_stream_operations.html

2
votes

I will post my own partial solution. It works without error on a 14.2kb png. Above this can result in a segfault, unless the 'magic' number, as the following shows, is adjusted.

I'm using the Boost Asio HTTP Server example C++11.

In connection.hpp, change...

//boost::array<char, 8192> buffer_;
boost::array<char, 512> buffer_;

Also, add...

boost::asio::streambuf strmbuffer_;

In connection.cpp, here is the connection::handle_read()...

void connection::handle_read(const boost::system::error_code& e, std::size_t bytes_transferred)
{
    if (!e)
    {
        //*** buffer_.data() for this file ***
        //
        //POST /post HTTP/1.1
        //Host: localhost:3003
        //User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:62.0) Gecko/20100101 Firefox/62.0
        //Accept: */*
        //Accept-Language: en-US,en;q=0.5
        //Accept-Encoding: gzip, deflate
        //Referer: http://localhost:3003/profile
        //Content-type: multipart/form-data
        //Content-Length: 14710
        //Cookie: knowstoryadmin=7bdbed41954cd4133a172acb92988e58
        //Connection: keep-alive
        //
        //-----------------------------14071968205478138611648202646
        //Content-Disposition: form-data; name="admin_profile_image_load"; filename="tlc-logo.png"
        //Content-Type: image/png
        //
        //�PNG
        //▒

        std::stringstream strm1;
        std::string buffer_data = buffer_.data();
        strm1 << buffer_data;

        std::string method;
        std::smatch match_method;
        std::regex regex_method ("\\b([^ ]*)( )([^ ]*)( HTTP/1.1)([^ ]*)");

        std::string content_type;
        std::smatch match_content_type;
        std::regex regex_content_type ("\\b(Content-type: )([^ ]*)");

        std::string line;

        while (std::getline(strm1, line)) {
            if (std::regex_search(line, match_method, regex_method)) {
                method = match_method[0];
                method = method.substr(0, method.find(' '));
                boost::trim(method);
                //std::cout << method << std::endl;
            }
            if (std::regex_search(line, match_content_type, regex_content_type))     {
                content_type = match_content_type[0];
                boost::erase_all(content_type, "Content-type:");
                boost::trim(content_type);
                //std::cout << content_type << std::endl;
            }
        }

        if (method == "POST") {
            if (content_type == "multipart/form-data") {

                std::string content_length;
                std::smatch match_content_length;
                std::regex regex_content_length ("\\b(Content-Length: )([^ ]*)");

                std::string filename;
                std::smatch match_filename;
                std::regex regex_filename ("\\b(filename)([^ ]*)");

                std::string action;
                std::smatch match_action;
                std::regex regex_action ("\\b(name)([^ ]*)");

                std::string boundary;
                std::smatch match_boundary;
                std::regex regex_boundary ("([-]{10,}[0-9]{10,})");

                std::string line;

                strm1.clear();
                strm1 << buffer_data;
                while (std::getline(strm1, line)) {
                    if (std::regex_search(line, match_content_length, regex_content_length)) {
                        //Content-Length: 14710
                        content_length = match_content_length[0];
                        boost::erase_all(content_length, "Content-Length:");
                        boost::trim(content_length);
                        //std::cout << content_length << std::endl;
                    }
                    if (std::regex_search(line, match_filename, regex_filename)) {
                        filename = match_filename[0];
                        boost::erase_all(filename, "\"");
                        boost::erase_all(action, ";");
                        boost::erase_all(filename, "filename=");
                        std::size_t found = filename.find_last_of(".");
                        std::size_t len = filename.length();
                        std::string mime = filename.substr(found, len);
                        boost::trim(filename);
                        //std::cout << filename << std::endl;
                        //std::cout << mime << std::endl;
                    }
                    if (std::regex_search(line, match_action, regex_action)) {
                        action = match_action[0];
                        boost::erase_all(action, "\"");
                        boost::erase_all(action, ";");
                        boost::erase_all(action, "name=");
                        boost::trim(action);
                        //std::cout << action << std::endl;
                    }
                    if (std::regex_search(line, match_boundary, regex_boundary)) {
                        boundary = match_boundary[0];
                        boost::trim(boundary);
                        //std::cout << boundary << std::endl;
                    }
                }

                //pubseekpos works as expected, but useless here
                //strmbuffer_.pubseekpos(bytes_transferred);

                //content length minus bytes_transfered does NOT yield
                //the right result. The number, 392, is the 'magic' number
                //adjustment for this file size, approx 14.2kb, that i found
                //by trial and error.
                //Adjusting the magic number is necessary for every image size
                //in order to avoid a segfault.
                //bytes_transferred, for each read(), is the only 'reliable'
                //number with which to work, as far as i know.
                //If there is a brainier way of calculating this,
                //i don't care, anymore.
                int n_content_length = std::stoi(content_length);
                int transfer = n_content_length - bytes_transferred + 392;
                auto self(shared_from_this());
                boost::asio::async_read(
                    socket_,
                    strmbuffer_,
                    boost::asio::transfer_exactly(transfer),
                    strand_.wrap(
                        [this, self, boundary](boost::system::error_code ec, std::size_t bytes_transferred)
                        {
                            std::stringstream strm2;
                            strm2 << &strmbuffer_;
                            std::string line;
                            unsigned bufsize = 512;
                            while (std::getline(strm2, line))
                            {
                                if(line.length() == 1){
                                    std::string output_file = "../../upload/test.png";
                                    std::ofstream outfile(output_file);
                                    char c;
                                    unsigned bl = boundary.length();
                                    bool endfile = false;
                                    if(outfile){
                                        char buffer[bufsize];
                                        while(!endfile){
                                            unsigned j = 0;
                                            unsigned k;
                                            while(j < bufsize && strm2.get(c) && !endfile){
                                                buffer[j] = c;
                                                k = 0;
                                                while(boundary[bl - 1 - k] == buffer[j - k]){
                                                    if(k >= bl - 1){
                                                        endfile = true;
                                                        break;
                                                    }
                                                    k++;
                                                }
                                                j++;
                                            }
                                            outfile.write(buffer, j);
                                            j = 0;
                                        };
                                        outfile.close();
                                        std::cout << "outfile close" << std::endl;
                                        break;
                                    }
                                }
                            }
                        }
                    )
                );
            }
            else {
                // POST AJAX
                std::cout << "connection " << method << std::endl;
            }
        }
        else {
            boost::tribool result;
            boost::tie(result, boost::tuples::ignore) = request_parser_.parse(
                request_, buffer_.data(), buffer_.data() + bytes_transferred);

            if (result)
            {
                request_handler_.handle_request(
                    request_,
                    reply_);

                boost::asio::async_write(
                    socket_,
                    reply_.to_buffers(),
                    strand_.wrap(
                        boost::bind(
                            &connection::handle_write,
                            shared_from_this(),
                            boost::asio::placeholders::error)
                    ));
            }
            else if (!result)
            {
                reply_ = reply::stock_reply(reply::bad_request);
                boost::asio::async_write(
                    socket_,
                    reply_.to_buffers(),
                    strand_.wrap(
                        boost::bind(&connection::handle_write, shared_from_this(),
                        boost::asio::placeholders::error)));
            }
            else
            {
                socket_.async_read_some(
                    boost::asio::buffer(buffer_),
                    strand_.wrap(
                        boost::bind(
                            &connection::handle_read,
                            shared_from_this(),
                            boost::asio::placeholders::error,
                            boost::asio::placeholders::bytes_transferred)));
            }
        }
    }
}

This partial solution is rather 'unobtrusive' for this Asio example. The request_handler class is unchanged and serves GET.

In connection::start(), i tried async_read. But the results were even less predictable. It meant that i had to append two strings, one made from strmbuffer1_, the other from strmbuffer2_, in order to construct (hack) together the stream in the file output loop. No matter how precise my preparations appeared in cout<<, there were roughly 500 or more bytes missing, writing an incomplete image file.

Somewhere in memory there exists, and must exist, the complete information that i need for an uploaded binary and a handle to it. But extracting it from the Asio classes seems unnecessarily tricky.

This is how far I've come with Boost Asio. And this is as far as i go. For what i want, my answers are not here.