How do you use boost/beast to parse and extract the payload from an HTTP POST request?

Question

I am trying to understand how to process the contents of an HTTP POST request using the Boost Beast library. I have slightly modified the Boost Beast advanced server example to get a handle on things.

I've added the following lines to the handle_request() method in the example (just before line 155):

    if ( req.method() == http::verb::post)
    {
      std::cout << req << std::endl;
    }

I have created a simple test file called foobar.dat containing the following:

This is a test!

I send it to the server using this curl command:

curl -F '[email protected]' http://localhost:8080

which results in the following output from the server:

POST / HTTP/1.1Host: localhost:8080
User-Agent: curl/7.58.0
Accept: */*
Content-Length: 218
Content-Type: multipart/form-data; boundary=------------------------9c747f078ebbe880

--------------------------9c747f078ebbe880
Content-Disposition: form-data; name="test"; filename="foobar.dat"
Content-Type: application/octet-stream

This is a test!

--------------------------9c747f078ebbe880--

So, I have the server receiving the expected message.

If I modify my test slightly to the following, I can extract the header fields individually as well as the body in one big buffer.

    if ( req.method() == http::verb::post)
    {
      std::cout << "Fields:" << std::endl;

      int field_count = 0;
      for(auto const& field : req)
          std::cout << "Field#"
                    << ++field_count << " : " << field.name() << " = " << field.value() << std::endl;

      std::cout << "Body:" << std::endl;
      int body_count = 0;
      for(auto it = boost::asio::buffer_sequence_begin(req.body().data());
          it != boost::asio::buffer_sequence_end(req.body().data()); ++it)
      {
        // This is the next buffer in the sequence
        boost::asio::const_buffer const buffer = *it;

        std::string body(boost::asio::buffer_cast<const char*>(buffer));
        std::cout << "Buffer#" << ++body_count << " = " << body << std::endl;
      }
    }

Produces the following output:

Fields:
Field#1 : Host = localhost:8080
Field#2 : User-Agent = curl/7.58.0
Field#3 : Accept = */*
Field#4 : Content-Length = 218
Field#5 : Content-Type = multipart/form-data; boundary=------------------------5510ea3ec81b8585
Body:
Buffer#1 = --------------------------5510ea3ec81b8585
Content-Disposition: form-data; name="test"; filename="foobar.dat"
Content-Type: application/octet-stream

This is a test!

--------------------------5510ea3ec81b8585--

Most of the examples I have found demonstrate how to create responses and requests using Boost Beast, but I cannot find any clear examples of what to do to parse the content and separate the various components of the message.

Specifically, how do I use Boost Beast to extract and separate the name ("test"), the filename ("foobar.dat"), and the file contents ("This is a test!") from the body in order to process the message further? Or, at this point, is it necessary to parse the data in the body of the message myself?

Tom Trebicky Tom Trebicky · Accepted Answer · 2020-05-08T13:18:23

That's outside the scope of Boost::Beast. You will have to do it by yourself regardless whether the information you seek resides in fields or body. I would suggest one of the handy string operation tools/utilities/libraries, e.g. Boost::Algorithm::String or Abseil (absl::StrSplit).

Side note: You can access the fields like you did or directly like so

auto field = req["<field_name>"];
// or
auto it = req.find("<field_name>");
it->name_string();

Lines in HTML bodies are separated by carriage return and new line \r\n. Body from headers by double \r\n. You can access individual lines by writing a fancy, optimized, parser or by using one of the available utilities. Split along lines, split along colon, split along semi-colon to gain access to name or filename and lastly split along equal sign to get the data out. Example using Abseil since I am not sure if boost can split along a full string (string_view):

std::vector<std::string_view> lines = absl::StrSplit(req.body(), "\r\n");
std::vector<std::string_view> headers = absl::StrSplit(line, ':', absl::SkipWhitespace());
// or possibly (never tried it myself but abseil is a great library so I assume this should work)
std::array<std::string_view, 2> headers = absl::StrSplit(line, ':', absl::SkipWhitespace());
std::vector<std::string_view> items = absl::StrSplit(header, ';', absl::SkipWhitespace());
std::vector<std::string_view> values = absl::StrSplit(item, '=', absl::SkipWhitespace());
// or possibly again using array

How do you use boost/beast to parse and extract the payload from an HTTP POST request?

1 Answers