rest - HTTP GET with request body

Question

I'm developing a new RESTful webservice for our application.

When doing a GET on certain entities, clients can request the contents of the entity. If they want to add some parameters (for example sorting a list) they can add these parameters in the query string.

Alternatively I want people to be able to specify these parameters in the request body. HTTP/1.1 does not seem to explicitly forbid this. This will allow them to specify more information, might make it easier to specify complex XML requests.

My questions:

• Is this a good idea altogether?
• Will HTTP clients have issues with using request bodies within a GET request?

http://tools.ietf.org/html/rfc2616

Answer 1

Roy Fielding's comment about including a body with a GET request.

Yes. In other words, any HTTP request message is allowed to contain a message body, and thus must parse messages with that in mind. Server semantics for GET, however, are restricted such that a body, if any, has no semantic meaning to the request. The requirements on parsing are separate from the requirements on method semantics.

So, yes, you can send a body with GET, and no, it is never useful to do so.

This is part of the layered design of HTTP/1.1 that will become clear again once the spec is partitioned (work in progress).

....Roy

Yes, you can send a request body with GET but it should not have any meaning. If you give it meaning by parsing it on the server and changing your response based on its contents, then you are ignoring this recommendation in the HTTP/1.1 spec, section 4.3:

...if the request method does not include defined semantics for an entity-body, then the message-body SHOULD be ignored when handling the request.

And the description of the GET method in the HTTP/1.1 spec, section 9.3:

The GET method means retrieve whatever information ([...]) is identified by the Request-URI.

which states that the request-body is not part of the identification of the resource in a GET request, only the request URI.

Update

The RFC2616 referenced as "HTTP/1.1 spec" is now obsolete. In 2014 it was replaced by RFCs 7230-7237. Quote "the message-body SHOULD be ignored when handling the request" has been deleted. It's now just "Request message framing is independent of method semantics, even if the method doesn't define any use for a message body" The 2nd quote "The GET method means retrieve whatever information ... is identified by the Request-URI" was deleted. - From a comment

From the HTTP 1.1 2014 Spec:

A payload within a GET request message has no defined semantics; sending a payload body on a GET request might cause some existing implementations to reject the request.

Answer 2

While you can do that, insofar as it isn't explicitly precluded by the HTTP specification, I would suggest avoiding it simply because people don't expect things to work that way. There are many phases in an HTTP request chain and while they "mostly" conform to the HTTP spec, the only thing you're assured is that they will behave as traditionally used by web browsers. (I'm thinking of things like transparent proxies, accelerators, A/V toolkits, etc.)

This is the spirit behind the Robustness Principle roughly "be liberal in what you accept, and conservative in what you send", you don't want to push the boundaries of a specification without good reason.

However, if you have a good reason, go for it.

Answer 3

You will likely encounter problems if you ever try to take advantage of caching. Proxies are not going to look in the GET body to see if the parameters have an impact on the response.

Answer 4

Neither restclient nor REST console support this but curl does.

The HTTP specification says in section 4.3

A message-body MUST NOT be included in a request if the specification of the request method (section 5.1.1) does not allow sending an entity-body in requests.

Section 5.1.1 redirects us to section 9.x for the various methods. None of them explicitly prohibit the inclusion of a message body. However...

Section 5.2 says

The exact resource identified by an Internet request is determined by examining both the Request-URI and the Host header field.

and Section 9.3 says

The GET method means retrieve whatever information (in the form of an entity) is identified by the Request-URI.

Which together suggest that when processing a GET request, a server is not required to examine anything other that the Request-URI and Host header field.

In summary, the HTTP spec doesn't prevent you from sending a message-body with GET but there is sufficient ambiguity that it wouldn't surprise me if it was not supported by all servers.

Answer 5

Elasticsearch accepts GET requests with a body. It even seems that this is the preferred way: Elasticsearch guide

Some client libraries (like the Ruby driver) can log the cry command to stdout in development mode and it is using this syntax extensively.

Answer 6

You can either send a GET with a body or send a POST and give up RESTish religiosity (it's not so bad, 5 years ago there was only one member of that faith -- his comments linked above).

Neither are great decisions, but sending a GET body may prevent problems for some clients -- and some servers.

Doing a POST might have obstacles with some RESTish frameworks.

Julian Reschke suggested above using a non-standard HTTP header like "SEARCH" which could be an elegant solution, except that it's even less likely to be supported.

It might be most productive to list clients that can and cannot do each of the above.

Clients that cannot send a GET with body (that I know of):

• XmlHTTPRequest Fiddler

Clients that can send a GET with body:

• most browsers

Servers & libraries that can retrieve a body from GET:

• Apache
• PHP

Servers (and proxies) that strip a body from GET:

• ?

Answer 7

What you're trying to achieve has been done for a long time with a much more common method, and one that doesn't rely on using a payload with GET.

You can simply build your specific search mediatype, or if you want to be more RESTful, use something like OpenSearch, and POST the request to the URI the server instructed, say /search. The server can then generate the search result or build the final URI and redirect using a 303.

This has the advantage of following the traditional PRG method, helps cache intermediaries cache the results, etc.

That said, URIs are encoded anyway for anything that is not ASCII, and so are application/x-www-form-urlencoded and multipart/form-data. I'd recommend using this rather than creating yet another custom json format if your intention is to support ReSTful scenarios.

Answer 8

I put this question to the IETF HTTP WG. The comment from Roy Fielding (author of http/1.1 document in 1998) was that

"... an implementation would be broken to do anything other than to parse and discard that body if received"

RFC 7213 (HTTPbis) states:

"A payload within a GET request message has no defined semantics;"

It seems clear now that the intention was that semantic meaning on GET request bodies is prohibited, which means that the request body can't be used to affect the result.

There are proxies out there that will definitely break your request in various ways if you include a body on GET.

So in summary, don't do it.

Answer 9

Which server will ignore it? – fijiaaron Aug 30 '12 at 21:27

Google for instance is doing worse than ignoring it, it will consider it an error!

Try it yourself with a simple netcat:

$ netcat www.google.com 80
GET / HTTP/1.1
Host: www.google.com
Content-length: 6

1234

(the 1234 content is followed by CR-LF, so that is a total of 6 bytes)

and you will get:

HTTP/1.1 400 Bad Request
Server: GFE/2.0
(....)
Error 400 (Bad Request)
400. That’s an error.
Your client has issued a malformed or illegal request. That’s all we know.

You do also get 400 Bad Request from Bing, Apple, etc... which are served by AkamaiGhost.

So I wouldn't advise using GET requests with a body entity.

Answer 10

From RFC 2616, section 4.3, "Message Body":

A server SHOULD read and forward a message-body on any request; if the request method does not include defined semantics for an entity-body, then the message-body SHOULD be ignored when handling the request.

That is, servers should always read any provided request body from the network (check Content-Length or read a chunked body, etc). Also, proxies should forward any such request body they receive. Then, if the RFC defines semantics for the body for the given method, the server can actually use the request body in generating a response. However, if the RFC does not define semantics for the body, then the server should ignore it.

This is in line with the quote from Fielding above.

Section 9.3, "GET", describes the semantics of the GET method, and doesn't mention request bodies. Therefore, a server should ignore any request body it receives on a GET request.

Answer 11

According to XMLHttpRequest, it's not valid. From the standard:

4.5.6 The send() method

client . send([body = null])

Initiates the request. The optional argument provides the request body. The argument is ignored if request method is GET or HEAD.

Throws an InvalidStateError exception if either state is not opened or the send() flag is set.

The send(body) method must run these steps:

1. If state is not opened, throw an InvalidStateError exception.
2. If the send() flag is set, throw an InvalidStateError exception.
3. If the request method is GET or HEAD, set body to null.
4. If body is null, go to the next step.

Although, I don't think it should because GET request might need big body content.

So, if you rely on XMLHttpRequest of a browser, it's likely it won't work.

Answer 12

If you really want to send cachable JSON/XML body to web application the only reasonable place to put your data is query string encoded with RFC4648: Base 64 Encoding with URL and Filename Safe Alphabet. Of course you could just urlencode JSON and put is in URL param's value, but Base64 gives smaller result. Keep in mind that there are URL size restrictions, see What is the maximum length of a URL in different browsers? .

You may think that Base64's padding = character may be bad for URL's param value, however it seems not - see this discussion: http://mail.python.org/pipermail/python-bugs-list/2007-February/037195.html . However you shouldn't put encoded data without param name because encoded string with padding will be interpreted as param key with empty value. I would use something like ?_b64=<encodeddata>.

Answer 13

I wouldn't advise this, it goes against standard practices, and doesn't offer that much in return. You want to keep the body for content, not options.

Answer 14

You have a list of options which are far better than using a request body with GET.

Let' assume you have categories and items for each category. Both to be identified by an id ("catid" / "itemid" for the sake of this example). You want to sort according to another parameter "sortby" in a specific "order". You want to pass parameters for "sortby" and "order":

You can:

1. Use query strings, e.g. example.com/category/{catid}/item/{itemid}?sortby=itemname&order=asc
2. Use mod_rewrite (or similar) for paths: example.com/category/{catid}/item/{itemid}/{sortby}/{order}
3. Use individual HTTP headers you pass with the request
4. Use a different method, e.g. POST, to retrieve a resource.

All have their downsides, but are far better than using a GET with a body.

Answer 15

What about nonconforming base64 encoded headers? "SOMETHINGAPP-PARAMS:sdfSD45fdg45/aS"

Length restrictions hm. Can't you make your POST handling distinguish between the meanings? If you want simple parameters like sorting, I don't see why this would be a problem. I guess it's certainty you're worried about.

Answer 16

I'm upset that REST as protocol doesn't support OOP and Get method is proof. As a solution, you can serialize your a DTO to JSON and then create a query string. On server side you'll able to deserialize the query string to the DTO.

Take a look on:

• Message-based design in ServiceStack
• Building RESTful Message Based Web Services with WCF

Message based approach can help you to solve Get method restriction. You'll able to send any DTO as with request body

Nelibur web service framework provides functionality which you can use

var client = new JsonServiceClient(Settings.Default.ServiceAddress);
var request = new GetClientRequest
    {
        Id = new Guid("2217239b0e-b35b-4d32-95c7-5db43e2bd573")
    };
var response = client.Get<GetClientRequest, ClientResponse>(request);

as you can see, the GetClientRequest was encoded to the following query string

http://localhost/clients/GetWithResponse?type=GetClientRequest&data=%7B%22Id%22:%2217239b0e-b35b-4d32-95c7-5db43e2bd573%22%7D

Answer 17

IMHO you could just send the JSON encoded (ie. encodeURIComponent) in the URL, this way you do not violate the HTTP specs and get your JSON to the server.

Answer 18

For example, it works with Curl, Apache and PHP.

PHP file:

<?php
echo $_SERVER['REQUEST_METHOD'] . PHP_EOL;
echo file_get_contents('php://input') . PHP_EOL;

Console command:

$ curl -X GET -H "Content-Type: application/json" -d '{"the": "body"}' 'http://localhost/test/get.php'

Output:

GET
{"the": "body"}

Answer 19

I am using the Spring framework's RestTemplate in my client programme and, at the server-side, I defined a GET request with a Json body. My primary purpose is the same as yours: when the request has numerous parameters, putting them in the body seems tidier than putting them in the prolonged URI string. Yes?

But, sadly, it's not working! The server-side threw the following exception:

org.springframework.http.converter.HttpMessageNotReadableException: Required request body is missing...

But I am pretty sure the message body is correctly provided by my client code, so what's wrong?

I traced into the RestTemplate.exchange() method and found the following:

// SimpleClientHttpRequestFactory.class
public class SimpleClientHttpRequestFactory implements ClientHttpRequestFactory, AsyncClientHttpRequestFactory {
    ...
    protected void prepareConnection(HttpURLConnection connection, String httpMethod) throws IOException {
        ...
        if (!"POST".equals(httpMethod) && !"PUT".equals(httpMethod) && !"PATCH".equals(httpMethod) && !"DELETE".equals(httpMethod)) {
            connection.setDoOutput(false);
        } else {
            connection.setDoOutput(true);
        }
        ...
    }
}

// SimpleBufferingClientHttpRequest.class
final class SimpleBufferingClientHttpRequest extends AbstractBufferingClientHttpRequest {
    ...
    protected ClientHttpResponse executeInternal(HttpHeaders headers, byte[] bufferedOutput) throws IOException {
        ...
        if (this.connection.getDoOutput() && this.outputStreaming) {
            this.connection.setFixedLengthStreamingMode(bufferedOutput.length);
        }

        this.connection.connect();
        if (this.connection.getDoOutput()) {
            FileCopyUtils.copy(bufferedOutput, this.connection.getOutputStream());
        } else {
            this.connection.getResponseCode();
        }
        ...
    }
}

Please notice that in the executeInternal() method, the input argument 'bufferedOutput' contains the message body provided by my code. I saw it through the debugger.

However, due to the prepareConnection(), the getDoOutput() in executeInternal() always returns false which, in turn, makes the bufferedOutput completely ignored! It's not copied into the output stream.

Consequently, my server programme received no message body and threw that exception.

This is an example about the RestTemplate of the Spring framework. The point is that, even if the message body is no longer forbidden by the HTTP spec, some client or server libraries or frameworks might still comply with the old spec and reject the message body from the GET request.

Answer 20

Even if a popular tool use this, as cited frequently on this page, I think it is still quite a bad idea, being too exotic, despite not forbidden by the spec.

Many intermediate infrastructures may just reject such requests.

By example, forget about using some of the available CDN in front of your web site, like this one:

If a viewer GET request includes a body, CloudFront returns an HTTP status code 403 (Forbidden) to the viewer.

And yes, your client libraries may also not support emitting such requests, as reported in this comment.

Answer 21

Create a Requestfactory class

import java.net.URI;

import javax.annotation.PostConstruct;

import org.apache.http.client.methods.HttpEntityEnclosingRequestBase;
import org.apache.http.client.methods.HttpUriRequest;
import org.springframework.http.HttpMethod;
import org.springframework.http.client.HttpComponentsClientHttpRequestFactory;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestTemplate;

@Component
public class RequestFactory {
    private RestTemplate restTemplate = new RestTemplate();

    @PostConstruct
    public void init() {
        this.restTemplate.setRequestFactory(new HttpComponentsClientHttpRequestWithBodyFactory());
    }

    private static final class HttpComponentsClientHttpRequestWithBodyFactory extends HttpComponentsClientHttpRequestFactory {
        @Override
        protected HttpUriRequest createHttpUriRequest(HttpMethod httpMethod, URI uri) {
            if (httpMethod == HttpMethod.GET) {
                return new HttpGetRequestWithEntity(uri);
            }
            return super.createHttpUriRequest(httpMethod, uri);
        }
    }

    private static final class HttpGetRequestWithEntity extends HttpEntityEnclosingRequestBase {
        public HttpGetRequestWithEntity(final URI uri) {
            super.setURI(uri);
        }

        @Override
        public String getMethod() {
            return HttpMethod.GET.name();
        }
    }

    public RestTemplate getRestTemplate() {
        return restTemplate;
    }
}

and @Autowired where ever you require and use, Here is one sample code GET request with RequestBody

 @RestController
 @RequestMapping("/v1/API")
public class APIServiceController {
    
    @Autowired
    private RequestFactory requestFactory;
    

    @RequestMapping(method = RequestMethod.GET, path = "/getData")
    public ResponseEntity<APIResponse> getLicenses(@RequestBody APIRequest2 APIRequest){
        APIResponse response = new APIResponse();
        HttpHeaders headers = new HttpHeaders();
        headers.setContentType(MediaType.APPLICATION_JSON);
        Gson gson = new Gson();
        try {
            StringBuilder createPartUrl = new StringBuilder(PART_URL).append(PART_URL2);
            
            HttpEntity<String> entity = new HttpEntity<String>(gson.toJson(APIRequest),headers);
            ResponseEntity<APIResponse> storeViewResponse = requestFactory.getRestTemplate().exchange(createPartUrl.toString(), HttpMethod.GET, entity, APIResponse.class); //.getForObject(createLicenseUrl.toString(), APIResponse.class, entity);
    
            if(storeViewResponse.hasBody()) {
                response = storeViewResponse.getBody();
            }
            return new ResponseEntity<APIResponse>(response, HttpStatus.OK);
        }catch (Exception e) {
            e.printStackTrace();
            return new ResponseEntity<APIResponse>(response, HttpStatus.INTERNAL_SERVER_ERROR);
        }
        
    }
}