Decode content-encoding gzip using Spring WebClient

Question

I am calling a web service using Spring WebClient (Spring 5.1.3). The service responds with content-type: application/json and content-encoding: gzip

ClientResponse.bodyToMono then fails with the error "JSON decoding error: Illegal character ((CTRL-CHAR, code 31))" which I assume is because the content has not been decoded before trying to parse the JSON.

Here is code snippet (simplified) of how I create the WebClient

HttpClient httpClient = HttpClient.create().secure(sslContextSpec -> sslContextSpec.sslContext(sslContext));
return WebClient.builder().clientConnector(new ReactorClientHttpConnector(httpClient)).build();

I then use the WebClient to make the call:

webClient.get().uri(uri)
    .accept(MediaType.APPLICATION_JSON)
    .header(HttpHeaders.ACCEPT_ENCODING, "gzip")
    .exchange()

The HTTP request has 2 headers:

Accept: application/json
Accept-Encoding: gzip

The response has the following headers:

set-cookie: xxx
content-type: application/json; charset=utf-8
content-length: 1175
content-encoding: gzip
cache-control: no-store, no-cache

By doing the following I am able to manually decode the GZIP content and get valid JSON from the result

webClient.get().uri(uri)
        .accept(MediaType.APPLICATION_JSON)
        .header("accept-encoding", "gzip")
        .exchange()
        .flatMap(encodedResponse -> encodedResponse.body((inputMessage, context) ->
                inputMessage.getBody().flatMap(dataBuffer -> {
                    ClientResponse.Builder decodedResponse = ClientResponse.from(encodedResponse);
                    try {
                        GZIPInputStream gz = new GZIPInputStream(dataBuffer.asInputStream());
                        decodedResponse.body(new String(gz.readAllBytes()));
                    } catch (IOException e) {
                        e.printStackTrace();
                    }
                    decodedResponse.headers(headers -> {
                        headers.remove("content-encoding");
                    });
                    return Mono.just(decodedResponse.build());
                }).flatMap(clientResponse -> clientResponse.bodyToMono(Map.class))

gzip decoding is supported by default by the reactor netty client. Could you post code snippets showing how the webclient is being created? Could you show as well the HTTP request and response headers and bodies? — Brian Clozel
@BrianClozel I have added some code snippets. I have now been able to manually decode the content successfully, but it still does not seem to handle this automatically — athom

Brian Clozel Brian Clozel · Accepted Answer · 2018-12-28T11:43:52

This feature is supported natively by the reactor netty client.

You should create HttpClient like this:

HttpClient httpClient = HttpClient.create()
             .secure(sslContextSpec -> sslContextSpec.sslContext(sslContext))
             .compress(true);

And then there's no need to add the accept encoding request header since it's done for you.

Note that this bit is done by the connector itself when you don't provide a custom HttpClient instance.

Decode content-encoding gzip using Spring WebClient

1 Answers