Apache HttpClientBuilder.create().setUserAgent(...) not working

Question

Here is how I am creating HttpClient :

private String userAgent= "Non empty user agent";
private HttpClient httpClient= HttpClientBuilder.create()
    .setDefaultCookieStore(new BasicCookieStore())
    .setUserAgent(userAgent)
    .setDefaultRequestConfig(RequestConfig.custom()
        .setCookieSpec(CookieSpecs.STANDARD)
        .build())
    .setDefaultHeaders(Arrays.asList(
        new BasicHeader("Accept-Encoding", "gzip"),
        new BasicHeader("Connection", "keep-alive")))
    .build();

But when sending some request, the USER AGENT is empty, here is the log :

12:51:33.807 [http-nio-5010-exec-9] DEBUG org.apache.http.headers - http-outgoing-0 >> GET / HTTP/1.1
12:51:33.807 [http-nio-5010-exec-9] DEBUG org.apache.http.headers - http-outgoing-0 >> Accept- Encoding: gzip
12:51:33.807 [http-nio-5010-exec-9] DEBUG org.apache.http.headers - http-outgoing-0 >> Host: www.somehost.com
12:51:33.807 [http-nio-5010-exec-9] DEBUG org.apache.http.headers - http-outgoing-0 >> Connection: keep-alive
12:51:33.807 [http-nio-5010-exec-9] DEBUG org.apache.http.headers - http-outgoing-0 >> User-Agent:

I tried to set the user Agent as default header like this :

.setDefaultRequestConfig(RequestConfig.custom()
    .setCookieSpec(CookieSpecs.STANDARD).build())
.setDefaultHeaders(Arrays.asList(
    new BasicHeader("Accept-Encoding", "gzip"),
    new BasicHeader("Connection", "keep-alive"),
    new BasicHeader("User-Agent", userAgent)))
.build();

but same result (user agent is empty).

Here is how I am sending request :

HttpResponse httpResponse = httpClient.execute(new HttpGet(baseUrl));

maven pom dependency for Http Client :

<dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.5.10</version>
</dependency>

Did you try to set the header normally with .setHeader("User-Agent", userAgent) instead of .setDefaultHeaders() just like described in stackoverflow.com/a/5027588/3329810 ? — malliaridis
I didn't, if I use .setHeader I have to do it for each request I perform, it is risky, I can forget to do ir for some request, .setDefaultHeaders on httpClientBuilder is meant to call .setHeader for each request performed using the httpClient — younes zeboudj

younes zeboudj younes zeboudj · Accepted Answer · 2019-12-07T22:31:45

The problem was in instantiation order : in fact I am using SpringBoot, and the user agent value is loaded from properties file, here is it :

 @Value("${com.myapplication.userAgent}")
 private String userAgent;
private HttpClient httpClient= HttpClientBuilder.create()
    .setDefaultCookieStore(new BasicCookieStore())
    .setUserAgent(userAgent)
    .setDefaultRequestConfig(RequestConfig.custom().setCookieSpec(CookieSpecs.STANDARD).build())
    .setDefaultHeaders(Arrays.asList(
            new BasicHeader("Accept-Encoding", "gzip"),
            new BasicHeader("Connection", "keep-alive"))
    )
    .build();

The JVM instantiates httpClient before value injection occurs for userAgent, so when instantiating httpClient the userAgent value is null.

Thanks for your mental effort,

Apache HttpClientBuilder.create().setUserAgent(...) not working

1 Answers