How to determine authentication requirements given 401 response?

Question

I'm using the following code on the client side to take response of a protected website, which works pretty fine whenever there is no authentication request from server side:

    final String targetURL   = "http://************";
    final String username    = "********";
    final String password    = "******";
    final int connectTimeout = 30000;
    
    final String username_password         = username + ":" + password;
    final byte[] usernamepassword          = (byte[])username_password.getBytes();
    final String encoded_username_password = java.util.Base64.getEncoder().encodeToString(usernamepassword);
    
    final URL url = new URL(targetURL);
    final URLConnection connection = url.openConnection();
    
    connection.setConnectTimeout(connectTimeout);
    connection.setRequestProperty("Authorization", "Basic " + encoded_username_password);
    connection.setRequestProperty("Accept", "text/html");
      
    // response headers
    final Map<String, List<String>> headerFields = connection.getHeaderFields();
    
    for (final Map.Entry<String, List<String>> entry : headerFields.entrySet()) {
        System.out.println("Key: " + entry.getKey() + ", Value: " + entry.getValue());
    }

However, when I try to connect with a specific (protected) webserver, the connection is refused with the 401 http status code.

Key: null, Value: [HTTP/1.1 401 Unauthorized]
Key: WWW-Authenticate, Value: [Digest realm="Login to ********", qop="auth", nonce="*******", opaque=""]
Key: Connection, Value: [close] Key: Content-Length, Value: [0]

Is it possible to infer from the above logged response what I'm missing ?

I mean, the snippet above is a well known Java basic-authentication template, there are many examples everywhere with the same structure, so the issue seems likely related to the parameters issued as argument of the setRequestProperty() method.

This is what I took on request header by using Mozilla >> Web Developer >> Debug (where some fields are hidden by me):

Host: xxx.xxx.xxx.xxx

User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:76.0) Gecko/20100101 Firefox/76.0

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8

Accept-Language: pt-BR,pt;q=0.8,en-US;q=0.5,en;q=0.3

Accept-Encoding: gzip, deflate

Connection: keep-alive

Upgrade-Insecure-Requests: 1

Pragma: no-cache

Cache-Control: no-cache

Authorization: Digest username="", realm="Login to xxxxxxx", nonce="xxx-xxx-xxx-xxx-xxx", uri="xxxxxxx", response="xxxxx", qop=auth, nc=00000001, cnonce="b2b43c8d9631354d"

Perhaps I was not clear on this point, I'm on the client side, so it is not a matter of filtering requests, I'm the one who is issuing requests to the webserver. — andre castro

Nesan Mano Nesan Mano · Accepted Answer · 2020-05-16T19:07:08

I am not sure to understand your question. My guess is you would like to filter the request based on the parameters and url so you can chhose to either serve content or reject and return 401.

In this case your question should be: How to filter which http requests requires authentication.

Depending on what framework your are using, in spring-boot, there is something called WebSecurityConfigurerAdapter. It basically helps you with your problem. You can filter all the requests coming to your server.

How to determine authentication requirements given 401 response?

2 Answers