Google cloud load balancer custom http header is missing

Question

While using Google Cloud HTTPS Load Balancer we hit the following bug. Couldn't find any information on it.

We have a custom http header in our request: X-<Company name>-abcde. If we are working directly against the server all is good, but once we are working through the load balancer, than our custom header is missing. We didn't find any reference in the documentation that there is a need to white list our headers or something like that.

Why my custom header is not being transferred to my backend server while working through Google Cloud Load Balancer? And how to make it work?

Thanks

Answer 1

Data

After a lot of testing, these are the results I've come up with:

1. The Google Cloud HTTPS Load Balancer does transfer custom HTTP headers to the backend service.
2. However, it changes them to lower-case.

So, in your case, X-Custom-Header is transformed to x-custom-header.

Solution

1. Simply change your code to read the lower-case version of your custom HTTP header. This is a simple fix, but one which may not be supported in the long-term by Google (there's not a word on this in Google's documentation so it's subject to change with no notice).
2. Petition Google to change this idiosyncratic behaviour or at the very least mention it clearly in their documentation.

A little extra

As far as I know, the RFC 2047 which specified the X- prefix for custom HTTP headers and propagated the pseudo-standard of a capital letter for each word has been deprecated and replaced by RFC 6648 which recommends against the X- prefix in general and mentions nothing regarding the rest of the words in the custom HTTP header key name. If I were Google, I would change this behaviour to pass custom HTTP headers as is and let developers deal with the strings as they've set them.

Answer 2

The RFC (RFC 7230) for HTTP/1.1 Message Syntax and Routing says that header fields have a case-insensitive field name. If you're relying on case to match the header that doesn't align with the RFC.

Way back in the day I looked through either the Tomcat of Jetty source and they worked with everything as a .toLower().

Go has a CanonicalMIMEHeaderKey where it'll format the headers in a common way to be sure everything is on the same page.

Python still harkens back to the RFC822 (hg.python.org/cpython/file/2.7/Lib/rfc822.py#l211) days, but it forces a .lower() on headers to standardize.

Basically though what the GCP HTTP(S) Load Balancer is doing is acceptable as far as the RFC is concerned.

Answer 3

This is most likely an application bug.

As other answers have stated, HTTP header names are case insensitive. Ime, every time headers appear to be case sensitive, it is because there is a request wrapper somewhere in the application call stack.

Request wrappers like this are common (usually necessary) in Java servlet filters. It's a common, newbie mistake to use case-sensitive matching (e.g. a regular Java HashMap<String, T>()) for the header names in the wrapper.

That's where I would start looking for your bug.

A reasonable way to create a Java Map<String, T> that is both case insensitive and that doesn't modify the keys is to use new TreeMap<String, T>( String.CASE_INSENSITIVE_ORDER ).