Why would large POST requests not reach my Azure web role?

2
votes

In my Azure web role there's something wrong with handling POST requests larger than about 30 million bytes. I'm aware that there's IIS filter that detects all requests larger than 30 million bytes and sets their results to 404. Yet I expect that these 404 codes will show up in IIS logs.

There's a user of my web role who faces the following behavior: POST requests to a specific URL in my web role which are smaller than about 30 million bytes come through okay (and I see them in IIS logs) but POST requests larger than that (to the very same URL) yield HTTP 503 (Service Unavailable) code for the client and I don't see those requests in IIS logs.

Here're the headers of a sample long POST

POST /mySpecificUrl?paramshere HTTP/1.1
User-Agent: Custom USer Agent
Content-Type: application/octet-stream
Proxy-Authorization: NTLM LongBase64StringHere
Host: my.hostname.here
Content-Length: content length here - about 32 megabytes
Expect: 100-continue

It looks like something is intercepting those requests - perhaps the load balancer or something and so they don't reach my role.

Now the user uses Fiddler and he claims that when he gets code 503 the following content is being returned:

Hello,
A communication error occurred: ""
The Web Server may be down, too busy, or experiencing other
problems preventing it from responding to requests.
You may wish to try again at a later time.

crafted as HTML. The headers that come along with this content don't contain a custom header that we inject into every response so it's yet another evidence that the request doesn't reach IIS.

How do I find out what causes this behavior and how do I control it (for example, change the request size threshold)?

2
iisazureazure-web-roles
Try to fake the Content-Length header and see if an entry is being added to the failed requests logs. – haim770
Are you posting this content Asynchrounous? – Thiago Custodio
@ThiagoCustodio: That's the other user code, but he uses our sample where the POST is synchronous so I'd guess that unless he tweaked it with a baseball bate it is synchronous. – sharptooth
@sharptooth Could you post your sample post in here? – Thiago Custodio
@ThiagoCustodio: I've added the headers. – sharptooth

2 Answers

1
votes

I'm totally sure that there's something blocking the request to reach IIS. Probably Firewall rules or Load Balancer, that's why you could not see the logs in IIS. The bad news, is that I could not find a way to tell that you'll post large content.

Try to apply this configs to the "ServiceDefinition.csdef" file and see if it will help in something:

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="WindowsAzure1" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition" schemaVersion="2013-03.2.0">
  <LoadBalancerProbes>
    <LoadBalancerProbe protocol="http" name="config" timeoutInSeconds="36000"></LoadBalancerProbe>
  </LoadBalancerProbes>
  <WebRole name="webRoleConfig">
    <Runtime executionContext="elevated"></Runtime>
  </WebRole>  
</ServiceDefinition>
0
votes

After much effort put into the investigation and a support request the conclusion is that there's some software outside Azure that emits that message perhaps some proxy server or something.

Our service was configured with <system.webServer><security><requestFiltering><requestLimits maxAllowedContentLength not matching <system.web><httpRuntime maxRequestLength> and this is known to cause IIS return HTTP 500 and close the connection for requests with lengths in certain range. It's just a guess but perhaps that other software would intercept the HTTP 500 response and emit the "Hello," message.

I asked this question on ServerFault to find which software is known to emit this message.