502 error when redirecting stream from another site

Question

In my WEB Api 2 controller I want to request file from one site and return this file from my controller. Here is the code

public HttpResponseMessage GetLecture()
{            
    HttpWebRequest request = WebRequest.CreateHttp("http://openmedia.yale.edu/cgi-bin/open_yale/media_downloader.cgi?file=/courses/spring11/phil181/mp3/phil181_01_011111.mp3");
    request.Referer = @"http://oyc.yale.edu/courses/";

    var receivedResponse = request.GetResponse();

    HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);                    
    response.Content = new StreamContent(receivedResponse.GetResponseStream());
    response.Content.Headers.ContentType = MediaTypeHeaderValue.Parse(receivedResponse.ContentType);
    response.Content.Headers.ContentDisposition = new ContentDispositionHeaderValue("attachment");
    response.Content.Headers.ContentDisposition.FileName = "phil181_01_011111.mp3";
    response.Content.Headers.ContentLength = receivedResponse.ContentLength;

    return response;
}

Locally it works fine and I can download the file but when I deploy it to Azure I'm getting 502 Error. Web server received an invalid response while acting as a gateway or proxy server.

Logging shows that it fails after returning response so no exceptions during method execution.

It's ~50MB file. For smaller files code works fine.

How can I make this code works on Azure for 50 MB files?

What if you read the whole response (for testing purposes) without streaming it ditectly? Maybe reading response while hosting in azure just fails for some reason? — Evk
It might be because request is taking long time. Check this Try by downloading small file first. — Shah
@Shah It doesn't look like request is taking long time. Whole method takes less then a second to execute. — Denis Palnitsky
I tried to save it to temp file. It saves stream and when I try to return stream from file it fails same way. — Denis Palnitsky
Based on your code, I could return the image stream from my azure blob storage to the client when deployed to my azure web app. Does your web app fail all time? Have you tried Remote debugging web apps? — Bruce Chen

ThePretendProgrammer ThePretendProgrammer · Accepted Answer · 2017-11-26T13:40:21

First off, I will recommend configuring remote IIS administration for your Azure Web App since that will allow you direct access to IIS logs and associated settings. I have a strong feeling that the problem has to do with IIS/ASP.NET limits. You already pointed out that the method completes execution within seconds so this is likely not a timeout problem.

You also mentioned that the code works for smaller file sizes but fails when the size is increased. This points towards IIS bufferingLimit for your website being set to a small value (or perhaps the default value of ~4 MB). Here is a link describing how you can increase this limit to the desired value. Hope this works for you. If it doesn't, I would recommend digging into those IIS logs for your website.

502 error when redirecting stream from another site

2 Answers