Not generating a complete response from a HttpWebResponse object in C#

Question

I am creating a HttpWebRequest object from another aspx page to save the response stream to my data store. The Url I am using to create the HttpWebRequest object has querystring to render the correct output. When I browse to the page using any old browser it renders correctly. When I try to retrieve the output stream using the HttpWebResponse.GetResponseStream() it renders my built in error check.

Why would it render correctly in the browser, but not using the HttpWebRequest and HttpWebResponse objects?

Here is the source code:

Code behind of target page:

protected void PageLoad(object sender, EventsArgs e)
{
   string output = string.Empty;

   if(Request.Querystring["a"] != null)
   {
      //generate output
      output = "The query string value is " + Request.QueryString["a"].ToString();
   }
   else
   {
      //generate message indicating the query string variable is missing
      output = "The query string value was not found";
   }

   Response.Write(output);
}

Code behind of page creating HttpWebRequest object

string url = "http://www.mysite.com/mypage.aspx?a=1";
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(url)

//this if statement was missing from original example
if(User.Length > 0)
{
    request.Credentials = new NetworkCredentials("myaccount", "mypassword", "mydomain");
    request.PreAuthenticate = true;
}

request.UserAgent = Request.UserAgent;
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
Stream resStream = response.GetResponseStream();  
Encoding encode = System.Text.Encoding.GetEncoding("utf-8");
StreamReader readStream = new StreamReader(resStream, encode, true, 2000);
int count = readStream.Read(read, 0, read.Length);
string str = Server.HtmlEncode(" ");

while (count > 0)
{
    // Dumps the 256 characters on a string and displays the string to the console.
    string strRead = new string(read, 0, count);
    str = str.Replace(str, str + Server.HtmlEncode(strRead.ToString()));
    count = readStream.Read(read, 0, 256);
}

// return what was found
result = str.ToString();

resStream.Close();
readStream.Close();

Update

@David McEwing - I am creating the HttpWebRequest with the full page name. The page is still generating the error output. I updated the code sample of the target page to demonstrate exactly what I am doing.

@Chris Lively - I am not redirecting to an error page, I generate a message indicating the query string value was not found. I updated the source code example.

Update 1:

I tried using Fiddler to trace the HttpWebRequest and it did not show up in the Web Sessions history window. Am I missing something in my source code to get a complete web request and response.

Update 2:

I did not include the following section of code in my example and it was culprit causing the issue. I was setting the Credentials property of the HttpWebRequest with a sevice account instead of my AD account which was causing the issue.

I updated my source code example

I just copied these two code snippets into a test site, the first going into default.aspx the other going into a page test.aspx that calls default.aspx. It works for me under both IIS6 and the ASP.NET development server. Are you pointing to the correct version of your test? — David McEwing
I found this problem. The credentials I was using causing the page to not render correctly. Update my question and posted an answer. — Michael Kniskern

David McEwing David McEwing · Accepted Answer · 2009-05-05T00:18:29

What webserver are you using? I can remember at one point in my past when doing something with IIS there was an issue where the redirect between http://example.com/ and http://example.com/default.asp dropped the query string.

Perhaps run Fiddler (or a protocol sniffer) and see if there is something happening that you aren't expecting.

Also check if passing in the full page name works. If it does the above is almost certainly the problem.

Not generating a complete response from a HttpWebResponse object in C#

3 Answers