Weird indy redirect post

Question

I'm trying to send a post request using Indy but I am facing some problems. In a form I have TIdHTTP, a TIdSSLIOHandlerSocketOpenSSL and a TIdCookieManager whith these properties:

TIdHTTP:

IdHTTP1.IOHandler := FSSLIO;
IdHTTP1.AllowCookies := True;
IdHTTP1.HandleRedirects := True;
IdHTTP1.ProxyParams.BasicAuthentication := False;
IdHTTP1.ProxyParams.ProxyPort := 0;
IdHTTP1.Request.ContentLength := -1;
IdHTTP1.Request.Accept := 'text/html, */*';
IdHTTP1.Request.BasicAuthentication := False;
IdHTTP1.Request.UserAgent := 'Mozilla/3.0 (compatible; Indy Library)';
IdHTTP1.HTTPOptions := [hoKeepOrigProtocol, hoForceEncodeParams];
IdHTTP1.OnRedirect := IdHTTP1Redirect;
IdHTTP1.CookieManager := IdCookieManager1;

TIdSSLIOHandlerSocketOpenSSL: default values

TIdCookieManager: default values

OnRedirect procedure:

Handled := True;

In a button the following request:

Params := TStringStream.Create('asdf=asdf',TEncoding.UTF8);    
edtmemo1.Text := IdHTTP1.Post('https://www.detran.mg.gov.br/habilitacao/1-habilitacao-quero-ser-condutor/consultar-resultado-exame-legislacao/-/busca_resultado_exames/', Params);
Params.Free;

But returns error with response code 301, but the wierd part is that the location is the same url that I am trying to send, so it enter in an infinite loop.

Response

HTTP/1.1 301 Moved Permanently
Date: Thu, 31 Aug 2017 20:23:32 GMT
Server: Apache
X-Powered-By: PHP/5.3.5
P3P: CP="A politica de privacidade deve estar disponivel no site ou pode ser solicitada via fale conosco."
Set-Cookie: SECCCAKEPHP=e19ttp30m5380ih41qal0gipg2; expires=Sat, 09-Sep-2017 04:23:32 GMT; path=/
Location: https://www.detran.mg.gov.br/habilitacao/1-habilitacao-quero-ser-condutor/consultar-resultado-exame-legislacao/-/busca_resultado_exames/
Cache-Control: max-age=604800
Expires: Thu, 07 Sep 2017 20:23:32 GMT
Content-Length: 0
Content-Type: text/html; charset=utf-8

Curl test:

curl -vv -X POST -F 'asdf=asdf' https://www.detran.mg.gov.br/habilitacao/1-habilitacao-quero-ser-condutor/consultar-resultado-exame-legislacao/-/busca_resultado_exames/

I tried the same request using curl and works with response code 200. Any idea of what could be happening?

The server is redirecting to the same URL, but it is also sending a cookie that was likely missing in the initial request. This is reasonable behavior if the server wants that cookie back. If the server gets stuck in an endless redirect loop, that is a different issue. Sniff the traffic to see what curl sends vs what TIdHTTP sends. To sniff curl's HTTPS traffic, you can use Fiddler. To sniff TIdHTTP's HTTPS traffic, you can assign a TIdLog... component, such as TIdLogFile, to the TIdHTTP.Intercept property. Does TIdHTTP send back that cookie? — Remy Lebeau
What command-line parameters are you passing to curl? What Params are you passing to TIdHTTP.Post()? Please provide a minimal reproducible example showing everything you are actually doing to reproduce the issue. Also, Delphi 2010 is 8 years old, are you using an equally old version of Indy? At the time of this writing, the current Indy version is 10.6.2.5434. — Remy Lebeau
I edited the question, added the curl request, and with those parameters curl doesn't handle redirects, but responds with a 200 code. The only way I could simulate the same problem in curl is if instead of https I use http. And yes, I know that Delphi 2010 is old and yes I use version 10.5.5 of Indy, we already tried to update, at least Indy, but we faced a LOT o problems and unfortunately we do not have enough time or people to deal with it. — Rigotti
Your TIdHTTP logic does not match you curl logic. I posted an answer explaining the difference. — Remy Lebeau

Remy Lebeau Remy Lebeau · Accepted Answer · 2017-09-01T18:50:59

If you read the curl documentation, you will see that the -F option posts data in multipart/form-data format:

-F, --form

(HTTP) This lets curl emulate a filled-in form in which a user has pressed the submit button. This causes curl to POST data using the Content-Type multipart/form-data according to RFC 2388. This enables uploading of binary files etc.

However, your TIdHTTP code is posting data that is in application/x-www-form-urlencoded format instead. And more importantly, you are not setting the TIdHTTP.Request.ContentType property to 'application/x-www-form-urlencoded' to match the data. If you post a TStream, you have to set the ContentType accordingly.

TIdHTTP prefers application/x-www-form-urlencoded data to be posted using a TStrings-derived object (like TStringList) instead of a TStream, so it can ensure the data is encoded and formatted correctly, eg:

var
  Params: TStringList;
begin
  ...
  Params := TStringList.Create;
  try
    Params.Add('asdf=asdf'); // <-- DO NOT url-encode the values here!
    // the TStrings version of Post() will set the Request.ContentType
    // to 'application/x-www-form-urlencoded' by default...
    edtmemo1.Text := IdHTTP1.Post('https://www.detran.mg.gov.br/habilitacao/1-habilitacao-quero-ser-condutor/consultar-resultado-exame-legislacao/-/busca_resultado_exames/', Params);
  finally
    Params.Free;
  end;
  ...
end;

TIdHTTP prefers multipart/form-data data to be posted using a TIdMultipartFormDataStream object instead (unless you pass it another TStream that contains pre-formatted MIME data in it).

So, to match what your curl command is sending, try this instead:

uses
  ..., IdMultipartFormData;

var
  Params: TIdMultipartFormDataStream;
begin
  ...
  Params := TIdMultipartFormDataStream.Create;
  try
    Params.AddFormField('asdf', 'asdf', 'utf-8');
    // the TIdMultipartFormDataStream version of Post() will set the
    // Request.ContentType to 'multipart/form-data' with a suitable
    // MIME 'boundary' attribute for you...
    edtmemo1.Text := IdHTTP1.Post('https://www.detran.mg.gov.br/habilitacao/1-habilitacao-quero-ser-condutor/consultar-resultado-exame-legislacao/-/busca_resultado_exames/', Params);
  finally
    Params.Free;
  end;
  ...
end;

Weird indy redirect post

1 Answers