Authorization failure TIdHTTP over HTTPS

Question

I want to approach the Exchange webservice and handle XML SOAP composition (request) and parsing (response) myself. Therefore, THTPPRIO seems a bit overkill.

I'm trying TIdHTTP but I'm stuck on the authentication; using Delphi XE2 update 4 with Indy 10.5.8.0

Here's the code:

idHTTP1.Request.CustomHeaders.AddValue('SOAPAction','"http://schemas.microsoft.com/exchange/services/2006/messages/ResolveNames"');
IdHTTP1.Post('https://webmail.mailserver.nl/ews/exchange.asmx',TSRequest,TSResponse);

TSRequest,TSResponse are UTF-8 TStringStreams, TSRequest contains the entire SOAP envelope
IdHTTP1.IOHandler is set to a TIdSSLIOHandlerSocketOpenSSL, per this advice TIdSSLIOHandlerSocketOpenSSL.Intercept is linked to a TIdLogDebug so that I can debug what's going on
OpenSSL DLLs are present in the DLL search path
EWS wants NTLM validation; TIdNTLMAuthentication is in the uses clause; I have set idHTTP1.Request.BasicAuthentication=false, Username and Password filled in
The IdHTTP1.OnSelectAuthorization event confirms the NTLM auth (parameters AuthenticationClass = TIdSSPINTLMAuthentication, and the AuthInfo TIdHeaderList contains'Negotiate', 'NTLM')
There is no proxy. I proxied through Fiddler to see what's going on, but that makes no difference.

I also tried setting user/PW run-time:

procedure TForm1.IdHTTP1Authorization(Sender: TObject; Authentication: TIdAuthentication; var Handled: Boolean);
begin
  Authentication.Username := '[email protected]';
  Authentication.Password := 'password';
  Handled := true;
end;

Sent HTTP (from TIdLogDebug.OnSend) starts with:

POST /ews/exchange.asmx HTTP/1.1
Content-Type: text/xml; charset=utf-8
Content-Length: 562
SOAPAction: "http://schemas.microsoft.com/exchange/services/2006/messages/ResolveNames"
Host: webmail.mailserver.nl
Accept: text/html, */*
Accept-Encoding: gzip,deflate, identity
User-Agent: Mozilla/3.0 (compatible; Indy Library)


<soapenv:Envelope

Received HTTP (from TIdLogDebug.OnReceive) is

HTTP/1.1 401 Unauthorized
Cache-Control: private
Server: Microsoft-IIS/7.5
X-AspNet-Version: 2.0.50727
Set-Cookie: exchangecookie=0157734634ba4a0fa3a7d0d8efb602f2; expires=Tue, 12-Nov-2013 13:38:56 GMT; path=/; HttpOnly
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
X-Powered-By: ASP.NET
Date: Mon, 12 Nov 2012 13:38:56 GMT
Content-Length: 0

The TIdSSLIOHandlerSocketOpenSSL.OnStatusInfo event logs

SSL status: "before/connect initialization"
SSL status: "before/connect initialization"
SSL status: "SSLv3 write client hello A"
SSL status: "SSLv3 read server hello A"
SSL status: "SSLv3 read server certificate A"
SSL status: "SSLv3 read server done A"
SSL status: "SSLv3 write client key exchange A"
SSL status: "SSLv3 write change cipher spec A"
SSL status: "SSLv3 write finished A"
SSL status: "SSLv3 flush data"
SSL status: "SSLv3 read finished A"
SSL status: "SSL negotiation finished successfully"
SSL status: "SSL negotiation finished successfully"
Cipher: name = AES128-SHA; description = AES128-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA1
; bits = 128; version = TLSv1/SSLv3;

What I miss in the outgoing HTTP is a line like (as e.g. generated by SOAPUI):

"Authorization: NTLM TlRMTVNTUAABAAAANQIIIBQAFAAyAAAAEgASACAAAABWAE0ASgBBAE4AVABUADcANABUAEkATQBFAFQARQBMAEwAQgBWAA==[\r][\n]"

Maybe I don't specify username/PW in the correct place?

Added 13 Nov after Remy's initial answer:

For comparison, I called the webservice from SOAPUI and this shows 6 packets going over the line, with twice a response UnAuthorized.

>> "POST /ews/exchange.asmx HTTP/1.1[\r][\n]"
>> "Accept-Encoding: gzip,deflate[\r][\n]"
>> "SOAPAction: "http://schemas.microsoft.com/exchange/services/2006/messages/ResolveNames"[\r][\n]"
>> "Content-Type: text/xml; charset=utf-8[\r][\n]"
>> "Content-Length: 548[\r][\n]"
>> "Host: webmail.timetellbv.nl[\r][\n]"
>> "Connection: Keep-Alive[\r][\n]"
>> "User-Agent: Apache-HttpClient/4.1.1 (java 1.5)[\r][\n]"
>> "[\r][\n]"
>> "<soapenv:Envelope [\n]"
[snip]
>> "</soapenv:Envelope>[\n]"
>> "[\n]"

<< "HTTP/1.1 401 Unauthorized[\r][\n]"
<< "Cache-Control: private[\r][\n]"
<< "Server: Microsoft-IIS/7.5[\r][\n]"
<< "X-AspNet-Version: 2.0.50727[\r][\n]"
<< "Set-Cookie: exchangecookie=a29f10ca2a6d484ea276737e87d8e733; expires=Wed, 13-Nov-2013 10:47:33 GMT; path=/; HttpOnly[\r][\n]"
<< "WWW-Authenticate: Negotiate[\r][\n]"
<< "WWW-Authenticate: NTLM[\r][\n]"
<< "X-Powered-By: ASP.NET[\r][\n]"
<< "Date: Tue, 13 Nov 2012 10:47:33 GMT[\r][\n]"
<< "Content-Length: 0[\r][\n]"
<< "[\r][\n]"

>> "POST /ews/exchange.asmx HTTP/1.1[\r][\n]"
>> "Accept-Encoding: gzip,deflate[\r][\n]"
>> "SOAPAction: "http://schemas.microsoft.com/exchange/services/2006/messages/ResolveNames"[\r][\n]"
>> "Content-Type: text/xml; charset=utf-8[\r][\n]"
>> "Content-Length: 548[\r][\n]"
>> "Host: webmail.timetellbv.nl[\r][\n]"
>> "Connection: Keep-Alive[\r][\n]"
>> "User-Agent: Apache-HttpClient/4.1.1 (java 1.5)[\r][\n]"
>> "Cookie: exchangecookie=a29f10ca2a6d484ea276737e87d8e733[\r][\n]"
>> "Cookie2: $Version=1[\r][\n]"
>> "Authorization: NTLM TlRMTVNTUAABAAAANQIIIBQAFAAyAAAAEgASACAAAABWAE0ASgBBAE4AVABUADcANABUAEkATQBFAFQARQBMAEwAQgBWAA==[\r][\n]"
>> "[\r][\n]"
>> "<soapenv:Envelope [\n]"
[snip]
>> "</soapenv:Envelope>[\n]"
>> "[\n]"

<< "HTTP/1.1 401 Unauthorized[\r][\n]"
<< "Server: Microsoft-IIS/7.5[\r][\n]"
<< "WWW-Authenticate: NTLM TlRMTVNTUAACAAAAFAAUADgAAAA1AokilFuJDu09j+sAAAAAAAAAAMAAwABMAAAABgGxHQAAAA9UAEkATQBFAFQARQBMAEwAQgBWAAIAFABUAEkATQBFAFQARQBMAEwAQgBWAAEAHABUAFQARQBYAEMASABBAE4ARwBFADIAMAAxADAABAAaAHQAaQBtAGUAdABlAGwAbABiAHYALgBuAGwAAwA4AFQAVABFAFgAQwBIAEEATgBHAEUAMgAwADEAMAAuAHQAaQBtAGUAdABlAGwAbABiAHYALgBuAGwABQAaAHQAaQBtAGUAdABlAGwAbABiAHYALgBuAGwABwAIADs8RkmMwc0BAAAAAA==[\r][\n]"
<< "WWW-Authenticate: Negotiate[\r][\n]"
<< "X-Powered-By: ASP.NET[\r][\n]"
<< "Date: Tue, 13 Nov 2012 10:47:33 GMT[\r][\n]"
<< "Content-Length: 0[\r][\n]"
<< "[\r][\n]"

>> "POST /ews/exchange.asmx HTTP/1.1[\r][\n]"
>> "Accept-Encoding: gzip,deflate[\r][\n]"
>> "SOAPAction: "http://schemas.microsoft.com/exchange/services/2006/messages/ResolveNames"[\r][\n]"
>> "Content-Type: text/xml; charset=utf-8[\r][\n]"
>> "Content-Length: 548[\r][\n]"
>> "Host: webmail.timetellbv.nl[\r][\n]"
>> "Connection: Keep-Alive[\r][\n]"
>> "User-Agent: Apache-HttpClient/4.1.1 (java 1.5)[\r][\n]"
>> "Cookie: exchangecookie=a29f10ca2a6d484ea276737e87d8e733[\r][\n]"
>> "Cookie2: $Version=1[\r][\n]"
>> "Authorization: NTLM TlRMTVNTUAADAAAAGAAYAEAAAADsAOwAWAAAABQAFABEAQAAEgASAFgBAAASABIAagEAAAAAAAB8AQAANQIIIFEqBrpeBXvnS3dcDcbKGMQS3VgaBa9Bi9YvouCOFFWFjH84AhcR7fgBAQAAAAAAAECzkUmMwc0BEt1YGgWvQYsAAAAAAgAUAFQASQBNAEUAVABFAEwATABCAFYAAQAcAFQAVABFAFgAQwBIAEEATgBHAEUAMgAwADEAMAAEABoAdABpAG0AZQB0AGUAbABsAGIAdgAuAG4AbAADADgAVABUAEUAWABDAEgAQQBOAEcARQAyADAAMQAwAC4AdABpAG0AZQB0AGUAbABsAGIAdgAuAG4AbAAFABoAdABpAG0AZQB0AGUAbABsAGIAdgAuAG4AbAAHAAgAOzxGSYzBzQEAAAAAVABJAE0ARQBUAEUATABMAEIAVgBkAGUAdgBlAGwAbwBwAGUAcgBWAE0ASgBBAE4AVABUADcANAA=[\r][\n]"
>> "[\r][\n]"
>> "<soapenv:Envelope [\n]"
[snip]
>> "</soapenv:Envelope>[\n]"
>> "[\n]"

<< "HTTP/1.1 200 OK[\r][\n]"
<< "Cache-Control: private[\r][\n]"
<< "Transfer-Encoding: chunked[\r][\n]"
<< "Content-Type: text/xml; charset=utf-8[\r][\n]"
<< "Server: Microsoft-IIS/7.5[\r][\n]"
<< "X-EwsPerformanceData: RpcC=2;RpcL=0;LdapC=1;LdapL=0;[\r][\n]"
<< "X-AspNet-Version: 2.0.50727[\r][\n]"
<< "Persistent-Auth: true[\r][\n]"
<< "X-Powered-By: ASP.NET[\r][\n]"
<< "Date: Tue, 13 Nov 2012 10:47:33 GMT[\r][\n]"
<< "[\r][\n]"
<< "877[\r][\n]"
<< "<?xml version="1.0" encoding="utf-8"?><s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
[snip]
<< "</s:Envelope>"
<< "[\r][\n]"
<< "0[\r][\n]"
<< "[\r][\n]"

So from Delphi I only see the first 2 packets exchanging. Strange thing is, if I click my 'Test' button again the exchange seems to continue???:

Request:

POST /ews/exchange.asmx HTTP/1.1
Content-Type: text/xml; charset=utf-8
Content-Length: 562
SOAPAction: "http://schemas.microsoft.com/exchange/services/2006/messages/ResolveNames"
Host: webmail.timetellbv.nl
Accept: text/html, */*
Accept-Encoding: gzip,deflate, identity
User-Agent: Mozilla/3.0 (compatible; Indy Library)
Authorization: NTLM TlRMTVNTUAABAAAAB4IIogAAAAAAAAAAAAAAAAAAAAAGAbEdAAAADw==


<soapenv:Envelope 
[snip]
</soapenv:Envelope>

Response:

HTTP/1.1 401 Unauthorized
Server: Microsoft-IIS/7.5
WWW-Authenticate: NTLM TlRMTVNTUAACAAAAFAAUADgAAAAFgomiqrTrZnWjEdQAAAAAAAAAAMAAwABMAAAABgGxHQAAAA9UAEkATQBFAFQARQBMAEwAQgBWAAIAFABUAEkATQBFAFQARQBMAEwAQgBWAAEAHABUAFQARQBYAEMASABBAE4ARwBFADIAMAAxADAABAAaAHQAaQBtAGUAdABlAGwAbABiAHYALgBuAGwAAwA4AFQAVABFAFgAQwBIAEEATgBHAEUAMgAwADEAMAAuAHQAaQBtAGUAdABlAGwAbABiAHYALgBuAGwABQAaAHQAaQBtAGUAdABlAGwAbABiAHYALgBuAGwABwAIAOZ26g+Owc0BAAAAAA==
Set-Cookie: exchangecookie=0c8362d303d742c6aae98bd1df574a4d; expires=Wed, 13-Nov-2013 11:00:16 GMT; path=/; HttpOnly
WWW-Authenticate: Negotiate
X-Powered-By: ASP.NET
Date: Tue, 13 Nov 2012 11:00:15 GMT
Content-Length: 0

And if I click my Test button a third time I get an actual EIdHTTPProtocolException and this data exchange:

Request:

POST /ews/exchange.asmx HTTP/1.1
Content-Type: text/xml; charset=utf-8
Content-Length: 562
SOAPAction: "http://schemas.microsoft.com/exchange/services/2006/messages/ResolveNames"
Host: webmail.timetellbv.nl
Accept: text/html, */*
Accept-Encoding: gzip,deflate, identity
User-Agent: Mozilla/3.0 (compatible; Indy Library)
Authorization: NTLM TlRMTVNTUAADAAAAGAAYAJ4AAABIAUgBtgAAABIAEgBYAAAAIgAiAGoAAAASABIAjAAAAAAAAAD+AQAABYKIogYBsR0AAAAPOcYXUTHWwFnGL17GZCkaYFYATQBKAEEATgBUAFQANwA0AGIAbwBiAEAAdABpAG0AZQB0AGUAbABsAGIAdgAuAG4AbABWAE0ASgBBAE4AVABUADcANAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACEi/nFDXHEgQ6OUgBv7Zw0AQEAAAAAAACsq9CKkcHNAbBRNkiUgQdnAAAAAAIAFABUAEkATQBFAFQARQBMAEwAQgBWAAEAHABUAFQARQBYAEMASABBAE4ARwBFADIAMAAxADAABAAaAHQAaQBtAGUAdABlAGwAbABiAHYALgBuAGwAAwA4AFQAVABFAFgAQwBIAEEATgBHAEUAMgAwADEAMAAuAHQAaQBtAGUAdABlAGwAbABiAHYALgBuAGwABQAaAHQAaQBtAGUAdABlAGwAbABiAHYALgBuAGwABwAIAKyr0IqRwc0BBgAEAAIAAAAIADAAMAAAAAAAAAAAAAAAADAAALSZIBVpzBPWjPvSVUels19vMlDT5yE5Q8qQ4mwV87EeCgAQAAAAAAAAAAAAAAAAAAAAAAAJAAAAAAAAAAAAAAAAAAAA


<soapenv:Envelope 
[snip]
</soapenv:Envelope>

Response:

HTTP/1.1 401 Unauthorized
Server: Microsoft-IIS/7.5
Set-Cookie: exchangecookie=2a4876f8adeb425384fb370cafa61ee6; expires=Wed, 13-Nov-2013 11:25:11 GMT; path=/; HttpOnly
WWW-Authenticate: Negotiate
WWW-Authenticate: NTLM
X-Powered-By: ASP.NET
Date: Tue, 13 Nov 2012 11:25:11 GMT
Content-Length: 0

Thanks
Jan

I have abandoned this approach in favor of using THTTPReqResp - that is much easier. See stackoverflow.com/questions/13345540/… Still curious for the solution though... — Jan Doggen

Remy Lebeau Remy Lebeau · Accepted Answer · 2012-11-12T19:41:17

Typically an Authorization header is not sent until the server asks for it via a 401 reply. You won't see it on the initial request, but TIdHTTP should be sending new requests with an Authorization header after processing the 401 reply. In any case, you should be using the TIdHTTP.Request.Username and TIdHTTP.Request.Password properties to set the initial credentials, then use the OnAuthorization event to provide new credentials as needed.

Authorization failure TIdHTTP over HTTPS

1 Answers