idhttp indy post, do request (parameters) with utf-8

Question

I'm trying to post a request, in utf-8 however the server is getting it in Ascii.

Tried the TstringList format of post.

Tried the stream format

Tried forcing the TStringStream to have UTF8 encoding.

Tried updating indy to the xe5 indy

Here is an example code:

var
  server:TIdHttp;
  Parameters,response:TStringStream;
begin
  response := TStringStream.Create;
  Parameters := TStringSTream.create(UTF8String('param1=Value1&param2=عربي/عرب&param3=Value3'),TEncoding.UTF8);
  Server.Post(TIdURI.URLEncode('http://www.example.com/page.php'),Parameters,response);
end;

Now the Arab coding is passed as Ascii in network sniffer.

0060 d8 b9 d8 b1 d8 a8 d9 8a 2f d8 b9 d8 b1 d8 a8 26 ........ /......&

How can I force Indy Http id to pass the request parameters in Utf-8 and not in Ascii?

On what did you base the conclusion that it's ascii? The number of bytes and the pattern seems to suggest that it contains UTF8 indeed. It's just the sniffer that ignores the encoding and displays raw bytes (the hex dump) and tries to display them as ascii. — GolezTrol
@GolezTrol beacuse unicode-table.com/en/#D8B9 d8b9 is not a arab letter, niehter is little endian b9d8 — none
Take a look at this table: utf8-chartable.de/… There you see that D8B9 in UTF-8 is ARABIC LETTER AIN or ع with unicode code point U+0639. UTF-8 is made to be more compact and better compatible with ASCII and Latin Ansi encoding, but a byte range in UTF-8 doesn't respresent the actual unicode code point. — GolezTrol

Remy Lebeau Remy Lebeau · Accepted Answer · 2013-11-10T17:25:59

TStringStream in D2009+ uses UnicodeString and is TEncoding-aware so DO NOT create a UTF8String manually:

var
  server: TIdHttp;
  Parameters,response: TStringStream;
begin
  response := TStringStream.Create;
  Parameters := TStringStream.Create('param1=Value1&param2=عربي/عرب&param3=Value3', TEncoding.UTF8);
  Server.Post('http://www.example.com/page.php',Parameters,response);
end;

Alternatively, the TStrings version also encodes to UTF-8 by default:

var
  server: TIdHttp;
  Parameters: TStringList;
  Response: TStringStream;
begin
  response := TStringStream.Create;
  Parameters := TStringList.Create;
  Parameters.Add('param1=Value1');
  Parameters.Add('param2=عربي/عرب');
  Parameters.Add('param3=Value3');
  Server.Post('http://www.example.com/page.php',Parameters,response);
end;

Either way, you should set the request charset before calling Post() so the server knows you are sending UTF-8 encoded data:

Server.Request.ContentType := 'application/x-www-form-urlencoded';
Server.Request.Charset := 'utf-8';

idhttp indy post, do request (parameters) with utf-8

1 Answers