Creating an amazon MWS signature with Delphi XE7 and Indy classes

Question

I need to generate a signature for amazon MWS and decided to find a solution with only the components and classes which come with Delphi. Because I am using Indy for the HTTP post itself, it seemed to be a good idea to use Indy classes for the calculation of the RFC 2104-compliant HMAC.

For others, who work on amazon integration, the creation of the "Canonicalized Query String" is explained in the amazon tutorial very well: http://docs.developer.amazonservices.com/en_DE/dev_guide/DG_ClientLibraries.html Be careful, just use #10 for line breaking, as #13#10 or #13 will fail with a wrong signature. It may also be important to add ":443" to the amazon Endpoint (Host), depending on the TIdHttp version, as explained in question #23573799.

To create a valid signature, we have to calculate a HMAC with SHA256 with the query string and the SecretKey we got from amazon after registration and then, the result has to be encoded in BASE64.

The query string is properly generated and identical to the string the amazon Scratchpad creates. But the call failed because the signature is not correct.

After some tests I realized that the signature I got from my query string is not the same as the result I got when I used PHP to generate it. The PHP result is considered as correct, as my PHP solution simply works with amazon since a long time, the Delphi result is different, which is not correct.

To make testing easier, I use '1234567890' as value for the query string and 'ABCDEFG' as replacement for the SecretKey. When the result I get with Delphi is the same as the result I get with PHP, the problem should be solved, I believe.

Here is how I get the correct result with PHP:

echo base64_encode(hash_hmac('sha256', '1234567890', 'ABCDEFG', TRUE));

This shows a result of

aRGlc3RY1pKmKX0hvorkVKNcPigiJX2rksqXzlAeCLg=

The following Delphi XE7 code returns the wrong result, while using the indy version that comes with Delphi XE7:

uses
  IdHash, IdHashSHA, IdHMACSHA1, IdSSLOpenSSL, IdGlobal, IdCoderMIME;

function GenerateSignature(const AData, AKey: string): string;
var
   AHMAC: TIdBytes;
begin
     IdSSLOpenSSL.LoadOpenSSLLibrary;

     With TIdHMACSHA256.Create do
      try
         Key:= ToBytes(AKey, IndyTextEncoding_UTF16LE);
         AHMAC:= HashValue(ToBytes(AData, IndyTextEncoding_UTF16LE));
         Result:= TIdEncoderMIME.EncodeBytes(AHMAC);
      finally
         Free;
      end;
end; 

Here the result, which is shown in a Memo with

Memo.Lines.Text:= GenerateSignature('1234567890', 'ABCDEFG'); 

is:

jg6Oddxvv57fFdcCPXrqGWB9YD5rSvtmGnZWL0X+y0Y=

I believe the problem has something to do with the encodings, so I have done some research around that. As the amazon tutorial (link see above) tells, amazon expects a utf8 encoding.

As the Indy function "ToBytes" expect a string, which is a UnicodeString in my Delphi version, I quit testing with other string types as UTF8String for parameters or variables, but I just do not know where utf8 should come into place. Also I do not know if the encodings I use in the code above are the correct ones. I choose UTF16LE because UnicodeString is utf16 encoded (see http://docwiki.embarcadero.com/RADStudio/Seattle/en/String_Types_(Delphi) for details) and LE (Little-Endian) is most commonly used on modern machines. Also the TEncodings of Delphi itself there is "Unicode" and "BigEndianUnicode", so "Unicode" seems to be LE and some kind of "standard" Unicode. Of course I tested to use IndyTextEncoding_UTF8 instead of IndyTextEncoding_UTF16LE in the code above, but it does not work anyway.

Because

TIdEncoderMIME.EncodeBytes(AHMAC);

is writing the TidBytes to a Stream first and then reading it all with 8bit encoding, this could be a source of problem also, so I also tested with

Result:= BytesToString(AHMAC, IndyTextEncoding_UTF16LE);
Result:= TIdEncoderMIME.EncodeString(Result, IndyTextEncoding_UTF16LE); 

but the result is the same.

If you like to see the main code for creating the request, here it is:

function TgboAmazon.MwsRequest(const AFolder, AVersion: string;
  const AParams: TStringList; const AEndPoint: string): string;
var
   i: Integer;
   SL: TStringList;
   AMethod, AHost, AURI, ARequest, AStrToSign, APath, ASignature: string;
   AKey, AValue, AQuery: string;
   AHTTP: TIdHTTP;
   AStream, AResultStream: TStringStream;
begin
     AMethod:= 'POST';
     AHost:= AEndPoint;
     AURI:= '/' + AFolder + '/' + AVersion;

     AQuery:= '';
     SL:= TStringList.Create;
     try
        SL.Assign(AParams);
        SL.Values['AWSAccessKeyId']:= FAWSAccessKeyId;
        SL.Values['SellerId']:= FSellerId;
        FOR i:=0 TO FMarketplaceIds.Count-1 DO
         begin
              SL.Values['MarketplaceId.Id.' + IntToStr(i+1)]:= FMarketplaceIds[i];
         end;

        SL.Values['Timestamp']:= GenerateTimeStamp(Now);
        SL.Values['SignatureMethod']:= 'HmacSHA256';
        SL.Values['SignatureVersion']:= '2';
        SL.Values['Version']:= AVersion;

        FOR i:=0 TO SL.Count-1 DO
         begin
              AKey:= UrlEncode(SL.Names[i]);
              AValue:= UrlEncode(SL.ValueFromIndex[i]);
              SL[i]:= AKey + '=' + AValue;
         end;

        SortList(SL);
        SL.Delimiter:= '&';
        AQuery:= SL.DelimitedText;

        AStrToSign:= AMethod + #10 + AHost + #10 + AURI + #10 + AQuery;
        TgboUtil.ShowMessage(AStrToSign);

        ASignature:= GenerateSignature(AStrToSign, FAWSSecretKey);
        TgboUtil.ShowMessage(ASignature);

        APath:= 'https://' + AHost + AURI + '?' + AQuery + '&Signature=' + Urlencode(ASignature);
        TgboUtil.ShowMessage(APath);
     finally
        SL.Free;
     end;

     AHTTP:= TIdHTTP.Create(nil);
     try
        AHTTP.IOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(AHTTP);
        AHTTP.Request.ContentType:= 'text/xml';
        AHTTP.Request.Connection:= 'Close';
        AHTTP.Request.CustomHeaders.Add('x-amazon-user-agent: MyApp/1.0 (Language=Delphi/XE7)');
        AHTTP.HTTPOptions:= AHTTP.HTTPOptions + [hoKeepOrigProtocol];
        AHTTP.ProtocolVersion:= pv1_0;
        AStream:= TStringStream.Create;
        AResultStream:= TStringStream.Create;
        try
           AHTTP.Post(APath, AStream, AResultStream);
           Result:= AResultStream.DataString;
           ShowMessage(Result);
        finally
           AStream.Free;
           AResultStream.Free;
        end;
     finally
        AHTTP.Free;
     end;
end;

Urlencode and GenerateTimestamp are my own functions and they do what the name promises, SortList is my own procedure which sorts the stringlist in a byte order as requested by amazon, TgboUtil.ShowMessage is my own ShowMessage alternative which shows the complete message with all characters and is used for debugging only. The http protocol is 1.0 for testing only, because I got a 403 (permission denied) as HTTP return earlier. I just wanted to exclude this as problem as the indy documentation said, that protocol version 1.1 is considered incomplete because of problematic server answers.

There are several posts regarding the amazon mws topic here, but that specific problem seems to be new.

This question here may help someone who just not have come so far, but also I hope that someone can provide a solution to just get the same signature value in Delphi as I got with PHP.

Thank you in advance.

Answer 1

Using the latest SVN snapshot of Indy 10, I am not able to reproduce your signature problem. When using UTF-8, your example key+value data produces the same result in Delphi as the PHP output. So, your GenerateSignature() function is fine, provided that:

1. you use IndyTextEncoding_UTF8 instead of IndyTextEncoding_UTF16LE.

2. you make sure that AData and AKey contain valid input data.

Also, you should make sure that TIdHashSHA256.IsAvailable returns true, otherwise TIdHashHMACSHA256.HashValue() will fail. this could happen, for instance, if OpenSSL fails to load.

Try this instead:

function GenerateSignature(const AData, AKey: string): string;
var
  AHMAC: TIdBytes;
begin
  IdSSLOpenSSL.LoadOpenSSLLibrary;

  if not TIdHashSHA256.IsAvailable then
    raise Exception.Create('SHA-256 hashing is not available!');

  with TIdHMACSHA256.Create do
  try
    Key := IndyTextEncoding_UTF8.GetBytes(AKey);
    AHMAC := HashValue(IndyTextEncoding_UTF8.GetBytes(AData));
  finally
    Free;
  end;

  Result := TIdEncoderMIME.EncodeBytes(AHMAC);
end; 

That being said, there are quite a few problems with your MwsRequest() function:

1. you are leaking the TIdSSLIOHandlerSocketOpenSSL object. You are not assigning an Owner to it, and TIdHTTP does not take ownership when assigned to its IOHandler property. In fact, assigning the IOHanlder is actually optional in your example, see New HTTPS functionality for TIdHTTP for why.

2. you are setting AHTTP.Request.ContentType to the wrong media type. You are not sending XML data, so don't set the media type to 'text/xml'. In this situation, you need to set it to 'application/x-www-form-urlencoded' instead.

3. when calling AHTTP.Post(), your AStream stream is empty, so you are not actually posting any data to the server. You are putting your AQuery data in the query string of the URL itself, but it actually belongs in AStream instead. If you want to sent the data in the URL query string, you have to use TIdHTTP.Get() instead of TIdHTTP.Post(), and change your AMethod value to 'GET' instead of 'POST'.

4. you are using the version of TIdHTTP.Post() that fills an output TStream. You are using a TStringStream to convert the response to a String without any regard to the actual charset used by the response data. Since you are not specifying any TEncoding object in the TStringStream constructor, it will use TEncoding.Default for decoding, which may not (and likely will not) match the response's actual charset. You should instead use the other version of Post() that returns a String so TIdHTTP can decode the response data based on the actual charset reported by the HTTPS server.

Try something more like this instead:

function TgboAmazon.MwsRequest(const AFolder, AVersion: string;
  const AParams: TStringList; const AEndPoint: string): string;
var
  i: Integer;
  SL: TStringList;
  AMethod, AHost, AURI, AQuery, AStrToSign, APath, ASignature: string;
  AHTTP: TIdHTTP;
begin
  AMethod := 'POST';
  AHost := AEndPoint;
  AURI := '/' + AFolder + '/' + AVersion;

  AQuery := '';
  SL := TStringList.Create;
  try
    SL.Assign(AParams);

    SL.Values['AWSAccessKeyId'] := FAWSAccessKeyId;
    SL.Values['SellerId'] := FSellerId;
    for i := 0 to FMarketplaceIds.Count-1 do
    begin
      SL.Values['MarketplaceId.Id.' + IntToStr(i+1)] := FMarketplaceIds[i];
    end;

    SL.Values['Timestamp'] := GenerateTimeStamp(Now);
    SL.Values['SignatureMethod'] := 'HmacSHA256';
    SL.Values['SignatureVersion'] := '2';
    SL.Values['Version'] := AVersion;
    SL.Values['Signature'] := '';

    SortList(SL);

    for i := 0 to SL.Count-1 do
        SL[i] := UrlEncode(SL.Names[i]) + '=' + UrlEncode(SL.ValueFromIndex[i]);

    SL.Delimiter := '&';
    SL.QuoteChar := #0;
    SL.StrictDelimiter := True;
    AQuery := SL.DelimitedText;
  finally
    SL.Free;
  end;

  AStrToSign := AMethod + #10 + Lowercase(AHost) + #10 + AURI + #10 + AQuery;
  TgboUtil.ShowMessage(AStrToSign);

  ASignature := GenerateSignature(AStrToSign, FAWSSecretKey);
  TgboUtil.ShowMessage(ASignature);

  APath := 'https://' + AHost + AURI;
  TgboUtil.ShowMessage(APath);

  AHTTP := TIdHTTP.Create(nil);
  try
    // this is actually optional in this example...
    AHTTP.IOHandler := TIdSSLIOHandlerSocketOpenSSL.Create(AHTTP);

    AHTTP.Request.ContentType := 'application/x-www-form-urlencoded';
    AHTTP.Request.Connection := 'close';
    AHTTP.Request.UserAgent := 'MyApp/1.0 (Language=Delphi/XE7)';
    AHTTP.Request.CustomHeaders.Values['x-amazon-user-agent'] := 'MyApp/1.0 (Language=Delphi/XE7)';
    AHTTP.HTTPOptions := AHTTP.HTTPOptions + [hoKeepOrigProtocol];
    AHTTP.ProtocolVersion := pv1_0;

    AStream := TStringStream.Create(AQuery + '&Signature=' + Urlencode(ASignature);
    try
      Result := AHTTP.Post(APath, AStream);
      ShowMessage(Result);
    finally
      AStream.Free;
    end;
  finally
    AHTTP.Free;
  end;
end;

However, since the response is documented as being XML, it would be better to return the response to the caller as a TStream (not using TStringStream, though) or TBytes instead of as a String. That way, instead of Indy decoding the bytes, let your XML parser decode the raw bytes on its own. XML has its own charset rules that are separate from HTTP, so let the XML parser do its job for you:

procedure TgboAmazon.MwsRequest(...; Response: TStream);
var
  ...
begin
  ...
  AHTTP.Post(APath, AStream, Response);
  ...
end;