How To Change Encoding From ISO-8859-1 To UTF-8 for POST Request

0
votes

I wanted to ask how it is possible to change the URL encoding (from ISO-8859-1 to UTF-8) for a POST request.

Now my code is as follows:

    cl_http_client=>create_by_url(
    EXPORTING
      url                = lc_url
    IMPORTING
      client             = lr_client
    EXCEPTIONS
      argument_not_found = 1
      plugin_not_active  = 2
      internal_error     = 3
      OTHERS             = 4  ).
    IF sy-subrc <> 0.
      MESSAGE e020(rest_core_texts).
      EXIT.
    ENDIF.

    lr_client->request->set_method( method = if_http_entity=>co_request_method_post ).
    lr_client->request->set_content_type( content_type = 'text/plain; charset=utf-8' ).
    lr_client->request->set_form_field( name = 'sUsername'  value = lc_uname ).
    Etc..

    CALL METHOD lr_client->send
      EXCEPTIONS
        http_communication_failure = 1
        http_invalid_state         = 2
        http_invalid_timeout       = 3
        http_processing_failed     = 4
        OTHERS                     = 5.
    IF sy-subrc NE 0.
      MESSAGE i400(sclnt_http).
      EXIT.
    ENDIF.

One of the form fields in the POST request contains the name of the person and those names might contain german umlaute (ä,ö,ü). The resulting URL is then encoded using an ISO codepage instead of UTF and the external system expects it to be encoded in UTF.

The result is that the external system stores the name the wrong way (e.g. Gr%e4%df) because the URL is encoded using ISO-8859-1 instead of UTF-8.

This is most likely caused by the fact that the system uses ISO-8859-1 by default (Table: TCP0C).

Now, I have tried transforming the variable holding the name using the class CL_ABAP_CODEPAGE from string to xstring and then vice versa as there is no method to directly transform a string variable to a different code page.

Unfortunately this has not yielded any success.

My second guess was to try to transform the http request body into UTF but I didn't find any suitable method nor function module which I could use.

Any suggestion would be much appreciated!

EDIT:

  • The system is a non-Unicode system.
  • The system codepage is ISO-8859-1.
2
http-postabap
It is all stored well according to percent-encoding. The other side needs to decode this post request in a right way. Stop trying what you are trying, it is the other side of the rope that needs to decode this POST-Request correcty. - Jagger
You need to check the URL standards and ask you partner system to handle request correctly (using some URLdecode functions, not "as is"). Here's the quote from the RFC3986, Section 2.1: A percent-encoding mechanism is used to represent a data octet in a component when that octet's corresponding character is outside the allowed. Sections 2.2 and 2.3 describe reserved and unreserved characters (which in total are allowed characters). So there's no way to pass unicode codepoints in URL without percent encoding. - astentx
@SandraRossi "UTF-8 corresponds to U+00E4 ie %E4" I don't think that this statement is correct. UTF-8 ä converted to percent encoding corresponds to %C3%A4 wheres ä in ISO-8859-1 in percent encoding corresponds to %E4. I have checked this via this online encoding tool dencode.com/en/string/url-encoding - Jakub Zvonek
@Jagger Thank you for the support. The percent encoding is fine, however, currently the umlaut, i.e. ä is encoded into %E4 which is ISO-8859-1 percent encoding whereas the target system expects it to be %C3%A4 which would correspond to UTF-8 percent encoding. - Jakub Zvonek
I don't know if it works, could you try lr_client->request->set_formfield_encoding( formfield_encoding = lr_client->request->CO_ENCODING_URL ).? - Sandra Rossi

2 Answers

1
votes

I have a solution on how to properly escape the URL in UTF-8 for the post request even though the systems default code page is not unicode.

Use the following method to escape the name and address, or other variables which might contain non-ASCII characters for that matter.

  CALL METHOD cl_http_utility=>escape_url
    EXPORTING
      unescaped = lv_pname
      options   = 1
    receiving
      escaped   = lv_pname.

Do not forget to pass the options parameter as otherwise the default system code page is used for escaping the string variable.

You can subsequently pass the escaped variable via set_form_field() as usual.

0
votes

I cannot reproduce your problem in a Unicode ABAP 7.52 system. With your code + lc_url = 'http://dummy/dummy'. and lc_uname = 'ändern'., the ICF trace gives:

POST /dummy?sUsername=%c3%a4ndern HTTP/1.0
Content-Type: text/plain; charset=utf-8
Content-Length: 0
user-agent: SAP NetWeaver Application Server (1.0;752)
host: dummy
accept-encoding: gzip

Concerning the query string, according to the note 1228903 - CL_HTTP_CLIENT: Escaping of special characters in URL:

Example of incorrect source code: CL_HTTP_UTILITY=>SET_REQUEST_URI( '/test?name=M%FCller' )
...
In Unicode systems, UTF-8 is used as the character set; in NON-Unicode systems, the character set of the session is used.
...
When escaping, Unicode systems use UTF8 as the code page and non-Unicode systems use the system code page.
...
Workaround: You can use the method SET_HEADER_FIELD of the REQUEST object to set the URL via the pseudo header field "~request_uri".

In your case, that would correspond to lr_client->request->SET_HEADER_FIELD( name = '~request_uri' value = '/dummy?sUsername=%c3%a4ndern'.