How to cast hex data string to a string db2 sql

Question

How would you decode a hex string to get the value in text format by using a select statement?

For example my data in hex is:

4f004e004c005900200046004f00520020004200410043004b002d005500500020004f004e0020004c004500560045004c0020004f004e004500200046004f00520020004300520041004e004500530020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020000000

I want to decode it to get the string value using a select statement. The value of the above is "ONLY FOR BACK-UP ON LEVEL ONE FOR CRANES"

what I have tried is :

    SELECT CAST('4f004e004c005900200046004f00520020004200410043004b002d005500500020004f004e0020004c004500560045004c0020004f004e004500200046004f00520020004300520041004e004500530020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020002000200020000000' 
    AS VARCHAR(30000) CCSID 37) from myschema.atable

The above sql returns the exact same hex string and not the decoded text string of "ONLY FOR BACK-UP ON LEVEL ONE FOR CRANES" what I expected.

Is it possible to do this with a cast? If it is what will the syntax be?

My problem that I have is a system stores text data in a blob field and I want to use a select statement to see what the text data is in the blob field.

Db : Db2 on Ibm

Edit:

I have managed to covert the string to the hex value by using :

    select hex(cast('ONLY FOR BACK-UP ON LEVEL ONE FOR CRANES' as varchar(100) ccsid 1208))
FROM myschema.atable

This gives me the string in hex :

4F4E4C5920464F52204241434B2D5550204F4E204C4556454C204F4E4520464F52204352414E4553

Now somehow I need to do the inverse and get the value.

Thanks.

Edit

Using the answer from Daniel Lema, I tried using the unhex function but my result that I got was :

|+<ßã|êâ ä.í&|+<áîá<|+áã|êäê +áë

Is this something to do with a CSSID? Or how should I convet the above to a readable string?

This is the table field definition if it will help the field with my data in is GDTXFT a BLOB :

Answer 1

I was able to take your shortened hex string and convert is to a valid EBCDIC string. The problem I ran into is that the original hex code you receive comes in UTF-16LE (Thanks Tom Blodget). IBM's CCSID system does not have a distinction between UTF-16BE and UTF-16LE so I am at a loss there on how to convert it properly.

If it is in UTF-8 as you generated later, the following would work for you. It's not the prettiest but throw it in a couple functions and it will work.

Create or replace function unpivothex (in_ varchar(30000))
    returns table (Hex_ char(2), Position_ int)
    return
    with returnstring (ST , POS )
    as 
    (Select substring(STR,1,2), 1
    from table(values in_) as A(STR)
    union all
    Select nullif(substring(STR,POS+2,2),'00'), POS+2
    from returnstring, table(values in_) as A(STR)
    where POS+2 <= length(in_)
    )
    Select ST, POS 
    from returnstring
    ;

Create or replace function converthextostring
   (in_string char(30000))
   returns varchar(30000)
   return
   (select listagg(char(varbinary_format(B.Hex_),1)) within group(order by In_table.Position_)
   from table(unpivothex(upper(in_string))) in_table
   join table(unpivothex(hex(cast('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz ' as char(53) CCSID 1208)))) A on In_table.Hex_ = A.Hex_
   join table(unpivothex(hex(cast('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz ' as char(53) CCSID 37)))) B on A.Position_ = B.Position_
   );

Here is a version if you're not on at least V7R2 TR6 or V7R3 TR2.

Create or replace function converthextostring
   (in_string char(30000))
   returns varchar(30000)
   return
   (select xmlserialize(
             xmlagg(
               xmltext(cast(char(varbinary_format(B.Hex_),1) as char(1) CCSID 37)) 
             order by In_table.Position_) 
           as varchar(30000))
   from table(unpivothex(upper(in_string))) in_table
   join table(unpivothex(hex(cast('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz ' as char(53) CCSID 1208)))) A on In_table.Hex_ = A.Hex_
   join table(unpivothex(hex(cast('ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz ' as char(53) CCSID 37)))) B on A.Position_ = B.Position_
   );

Answer 2

I tried the following solution I found published by Marcin Rudzki at Convert HEX value to CHAR on DB2, tested in my own Db2 for LUW v11 with a small modification.

the solution consists on creating a function just as Marcin suggested:

CREATE FUNCTION unhex(in VARCHAR(32000) FOR BIT DATA)
RETURNS VARCHAR(32000) 
LANGUAGE SQL
CONTAINS SQL
DETERMINISTIC NO EXTERNAL ACTION
BEGIN ATOMIC
RETURN in;
END

To test the solution, lets create an HEXSAMPLE table with a HEXSTRING column loaded with the string representation of a HEX sequence:

INSERT INTO HEXSAMPLE (HEXSTRING) VALUES ('4F4E4C5920464F52204241434B2D5550204F4E204C4556454C204F4E4520464F52204352414E4553')

Then exec the following query (and here it is different from the original proposal):

SELECT UNHEX(CAST(HEXTORAW(HEXSTRING) AS VARCHAR(2000) FOR BIT DATA)) as TEXT, HEXSTRING FROM HEXSAMPLE

With result:

 TEXT                                           HEXSTRING
 ----------------------------------------   --------------------------------------------------------------------------------
 ONLY FOR BACK-UP ON LEVEL ONE FOR CRANES   4F4E4C5920464F52204241434B2D5550204F4E204C4556454C204F4E4520464F52204352414E4553

I hope someone else can find a more direct solution. Also, if someone can explain why it works, it will be very interesting.

Answer 3

I question why you need to do this...

There's valid reasons to convert a hex string back to it's character equivalent...for instance somebody sends you a 32 byte string UUID and you want it back it it's 16 byte binary form.

But there's no reason ONLY FOR BACK-UP ON LEVEL ONE FOR CRANES should have been transformed to hex.

I suspect you need to post a new question asking why you're not getting readable strings in the first place.

However, in answer to this question... IBM i has an MI function Convert Character to Hex (CVTCH) that is easily called from any ILE langage. You could wrap that function call up into a user defined function in order to use it from SQL.

Note that you'll need to know what the hex string represents, EBCDIC, ASCII or Unicode, because you'll need to be able to tell the system what you've started with. From there there are ways to convert between encoding.

Here's an article that shows how to call the MI function from RPG. Utilizing MI Functions in RPG Programs

A more modern free form version of the prototype that takes advantage of enhancements to the CCSID keyword might look like

dcl-pr FromHex extproc('cvtch');
  charString char(32767) ccsid(*UTF8) options(*varsize);
  hexString  char(65534) ccsid(*HEX) const options(*varsize);
  hexStringLen int(10) value;
end-pr;

With the above prototype, the system will treat the character string that comes back as UTF8 (ccsid 1208). But all I'm doing is telling the system how to interpret the bytes that come back. If the string was actually EBCDIC, I'm going to get garbage.

I think you could even defined the cvtch function directly as an external UDF without needing an ILE wrapper. I'd have to play around with that...

Disregard that idea...cvtch only has parameters, not a return value. Using an ILE wrapper is the best way to move the output parameter to a return value for use as a UDF.

Answer 4

The problem is that your original string is in ASCII format (actually with x'00' byte after each letter), and you have to convert it to EBCDIC.
Below is the solution for latin capital letters only:

select cast(translate(replace(mycol, x'00', x'')  
, x'C1C2C3C4C5C6C7C8C9D1D2D3D4D5D6D7D8D9E2E3E4E5E6E7E8E940'  
, x'4142434445464748494A4B4C4D4E4F505152535455565758595A20'
) as varchar(500) ccsid 37)
from mytab;

Every ASCII character is translated to the corresponding EBCDIC one.
x'00' symbols are removed.

Answer 5

cast (col_name as varchar(2000) ccsid ascii for sbcs data)