IBM MQ server encodes messages

Question

I'm sending XML to an IBM MQ Queue that contains a CDATA section. That CDATA section contains these special characters: $§)#ÜÖ&!^. For some reason, they are showing up within the MQ Queue as $�)#��&!^. This causes the other send to take it off the queue with these characters and ending up having an invalid signature because the messages no longer match up.

We've verified that the message when we do a .Put() does contain an XML string with those special characters. I've ensured that the message has .CharacterSet property assigned to it that matches what we will eventually pull off the queue.

What other places can possibly be auto-encoding the special characters when it's put on the queue? Our application is in a .NET windows environment, but the MQ server is on a Linux box. Is this something to consider?

string xmlMsg = "<message><data><![CDATA[<value>$§)#ÜÖ&amp;!^</value>]]</data></message>"; // This is in a CDATA section.
mQMessage = new MQMessage
{
    CharacterSet = 1208,
};

mQMessage.WriteBytes(xmlMsg);

_queue.Put(mQMessage);

Does the message contain the bad characters right after it is put to the queue, or when the downstream get happens? — JoshMc
@JoshMc If I view the message within the queue in IBM MQ Explorer, those special characters appear as $�)#��&!^. When I am doing a .Put() the message I am putting on the queue has the expected characters: $§)#ÜÖ&!^. — Jimenemex
Can you provide sample minimal .Net code that produces this issue? By default .Net represents things in CCSID 1200 (UTF-16). What CCSID are you setting on the message (this will show up in your code if you can add that to the question). — JoshMc
Most likely, the receiving application is issuing a Get with Convert. You have 3 options: (1) the sending application can set the MQMD.Format field to CMQC.MQFMT_NONE or (2) change the receiving application to perform the Get without Convert or (3) change the CDATA to be a HEX representation of the data. — Roger
@SonBrooks For our issue, we actually had to use WriteString as after horrible documentation and dotPeeking the code the sender was using, they were writing and reading to the queue as a string. — Jimenemex

Attila Repasi Attila Repasi · Accepted Answer · 2020-04-10T06:55:53

By default MQ doesn't change the character set of your message. So by default it is the responsibility of the sending and receiving applications to agree and maintain a character set that suits both.

You can request MQ to do character set conversion either in the receiving application, when that calls a get, or on the sender channels when the message is transmitted between queue managers. But even if you request character set conversion from MQ, it is still the sending applications responsibility to actually write the data into the message using the character set the application is setting on the MQ message header.

Based on your code it seems your sending application doesn't use the correct character set when it writes the bytes to the message. If you use WriteBytes, you need to manually convert the string into bytes using the desired character set.

I'd suggest you to use the WriteString method, which is designed to use the chracter set specified in the CharacterSet property:

The WriteString method converts from Unicode to the character set encoded in CharacterSet. If CharacterSet is set to its default value, MQC.MQCCSI_Q_MGR, which is 0, no conversion takes place and CharacterSet is set to 1200. If you set CharacterSet to some other value, WriteString converts from Unicode to the alternate value.

https://www.ibm.com/support/knowledgecenter/SSFKSJ_7.5.0/com.ibm.mq.ref.dev.doc/q111220_.htm

And by the way, for debugging character set issues you have to be very careful what tools you use to check the message, as your tool needs to be able to interpret the character set of the message. For example MQ Explorer uses the character set of your workstation where you run it, so it will show every message with that one character set, so is not suitable to debug these issues. The best is to get the message off the queue without asking the QM for conversion with rfhutil for example, save it to a file and look at it with a hex editor.

IBM MQ server encodes messages

1 Answers