ICU conversion with ucnv_toUnicode

Question

I am new to ICU (IBM's unicode library). I am reading a large file in chunks, and I am trying to convert it from UTF-8 to UTF-16.

I am using ucnv_toUnicode, and I am running into a problem: how do I determine how many bytes where written by the converter to the target?

        ucnv_toUnicode(conv, &target, targetLimit, 
                   &source, sourceLimit, NULL,
                   feof(f)?TRUE:FALSE,         
                   &status);

target is a 4096 byte buffer.

According to the api docs, it will be moved by ucnv_toUnicode to point after the last UChar copied. It seems like I should be able to do some sort of arithmatic between target and the original position to determine this, but I'm new to C. Can anyone give me a hand?

Now suppose that I want to fwrite() what was put into target. what would I pass to fwrite for the size and number of units?

Kerrek SB Kerrek SB · Accepted Answer · 2012-09-19T22:12:55

As the documentation says:

Target [...] starts out pointer at the first available UChar in the output buffer, and ends up pointing after the last UChar written to the output.

So it's simple:

char input[SLEN];
UChar output[TLEN];

char const * source = input;
UChar * target = output;

ucnv_toUnicode(conv, &target, output + TLEN,
               &source, input + SLEN, NULL, feof(f), &status);

Now you have written target - output UChars to the output buffer.

The same goes for the input; you'll have consumed source - input chars (= bytes).

ICU conversion with ucnv_toUnicode

1 Answers