Delphi XE, Firebird and UTF8

Question

I'm upgrading a D7 program to XE, and under Delphi 7 I had code like this...

ParamByName ('Somefield').AsString:=someutf8rawbytestring;

Under XE if someutf8rawbytestring contains unicode characters such as Cyrillic script, then they appear as ???? in the DB.

I see that someutf8rawbytestring is 8 characters long, for my 4 character string, which is correct. But in the DB there are just four characters.

I'm using Firebird 2 through TIBQuery with XE and updating a Varchar field with character type 'NONE'.

So what it looks like is that the utf8 is being detected and converted somehow back to unicode data points, and then that is failing a string conversion for the DB. I've tried setting the varchar field to UTF8 encoding but with the same result.

So how should this be handled?

EDIT: I can use a database tool and edit my DB field to have some non-ASCII data and when I read it back it comes as a utf8 encoded string that I can use UTF8decode on and it's correct. But writing data back to this field seems impossible without getting a bunch of ???? in the DB. I've tried ParamByName ('Somefield').AsString:=somewidestring; and ParamByName ('Somefield').AsWideString:=somewidestring; and I just get rubbish in the DB...

EDIT2: Here's the code (in one iteration) ...



procedure TFormnameEdit.savename(id : integer);
begin
    With DataModule.UpdateNameQuery do begin
        ParamByName ('Name').AsString:=UTF8Encode(NameEdit.Text);
        ParamByName ('ID').AsInteger:=id;
        ExecSQL;
        Transaction.Commit;
    end;
end;

What’s the definition of someutf8rawbytestring? (Both in D7 and in XE) — Martijn
With InterBase I used ParamByName ('Somefield').AsWideString and UTF8 encoding setting — mjn
In D7 it's a 'string', in XE it's a Rawbytestring. In both cases it's a utf8 encoded 8 bit string. — Terry
Can you please provide the exact code that you're using (including declaration of any variables)? — LightBulb
@Terry, did you try without UTF8Encode? Delphi 2009+ has full unicode support so there's no need for such conversions. Additionally, check if both your Firebird database and TIBDatabase component have their CHARACTER SET set to UTF8. — LightBulb

Terry Terry · Accepted Answer · 2012-06-06T04:52:48

As @Lightbulb recommended, adding lc_ctype=UTF8 to the TIBDatabase params solved the problem.

Delphi XE, Firebird and UTF8

1 Answers