Character Encoding Issue UTF-8 and ISO-8859-1

Question

I have a web application that I'm having problems getting Japanese/Chinese characters to display properly. The thing being that i can display these characters properly when I am hard coding them into an HTML document.

Characters such as:

アイヌの工芸 : ペンシルバニア大学考古学人類学博物館ヒラーコレクション

But when I grab them out of this proprietary database it comes out as junk:

ã¢ã¤ãã®å·¥è¸ : ãã³ã·ã«ããã¢å¤§å¦èå¤å¦äººé¡å¦åç©é¤¨ãã©ã¼ã³ã¬ã¯ã·ã§ã³

Now i have the html document encoded in utf-8

<meta http-equiv="content-type" content="text/html; charset=utf-8"/>

The actual html file itself is saved as "Encoded in utf-8" and not ISO-8859-1 or Western Latin etc.

So the weird thing is that when I use iconv to take the junk character string and convert it from utf-8 to ISO-8859-1 it displays correctly.

iconv("UTF-8", "ISO-8859-1//TRANSLIT", $junk_string)

It seems like the junk string is UTF-8 and when I convert the string to ISO-8859-1 it then displays the characters correctly. This doesn't make sense to me at all.

So I sort of have an answer to my problem but I do not know why it works. I thought that having encoding in UTF-8 was supposed to fix this kind of thing. And I am using Verdana but have tried a couple of other fonts with no success. And the weird thing being that I can hard code the characters with no problem into the html page and they display fine. But when get the same data from the database it is displayed as junk without me changing the encoding to ISO-8859-1.

Anyone have any insight here? And instead of doing this to every piece of data gotten from the database is there a way I can change this on the individual page level? I also tried to change the encoding to

<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"/>

And the characters from the database still do not display correctly.

jeroen jeroen · Accepted Answer · 2011-12-16T16:43:01

Just a guess, but when a database is utf8 and the html document is utf8, the problem most likely is the database connection, at least in my experience with MySQL.

For example for MySQL (the old / regular version), you need to set the character set after opening a database:

mysql_set_charset('utf8');

For PDO / MySQL it would be something like:

$db->exec('SET CHARACTER SET utf8');

Character Encoding Issue UTF-8 and ISO-8859-1

3 Answers