Symfony/Doctrine: Invalid byte sequence

1
votes

We are working on a REST API with Symfony & Doctrine, installed on Cent OS. When using the REST API via our website (charset=UTF8) everything works fine.

But when external users are using the API, and trying the send some accent characters (à, î, ...), some of them are facing the following error:

[2014-07-11 09:53:22] request.CRITICAL: Uncaught PHP Exception Doctrine\DBAL\DBALException: "An exception occurred while executing 'INSERT INTO xyz.abc (id, creation_date, ip, type, message) VALUES (?, ?, ?, ?, ?)' with params [19, null, "173.0.81.1", "VALID_TRANSACTION", "\x74\x78\x6e\x5f..."]: SQLSTATE[22021]: Character not in repertoire: 7 ERROR: invalid byte sequence for encoding "UTF8": 0xe1 0x6e 0x3b"

/var/www/xyz/symfony/vendor/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php:91, PDOException: SQLSTATE[22021]: Character not in repertoire: 7 ERROR: invalid byte sequence for encoding \"UTF8\": 0xe1 0x6e 0x3b at /var/www/xyz/symfony/vendor/doctrine/dbal/lib/Doctrine/DBAL/Statement.php:165)"} []

Our PostgreSQL database (psql -l) is configured like this:

  • Name: XYZ
  • Encoding: UTF8
  • Collating: en_US.UTF-8
  • Char. type: en_US.UTF-8

I see these errors in the logs, but I could never reproduce it myself. I trying the use the API with Content-Type: application/json; charset=... but it also works.

I really don't understand why this error is happening. And also why in the error, the content is encoded (\x74\x78\x6e\x5f...). If I try to connect to pgsql and insert the accent characters (not encoded), it also works...

Does anybody have an idea how to fix this ?

Thanks !

1
postgresqlsymfonydoctrine-orm

1 Answers

1
votes

Most likely someone tries to call your REST API with a non-utf8 valid message.

Doctrine's BDALException prints the bound parameters with json_encode, but if that fails (besause it's not utf-8) it will print it hex.

PostgreSQL also complains about it contains an invalid utf8 character-sequence.

The message starts with txn_ (which seems valid) but later on there is "\xe1\x6e\x3b" in it, which is not valid. It could be some improper use of your API.