'Malformed UTF-8 characters, possibly incorrectly encoded' in Laravel

Question

I'm using Laravel (a PHP framework) to write a service for mobile and have the data returned in JSON format. In the data result there are some fields encoded in UTF-8.

The following statement

return JsonResponse::create($data); 

returns the error below

InvalidArgumentException
HELP
Malformed UTF-8 characters, possibly incorrectly encoded

Open: /var/www/html/vendor/symfony/http-foundation/Symfony/Component/HttpFoundation/JsonResponse.php
        } catch (\Exception $exception) {
            restore_error_handler();

            throw $exception;
        }

        if (JSON_ERROR_NONE !== json_last_error()) {
            throw new \InvalidArgumentException($this->transformJsonError());
        }

I've changed:

return JsonResponse::create($data);

to

return JsonResponse::create($data, 200, array('Content-Type'=>'application/json; charset=utf-8' ));

but it still isn't working.

How can I fix it?

Answer 1

I wrote this method to handle UTF8 arrays and JSON problems. It works fine with array (simple and multidimensional).

/**
 * Encode array from latin1 to utf8 recursively
 * @param $dat
 * @return array|string
 */
   public static function convert_from_latin1_to_utf8_recursively($dat)
   {
      if (is_string($dat)) {
         return utf8_encode($dat);
      } elseif (is_array($dat)) {
         $ret = [];
         foreach ($dat as $i => $d) $ret[ $i ] = self::convert_from_latin1_to_utf8_recursively($d);

         return $ret;
      } elseif (is_object($dat)) {
         foreach ($dat as $i => $d) $dat->$i = self::convert_from_latin1_to_utf8_recursively($d);

         return $dat;
      } else {
         return $dat;
      }
   }
// Sample use
// Just pass your array or string and the UTF8 encode will be fixed
$data = convert_from_latin1_to_utf8_recursively($data);

Answer 2

I found the answer to this problem here

Just do

mb_convert_encoding($data['name'], 'UTF-8', 'UTF-8');

Answer 3

In my case I had a ucfirst on the asian letters string. This was not possible and produced a non utf8 string.

Answer 4

I've experienced the same problem. The thing is that I forgot to start the apache and mysql in xampp... :S

Answer 5

In my case, this causes error:

return response->json(["message" => "Model status successfully updated!", "data" => $model], 200);

but this not:

return response->json(["message" => "Model status successfully updated!", "data" => $model->toJson()], 200);

Answer 6

I know it's already an old question, but i had the same error today. For me setting the connection variable on model did the work.

/**
 * Table properties
 */
protected $connection = 'mysql-utf8';
protected $table = 'notification';
protected $primaryKey = 'id';

I don't know if the issue was with the database (probably), but the texts fields with special chars (like ~, ´ e etc) were all messed up.

---- Editing

That $connection var is used to select wich db connection your model will use. Sometimes it happens that in database.php (under /config folder) you have multiples connections and the default one is not using UTF-8 charset.

In any case, be sure to properly use charset and collation into your connection.

'connections' => [

    'mysql' => [
        'driver' => 'mysql',
        'host' => env('DB_HOST', '127.0.0.1'),
        'port' => env('DB_PORT', '3306'),
        'database' => env('DB_DATABASE', 'your_database'),
        'username' => env('DB_USERNAME', 'root'),
        'password' => env('DB_PASSWORD', 'database_password'),
        'unix_socket' => env('DB_SOCKET', ''),
        'prefix' => '',
        'strict' => false,
        'engine' => null
    ],

    'mysql-utf8' => [
        'driver' => 'mysql',
        'host' => env('DB_HOST', '127.0.0.1'),
        'port' => env('DB_PORT', '3306'),
        'database' => env('DB_DATABASE', 'your_database'),
        'username' => env('DB_USERNAME', 'root'),
        'password' => env('DB_PASSWORD', 'database_password'),
        'unix_socket' => env('DB_SOCKET', ''),
        'charset' => 'utf8',
        'collation' => 'utf8_unicode_ci',
        'prefix' => '',
        'strict' => false,
        'engine' => null
    ],

Answer 7

In Laravel 7.x, this helped me to get rid of this error.

$newString = mb_convert_encoding($arr, "UTF-8", "auto");
return response()->json($newString);

Answer 8

For more solution i have completed the solution of (the great) Tiago Gouvêa exposed before only for strings and arrays

i used md_convert_encoding() function instead of utf8_encode(), it works for me : (12 hours loosed ...)

// this object return me a big array who have multiple arrays imbricated

$get_days = program::get_days($ARR, $client);

// and i use this function for well parsing what the server return *

function convert_to_utf8_recursively($dat){
    if( is_string($dat) ){
        return mb_convert_encoding($dat, 'UTF-8', 'UTF-8');
    }
    elseif( is_array($dat) ){
        $ret = [];
        foreach($dat as $i => $d){
            $ret[$i] = convert_to_utf8_recursively($d);
        }
        return $ret;
    }
    else{
        return $dat;
    }
}
                
// use 
$data = convert_to_utf8_recursively($get_days);

• what the server return * i talk to that because i test the same code in two different servers the first return me a well formatted json , without any function, as we usually do BUT the second server does not send me back anything if I do not apply this function ...

Answer 9

First thing Correctly specify all settings in the .ENV file for Mail.

I LOST 6 HOURS FOR THIS

Example

MAIL_DRIVER=smtp
MAIL_MAILER=smtp
MAIL_HOST=smtp.yandex.ru
MAIL_PORT=465
MAIL_USERNAME=[e-mail]
MAIL_PASSWORD=password //Create password for Apps in settings. NOT PASTE YOUR REAL MAIL PASSWORD
MAIL_ENCRYPTION=SSL
MAIL_FROM_ADDRESS=[e-mail]
MAIL_FROM_NAME="${APP_NAME}"

Answer 10

I got this error and i fixed the issue with iconv function like following:

iconv('latin5', 'utf-8', $data['index']);

Answer 11

This happened to me when Laravel tried to return an error in Json format (while developing API). After solving the bug, this error disappeared.

Answer 12

Set the charset at after you made the connection to db like

// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
    die("Connection failed: " . $conn->connect_error);
}

if (!$conn->set_charset("utf8")) {
    printf("Error loading character set utf8: %s\n", $conn->error);
    exit();
} else {
    printf("Current character set: %s\n", $conn->character_set_name());
}