UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 38: invalid start byte

Question

I am importing my model from Tensorflow and just want to optimize the trained model using the following piece of code:

input_graph_def = tf.GraphDef()
with tf.gfile.Open(output_frozen_graph_name, "r") as f:
    data = f.read()
    input_graph_def.ParseFromString(data)

output_graph_def = optimize_for_inference_lib.optimize_for_inference(
        input_graph_def,
        ["input"], 
        ["y_"],
        tf.float32.as_datatype_enum)

f = tf.gfile.FastGFile("optimized_shoaib-har_agm.pb", "w")
f.write(output_graph_def.SerializeToString())

And it shows this error:

Traceback (most recent call last): File "", line 2, in data = f.read() File "C:\Users\Chaine\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 125, in read pywrap_tensorflow.ReadFromStream(self._read_buf, length, status)) File "C:\Users\Chaine\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\lib\io\file_io.py", line 93, in _prepare_value return compat.as_str_any(val) File "C:\Users\Chaine\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\util\compat.py", line 106, in as_str_any return as_str(value) File "C:\Users\Chaine\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\util\compat.py", line 84, in as_text return bytes_or_text.decode(encoding) UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 38: invalid start byte

Before it was working fine. I even imported it to Android Studio already. Now all of the sudden I am getting this exception. Is it because I installed something in my machine?

I was able to install the app on my smartphone without any errors. And now it is giving me errors. I am using the same exact code.

I was able to install it on my smartphone without any errors. And now it is giving me errors. I am using the same exact code. — Chaine
I was using this code last week without any errors. And now I tried running it again and it is giving me errors. — Chaine

Enigma Enigma · Accepted Answer · 2017-10-03T15:47:01

Python is trying to decode all characters in the file "output_frozen_graph_name". Not sure what changed in the file if it was working for you before, but clearly it looks like some characters are not 'UTF-8' compatible. Now, they can be 'UTF-16' or some other codec format. One way to know is to read the contents in byte format and decode it yourself. Try to read as follows to check the contents:

input_graph_def = tf.GraphDef()
with tf.gfile.Open(output_frozen_graph_name, "rb") as f:
    data = f.read()
    input_graph_def.ParseFromString(data)

output_graph_def = optimize_for_inference_lib.optimize_for_inference(
        input_graph_def,
        ["input"], 
        ["y_"],
        tf.float32.as_datatype_enum)

f = tf.gfile.FastGFile("optimized_shoaib-har_agm.pb", "w")
f.write(output_graph_def.SerializeToString())

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 38: invalid start byte

1 Answers