Python: '\x00' randomly appearing in serial from arduino to raspberry pi

Question

I am streaming data from an arduino back to a raspberry pi through the /dev/ttyUSB0 port. I am using python on the pi side for reading the incoming serial data. For debug purposes I am currently printing out each serial.readLine(). Two lines of debug can be seen below:

['A', '337', '731', '88\r\n']

['A', '33\x009', '1931', '8\r\n']

Every once in a while a random \x00 appears in the data and I can't figure out where it is coming from. I haven't streamlined the serial communication from the arduino so I believe it is sending each digit as a separate byte like 3 3 \x00 9 , so that can obviously be improved.

Also this only seems to happen when I run the serial communication in a seperate thread in python... I'm not too familiar with multithreading in python but this seemed too repeatable to just be coincidental.

Some of the relevent code Arduino:

Serial2.print(F("A,"));
Serial2.print(startingAngle + ix);
Serial2.print(F(","));
Serial2.print(int(aryDist[ix]));
Serial2.print(F(","));
Serial2.println(aryQuality[ix]);

Python:

queue = Queue()
lidar_thread = Thread( target=lidar.process, args=(queue, ))
lidar_thread.start()

In the thread function:

with serial.Serial('/dev/ttyUSB0', 115200, timeout=1) as ser:
   ser.write(b'ShowDist\n')
   while True:
      s = ser.readline()
      vals = s.split(',')

Again, if I just call the function outside of a thread it seems to work fine. Any ideas on where this '\x00' comes from and why it seems to only happen when multi threaded? In the end I want to cast each value as an int which breaks when that extra byte gets included... I would also be interested in a computationally inexpensive way of checking if the byte is okay before parsing... from what I have read the try catch approach can slow things down.

Thanks

Is it possible that you just got a timeout when using readline, timeout as understands gives you a EOF pyserial.readthedocs.io/en/latest/shortintro.html at the same time it doesnt make sense, as the EOF val is -1? \0 is usually used as a terminator in strings, is it possible it comes from the Arduino? — David Bern
"Any ideas on where this '\x00' comes from." -- From Linux man page for termios: "If neither IGNPAR nor PARMRK is set, read a character with a parity error or framing error as \0." — sawdust

Ilias Patsiaouras Ilias Patsiaouras · Accepted Answer · 2018-11-02T12:17:39

I experienced the same behavior with similar setup in ubuntu using pyserial in Python 2.7. I am implementing a driver for Serial communication rs-232 with a device. As my software assume exclusive handling of the device I digged into it and found out that Linux does not support exclusive access to the serial port (or at least not straight forward). So in order to see what can go wrong with my driver (cause of the non exclusiveness), I opened thru python console the same port, while my driver was running. This gave the exact same problem you are describing, funny thing is even if I close the port in the console the problem persists. The problem will not stop unless you close completely the thread (in my case exit ipython), that was used to open the Serial port.

Python: '\x00' randomly appearing in serial from arduino to raspberry pi

1 Answers