Python-Arduino Serial Half-Duplex: Arduino reads previous serial input instead of current

Question

I am using Python 3.7.5 with the latest version of serial library. I am trying to make an RFID authentication device via python and arduino. User has to scan their ID in the RFID connected to the arduino and the arduino must send the UID to the python software. In my laptop, a thread listening for serial is running. It checks the UID and it will send 'O' if it is allowed and 'X' if it is not. In the arduino, the program then waits if there is data sent through the serial then checks the input if it is 'O' or not. If the RX is 'O' then the LED must light green, otherwise red.

My problem is that when I first scan the CORRECT uid, it goes green, no problem. If I scan another CORRECT uid it goes green again, no problem. If I scan an INCORRECT uid, it goes green, but in my python code it SHOULD BE RED. Then if I scan a CORRECT uid, it goes RED whereas is should be GREEN. I tried adding delays to both arduino and python to wait for the previous input to clear and also tried flushing after transmission with no luck.

tl;dr The arduino is outputting results ONE uid scan late and I don't know what else to do.

Python:

# Reads data from the serial monitor without interrupting the main thread

from serial import Serial
import time
from threading import Thread

class SerialListener:
    def __init__(self, baudrate=9600, timeout=1):
        try:
            self.ser = Serial('/dev/ttyACM0', baudrate, timeout=timeout)
        except:
            self.ser = Serial('/dev/ttyACM1', baudrate, timeout=timeout)

        self.stopped = False
        self.paused = False
        self.stream = ''
        time.sleep(1) # Wait for serial buffer to reset

    def start(self):
        Thread(target=self.update, args=()).start()
        return self

    def update(self):
        if not self.paused:
            while True:
                if self.stopped:
                    self.ser.close()
                    print("Serial Thread Stopped")
                    print("Serial Port Closed")
                    break
                try:
                    self.stream = self.ser.readline().decode('utf-8')
                except:
                    self.stream = self.ser.readline().decode('ascii')
                self.stream = self.stream.rstrip()

    def stop(self):
        self.stopped = True

    def pause(self):
        self.paused = True

    def flush(self):
        self.ser.flush()

    def readDistance(self):
        try:
            return float(self.stream)
        except:
            return -1   # Returns -1 if there is an error in reading

    def readRFID(self):
        return self.stream

    def write(self, msg):
        self.ser.write(msg.encode())

if __name__ == "__main__": # FOR DEBUGGING ONLY
    uno = SerialListener().start()
    uno.flush()
    print("Serial Started")
    uid = ''
    while True:
        uid = uno.readRFID()
        if uid is not '':
            uno.flush()
            time.sleep(0.1)
            if uid == "5BEE9F0D":
                uno.write('O')
                print("SHOULD BE GREEN")
            else:
                uno.write('X')
                print("SHOULD BE RED")
            print(uid)

    uno.stop()

Arduino:

#include <MFRC522.h>
#define GREEN_LED 6
#define RED_LED 7
#define BUZZER 8

MFRC522 rfid(10, 9);

unsigned long timer = 0;
bool readStatus = false;

void setup() {
  pinMode(RED_LED, OUTPUT);
  pinMode(GREEN_LED, OUTPUT);
  Serial.begin(9600);
  SPI.begin();
  rfid.PCD_Init();
  for(int i = 0; i < 10; i++)
    Serial.write('\n');
  delay(5);
  digitalWrite(RED_LED, HIGH);
}

void loop() {
  while(!readStatus){
    if(rfid.PICC_IsNewCardPresent()){
      if(rfid.PICC_ReadCardSerial()){
        byte uid[rfid.uid.size];
        if((millis() - timer) > 1000){
          for(int i = 0; i < rfid.uid.size; i++)
            uid[i] = rfid.uid.uidByte[i];

          for(int i = 0; i < sizeof(uid); i++){
            if(uid[i] < 0x10)
              Serial.print('0');

            Serial.print(uid[i], HEX);
          }
          Serial.println();
          readStatus = true;
          timer = millis();
        }
        Serial.flush();
      }
    }
  }

  if(readStatus){
    while(!Serial.available());

    char rx = 'X';

    while(Serial.available()){
      rx = Serial.read();
    }

    if(rx == 'O'){
    digitalWrite(GREEN_LED, HIGH);
    digitalWrite(RED_LED, LOW);
    tone(BUZZER, 2500);
    delay(100);
    noTone(BUZZER);
    readStatus = false;
    }
    else{
      digitalWrite(RED_LED, LOW);
      digitalWrite(GREEN_LED, LOW);
      tone(BUZZER, 1000);
      delay(50);
      noTone(BUZZER);
      delay(30);
      tone(BUZZER, 1000);
      delay(50);
      noTone(BUZZER);
      digitalWrite(RED_LED, HIGH);
      readStatus = false;
    }
  }
}

Output:

Serial Started

SHOULD BE RED
05520320 // it is red

SHOULD BE RED
05520320 // it is red

SHOULD BE GREEN
5BEE9F0D // it is red

SHOULD BE GREEN
5BEE9F0D // it is green

SHOULD BE RED
05520320 // it is green

"The arduino is outputting results ONE uid scan late" -- No, you have not provided any evidence of that, but rather that you are occasionally seeing incorrect responses, e.g. "it" is red when "it" should be green. You then conclude this incorrect response is due to a "delay". — sawdust
It's not due to a delay, what I meant was the arduino reads the previous reading from the python instead of what it recently sent to the arduino. For comparison, it is like this: pythonData = O; serialData = X; digitalWrite(LED, serialData); serialData = pythonData;. In other words, it outputs the previous serial input instead of the current serial input. — Rex Endozo
Okay, when I scan ID A, It sends the UID of A to python through SPI. The python reads the SPI and when it receives the UID of A, it checks if the UID is allowed. In this case we define ID A as allowed so python sends 'O' to arduino through SPI. Arduino is doing nothing until it receives serial data from python, then comes the 'O'. The led will light green since the data received is 'O'. Now back to the scanning, repeat all the process but this time scan ID B and python sends 'X' but in the arduino, the RX is apparently 'O'. Sadly, I cannot look at both serial monitors to check what uno receive — Rex Endozo
"In other words, it outputs the previous serial input instead of the current serial input." -- What is "it"? Where do you report what the "current serial input" is? Where is the evidence to backup your assertions? Your Arduino logic is not robust. if(rx == 'O') ... else is problematic because it does not validate one case of the input before the action is performed. BTW SPI is not a "serial port" . — sawdust
Hmmm I think it must be because python writes to the serial more than once as the condition is set to if uid is ''. I'm sorry I forgot to mention this, but the python reads the uid multiple times before it receives nothing. It must be because of this and I think fixing this will fix the problem. I'm trying it now. — Rex Endozo

Rex Endozo Rex Endozo · Accepted Answer · 2019-12-04T07:14:22

Okay I solved the problem by modifying the timeout in pyserial and adding a 1 second delay between reading.

The problem was that the pyserial writes to the serial more than once because of how I set the condition.

while True:
    uid = uno.readRFID()
    if uid is not '':
        uno.flush()
        time.sleep(0.1)
        if uid == "5BEE9F0D":
            uno.write('O')
            print("SHOULD BE GREEN")
        else:
            uno.write('X')
            print("SHOULD BE RED")
        print(uid)

Since the timeout I set was 1, PySerial will continue to read the UID until timeout. This will cause the uid to be not equal to '' for a second and will send data to the serial multiple times unnecessarily.

In the Arduino, it only reads one byte of character from the buffer and after reading it, removes what it read from the buffer. However, python didn't send only 1 character to the serial, but more than once. That is why the Arduino outputs incorrectly as it reads the character from the PREVIOUS buffer instead of the new character that the python sent to the serial

To solve this problem, I made the timeout lesser so that the input of the serial will clear after timeout has reached. Do not set the timeout value too low or else the data might not be read.

def __init__(self, baudrate=9600, timeout=0.5):

Secondly, I added a delay between reading from serial on the main thread

uid = ''
while True:
    while uid is '':
        uid = uno.readRFID()
    time.sleep(0.1)

    if uid == "5BEE9F0D":
        uno.write('O')
    else:
        uno.write('X')
    print(uid)
    time.sleep(1)
    uid = ''

The code in the Arduino works so I didn't change that, the only problem was within the python code itself.

Python-Arduino Serial Half-Duplex: Arduino reads previous serial input instead of current

1 Answers