Cannot receive large amount of serial data from Arduino

Question

I am trying to receive sensor data from Arduino and write the readings into a file using Processing IDE (using serial communication / USB).

After doing a large amount of tests, I am pretty sure it's the Processing side unable to process the data. Only the first few (< 100) "samples" are written successfully, after that Serial.available() always returns false.

I am sending two-byte chunks from Arduino, 57600 baud, default settings (8 bit, no parity, 1 stop bit).

Arduino code:

unsigned int data = 0;
unsigned char buf[2];

void setup() {
  Serial.begin(57600);
}

void loop() {

  data = analogRead(A0);

  buf[0] = data & 0xFF; // low byte
  buf[1] = data >> 8;   // high byte

  Serial.write(buf, 2);
  Serial.flush();

  delayMicroseconds(300);
}

Processing code:

import processing.serial.*;

Serial serialPort;
String serialData;
PrintWriter output;

int recordingTime = 1000;          // how many miliseconds of data stream to record
byte[] dataBuffer = new byte[2];   // reserve memory for 2 bytes and initialize to 0 (java stuff)
int receivedBytes = 0;

void setup()
{
  serialPort = new Serial(this, "/dev/ttyUSB0", 57600);
  output = createWriter("serialData.txt");
}

void draw()
{

  while(millis() < recordingTime) {

    if (serialPort.available() > 0) {
      receivedBytes = serialPort.readBytes(dataBuffer);

      output.print("\n");
      output.print("Received bytes: ");
      output.print(receivedBytes);
      output.print("\n");

      output.println(binary(dataBuffer[0]));  // low byte
      output.println(binary(dataBuffer[1]));  // high byte
      output.println("");
    }    
    else {
      output.print("\n");
      output.println("No data available");
    }

  }

  output.flush();
  output.close();
  exit();  
}

Output:

Received bytes: 2
11101001
00000011


Received bytes: 2
11101001
00000011


Received bytes: 2
11101001
00000011

...after some lines...

No data available

No data available

No data available

No data available

No data available

No data available

Why is this happening? Why is there "no data available" after few samples? If I watch the serial monitor output in Arduino IDE, it works fine.

Jurc192 Jurc192 · Accepted Answer · 2018-03-18T17:39:15

I can read the serial data from Arduino using screen and using Python. Still cannot get it to work in Processing - only receiving few samples (17 exactly).

Screen command: $ screen <port name> <baud rate> (press ctrl+a and then shift+k to stop the program; add -L flag for logging to a file). Works as expected.

I managed to achieve the same results using Pyserial library in Python:

#
#   log_serial.py
#   Writes incoming serial data to file.

import time
import serial

# Edit this parameters =========================================================

serialPort = "/dev/ttyUSB0"
baudrate = 57600
recordTime = 1000   # milliseconds

# ==============================================================================


def millis():
    """
    Returns current (wall-)time in milliseconds
    """
    return int(round(time.time() * 1000))


ser = serial.Serial(serialPort, baudrate)

with open("output.txt", "w") as f:

    startTime = millis()

    f.write("Recording started at: ")
    f.write(str(startTime))
    f.write("\n")

    while (millis() - startTime) <= recordTime:
        inData = ser.read(2)                                        # reads two bytes
        inInt = int.from_bytes(inData, byteorder='little')          # merges them into an integer

        f.write(str(inInt))
        f.write("\n")

    f.write("Recording finished at: ")
    f.write(str(millis()))
    f.write("\n")

Still don't know why the Processing version can't handle it...
There was a bug (wrong while loop condition) in my first Processing code. Here is updated version, using Serial.read() rather than Serial.readBytes(buffer). Still doesn't solve my problem, only getting 17 samples:

import processing.serial.*;

Serial serialPort;
String serialData;
PrintWriter output;

int recordingTime = 5000;          // how many miliseconds of data stream to record

void setup()
{
  serialPort = new Serial(this, "/dev/ttyUSB0", 57600);
  output = createWriter("serialData.txt");
}

void draw()
{

  int startTime = millis();
  while((millis() - startTime) <= recordingTime) {

    if (serialPort.available() > 0) {

      int b1 = serialPort.read();
      int b2 = serialPort.read();

      int value = (b2 << 8) + (b1 & 0xFF);
      output.println(value);
    }

  }

  output.flush();
  output.close();
  exit();  
}

Cannot receive large amount of serial data from Arduino

3 Answers