Constantly reading from a serial port with a background thread

Question

As serial port communication is asynchronous, I figured out early on into my project involving communication with a RS 232 device that I will have to have a background thread constantly reading the port for data received. Now, I'm using IronPython (.NET 4.0) so I have access to the slick SerialPort class built into .NET. This lets me write code like this:

self.port = System.IO.Ports.SerialPort('COM1', 9600, System.IO.Ports.Parity.None, 8, System.IO.Ports.StopBits.One)
self.port.Open()
reading = self.port.ReadExisting() #grabs all bytes currently sitting in the input buffer

Simple enough. But as I mentioned I want to be constantly checking this port for new data as it arrives. Ideally, I would have the OS tell me anytime there's data waiting. Whaddaya know, my prayers have been answered, there is a DataReceived event provided!

self.port.DataReceived += self.OnDataReceived

def OnDataReceived(self, sender, event):
    reading = self.port.ReadExisting()
    ...

Too bad this is worthless, though, because this event isn't guaranteed to be raised!

The DataReceived event is not guaranteed to be raised for every byte received.

So back to writing a listener thread, then. I've accomplished this rather quickly with a BackgroundWorker that just calls port.ReadExisting() over and over again. It reads bytes as they come in, and when it sees a line ending (\r\n), it places what it read into a linebuffer. Then other parts of my program look at the linebuffer to see if there are any complete lines waiting to be used.

Now, this is a classic producer-consumer problem, obviously. The producer is the BackgroundWorker, placing complete lines into linebuffer. The consumer is some code that consumes those lines from the linebuffer as fast as possible.

However, the consumer is sort of inefficient. Right now he's constantly checking the linebuffer, getting disappointed each time to find it empty; though every once in a while does find a line waiting inside. What's the best way to optimize this so that the consumer only wakes up when there is a line available? That way the consumer isn't spinning around constantly accessing the linebuffer, which might introduce some concurrency issues.

Also, if there is a simpler/better way of reading constantly from a serial port, I'm open to suggestions!

Answer 1

I don't see why you can't use the DataReceived event. As the docs state

The DataReceived event is not guaranteed to be raised for every byte received. Use the BytesToRead property to determine how much data is left to be read in the buffer.

All that is saying is that you aren't guaranteed to get a separate event for each individual byte of data. You may need to check and see if there more than one byte available on the port using the BytesToRead property and read the corresponding number of bytes.

Answer 2

There is no need to have a spinning thread that polls the serial port.

I suggest using SerialPort.BaseStream.BeginRead(...) method. It is much better than trying to use the event in the SerialPort class. The call to BeginRead returns right away and registers an asynchronous callback that is invoked once a read is complete. In the callback method, you call EndRead and that returns the number of bytes read into the supplied buffer. The BaseStream(SerialStream) inherits from Stream and follows the general pattern for .Net Streams, which is very useful.

But it's important to keep in mind that it is a .Net thread that invokes the callback, so you need to processes the data fast, or pass off any heavy lifting to one of your own threads. I highly suggest reading the following link, particularly the Remarks section.

http://msdn.microsoft.com/en-us/library/system.io.stream.beginread.aspx

Answer 3

Why doesn't your producer raise an event when a full line is placed in the LineBuffer

Also, rom reading the docs, I believe it's simply saying that reading 100 bytes != 100 events raised - it's not saying the events can't be relied upon.

Answer 4

Here is some VB code that expresses my opinion on how this should be done:

Dim rcvQ As New Queue(Of Byte()) 'a queue of buffers
Dim rcvQLock As New Object

Private Sub SerialPort1_DataReceived(ByVal sender As System.Object, _
                                     ByVal e As System.IO.Ports.SerialDataReceivedEventArgs) _
                                 Handles SerialPort1.DataReceived

    'an approach
    Do While SerialPort1.IsOpen AndAlso SerialPort1.BytesToRead <> 0
        'what if the number of bytes available changes at ANY point?
        Dim bytsToRead As Integer = SerialPort1.BytesToRead
        Dim buf(bytsToRead - 1) As Byte

        bytsToRead = SerialPort1.Read(buf, 0, bytsToRead)

        'place the buffer in a queue that can be processed somewhere else
        Threading.Monitor.Enter(rcvQLock)
        rcvQ.Enqueue(buf)
        Threading.Monitor.Exit(rcvQLock)

    Loop

End Sub

For what it is worth, code very similar to this has processed serial port data at near 1Mbps.