How to synchronize readout of binary streams on serial port of Matlab

Question

I'm having an issue which is partially Matlab- and partially general programming-related, I'm hoping that somebody can help me brainstorm for solutions.

I have an external microcontroller that generates a large stream of binary data (~40kb) every 400ms and sends it via UART to a PC running Matlab scripts. The data is not encoded in hexa or dec characters, but true binary (hence, there's no terminator defined as all 256 values are possible, valid combinations of data). Baudrate is set at 1024000. In short, it takes roughly 375ms for a whole stream of data to be sent, with 25ms of dead time in between streams

In Matlab, the serial port is configured correctly (also 1024000, 8x bits, 1x stop bit, no parity, no hardware flow control, etc.). I am able to readout the data I'm sending via the microcontroller correctly (i.e. there's no corruption of data), but I'm not being able to synchronize the serial readout on Matlab. My script is as follows:

function data_show = GetDATA

if ~isempty(instrfind)
    fclose(instrfind);
end

DATA_TOTAL_SIZE = 38400;

DATA_buffer = uint8(zeros(DATA_TOTAL_SIZE,1));
DATA_show = reshape(DATA_buffer(1:2:end)',[160,120])';
f_data_in = false;
f_data_out = true;

serialport = serial('COM11','BaudRate',1024000,'DataBits',8,'FlowControl','none','Parity','none','StopBits',1,...
            'BytesAvailableFcnCount',DATA_TOTAL_SIZE,'BytesAvailableFcnMode','byte','InputBufferSize',DATA_TOTAL_SIZE * 2,...
            'BytesAvailableFcn',@GetPortData);

fopen(serialport);

while (get(serialport,'BytesAvailable') ~= 0)   % Skip first packet which might be incomplete
    fread(serialport,DATA_TOTAL_SIZE,'uint8');
end

f_data_out = true;

while (1)
    if (f_data_in)
        DATA_buffer = fread(serialport,DATA_TOTAL_SIZE,'uint8');
        DATA_show = reshape(DATA_buffer(1:2:end)',[160,120])';  %Reshape array as matrix
        DATAsc(DATA_show);
        disp('DATA');
    end
    pause(0.01);
end

fclose(serialport);
delete(serialport);

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

function GetPortData (obj,~)

    if f_data_out
        f_data_in = true;
    end

end
end

The problem I see is that what I end up reading is always the correct size, but belongs to multiple streams, because I haven't found a way to tell Matlab that these 25ms of no data should be used to synchronize (i.e. data from before and after that blank period should belong to different streams).

Does anyone have any suggestions for this?

Thanks a lot!

JasonBourne JasonBourne · Accepted Answer · 2015-03-14T17:37:32

For completeness, I would like to post the current implementation I have fixing this issue, which is probably not a suitable solution in all cases but might be useful in some.

The approach I took consists in moving into a bi-directional communication protocol, in which Matlab initiates the streaming by sending a very short command as a trigger (e.g. single, non-printable character). Given the high baudrate it does not add significant delay due to processing in the microcontroller's side.

The microcontroller, upon reception of this trigger, proceeds to transmit only one full package (as opposed to continuously streaming package at a 5Hz rate). By forcing Matlab to pickup a serial package of the known length right after issuing the trigger, it ensures that only one package and without synchronization issues is received.

Then it becomes just a matter of encapsulating the Matlab script in a routine with a 5Hz tick given by a timer, in which the sequence is repeated (send trigger, retrieve package, do whatever processing, and repeat).

Advantages of this:

It solves the synchronization problems

Disadvantages of this:

Having Matlab running on a timer tick does not ensure perfect periodicity, and hence the triggers might not always be sent at exactly 5Hz. If triggers are sent at "inconvenient" times for the microcontroller, packages might need to be skipped in order to avoid that a package is updated in memory while it is still being transmitted (since transmission takes a significant part of the 200ms time slot)
From experience, performance can vary a lot depending on what the PC running Matlab is doing. For example, it works fine when the PC is left on its own to do the acquisition, but if another program is used (e.g. Chrome), Matlab begins to lag and that results in delays in transmission of triggers.

As mentioned above, it's not a complete answer, but it is an approach that might be sufficient in some situations. If someone has a more efficient option, please fell free to share!

How to synchronize readout of binary streams on serial port of Matlab

1 Answers