Reading file starting at end in MATLAB

Question

I was wondering if anyone knows how to open and read from a file in MATLAB where you begin reading from the end of the file. The file is constantly being updated (at some nonconstant rate between reads) and I want to read the last six lines of the file each time.

I would also to include a test to verify that I don't reread the same lines twice. Each line is formatted as follows (each variable is a floating point number):

timestamp accx accy accz gyrox gyroy gyroz magx magy magz

I was trying to use fseek, to change the position to the last line of the file, but this only allows me to read the last line of the file I think, not read the file backwards unless I specify a certain number of bytes, which I wouldn't know the number of bytes exactly.

Answer 1

If you're on a unix based system (Linux/Mac), you can directly use system commands to do what you want. Here's a sample test file:

12345 accx accy accz gyrox gyroy gyroz magx magy magz
23456 accx accy accz gyrox gyroy gyroz magx magy magz
34567 accx accy accz gyrox gyroy gyroz magx magy magz
45678 accx accy accz gyrox gyroy gyroz magx magy magz
56789 accx accy accz gyrox gyroy gyroz magx magy magz
67890 accx accy accz gyrox gyroy gyroz magx magy magz

You can read it using tail on unix and into MATLAB directly using the system command.

[~, str]=system('tail -n 2 filename')
str =

    56789 accx accy accz gyrox gyroy gyroz magx magy magz
    67890 accx accy accz gyrox gyroy gyroz magx magy magz

Replace the 2 in -n 2 with how many ever lines you want to read.

Next, to make sure you read the same line, you might want to store the timestamps (first column). The simplest way to do that is again let unix do it for you

[~, timestamp]=system('tail -n 2 filename | awk ''{print $1}''')

timestamp =

56789
67890

Convert it to numbers using str2num and store these each time you read and then use the function ismember to check if a new timestamp is already part of your previously read timestamps.

Answer 2

The only way to know what line you are reading from a file is to start at the beginning and count newlines. There is no way to start at the end of the file and directly seek backwards by a certain number of lines.

You could write a function that reads backwards from the end of the file until it sees N newlines, and then outputs this chunk. This is precisely how tail works. Here is a comment from GNU tail.c:

/* Print the last N_LINES lines from the end of file FD.
   Go backward through the file, reading `BUFSIZ' bytes at a time (except
   probably the first), until we hit the start of the file or have
   read NUMBER newlines.
   START_POS is the starting position of the read pointer for the file
   associated with FD (may be nonzero).
   END_POS is the file offset of EOF (one larger than offset of last byte).
   Return true if successful.  */

If the file is not too huge, the easiest thing to do would be to write a function that reads a line at a time, only keeping the last N lines.

Answer 3

You could search the end chunk of the file (using fseek() to set the position for the EOF well in advance of your 6 lines and then search for the last 6 or 7 newline characters and with find(). Then you could extract your data because you know where the last 6 lines are in the original chunk that you read.

fid=fopen(filename,'r');
fseek(fid,500,'eof');
dat=fread(fid,Inf,'*char');
linestart=find(dat=="\n",7,'last'); % choose 7 newlines because there will be one at the end?