matlab working with long time series

Question

I have a datset which starts similar to the following:

year month  daynum  hour     depth        wtemp
1989    5   136      0          0.01     14.32
1989    5   136      0          0.5      14.32
1989    5   136      0          1        13.52
1989    5   136      0          2        12.31
1989    5   136      0          3        10.16
1989    5   136      0          4        8.61
1989    5   136      0          6        6.86
1989    5   136      0          8        6.03
1989    5   136      0          10       5.45
1989    5   136      1          0.01     13.38
1989    5   136      1          0.5      13.41
1989    5   136      1          1        13.12
1989    5   136      1          2        10.88
1989    5   136      1          3        10.09
1989    5   136      1          4        9.09
1989    5   136      1          6        7.24
1989    5   136      1          8        6.45
1989    5   136      1          10       5.67
1989    5   136      2          0.01     13.2
1989    5   136      2          0.5      13.24
1989    5   136      2          1        13.24
1989    5   136      2          2        11.07
1989    5   136      2          3        10.14
1989    5   136      2          4        9.02
1989    5   136      2          6        7.23
1989    5   136      2          8        6.23
1989    5   136      2          10       5.59
1989    5   136      3          0.01     13.11
1989    5   136      3          0.5      13.15
1989    5   136      3          1        13.11
1989    5   136      3          2        11.26
1989    5   136      3          3        10.25
1989    5   136      3          4         9.2
1989    5   136      3          6         6.99
1989    5   136      3          8         5.96
1989    5   136      3          10        5.5

This data set contains over 1,000,000 rows of data. when I import the data into matlab I am left with this structure (although without the headers). I would like to know how I could return the indices for the minimum and maximum depth (i.e. column 5) for each individual time period?

The answer in this case would be something like:

1,9
10,18,

and so on.

Keep in mind that the number of measurements and therefore the number of depths will vary throughout the period of investigation, therefore I will have to use the information in year, month, daynum, hour, and depth.

The main aim here is to build a matrix which contains only the temperature recorded at the minimal and maximum depth for each hour.

So far I have tried (data is the name of the matrix):

[~,~,b] = unique(data(:,1:4),'rows');
MaxMin_temp = [abs(accumarray(b,data(:,end),[],@min)), abs(accumarray(b,data(:,end),[],@max))];

which returns the temperature at the minimum and maximum depth for each individual day. From this I was hoping of a more sophisticated way of obtaining the entire columns for these values.

What have you tried so far ? You've asked a fair number of Matlab questions recently, you are slowly getting SO to teach you, what have you learned, what can you contribute back to the community ? — High Performance Mark
I've asked 3 questions in 2 weeks. Is this not acceptable? I generally don't provide my attempts in the question, I thought that a long winded question would scare people away from answering it, but If I need to show my attempts that is fair enough. — KatyB
Yes, the rate at which you ask questions is perfectly acceptable. But by hiding your own efforts to answer your questions you invite snippy comments such as mine (above) which accuse you of not trying hard enough before asking SO to help. SO loves a tryer, the only way to show us that you are one such and therefore to be loved, is to show us what you have tried. Spend some time looking around SO, especially at the questions which get good answers. — High Performance Mark
Getting the index from min/max when used with accumarray was recently covered here: stackoverflow.com/questions/15440874/… — H.Muster

Shai Shai · Accepted Answer · 2013-03-20T12:12:55

use datenum to assign each time period an integer index.
Then you can use accumarray twice: once with @min and once with @max as the "accumulating" function to compute the min (max resp.) ot the binned values of the depth or temperature columns.

matlab working with long time series

1 Answers