How to do a FIFO push-operation for rows on Pandas dataframe in Python?

1
votes

I need to maintain a Pandas dataframe with 500 rows, and as the next row becomes available I want to push that new row in and throw out the oldest row from the dataframe. e.g. Let's say I maintain row 0 as newest, and row 500 as oldest. When I get a new data, I would push data to row 0, and it will shift row 0 to row 1, and so on until it pushes row 499 to row 500 (and row 500 gets deleted).

Is there a way to do such a FIFO operation on Pandas? Thanks guys!

1
pythonpandas
1. do you mean, that you read data from a fifo? 2. using dataframe as a queue is very inefficient, because it takes O(N) time to insert/remove a row in your scenario. Just use a queue. - Eli Korvigo
You could df.shift(1); df.loc[0] = new_row. That'll be slow tho. - Zero
@JohnGalt, yes sir! That is exactly what I was looking for. Thanks a lot. - rrlamichhane
@JohnGalt df.shift(1); df.iloc[0] = new_row - piRSquared

1 Answers

5
votes

@JohnGalt posted an answer to this on the comments. Thanks a lot. I just wanted to put the answer here just in case if people are looking for similar information in the future.

df.shift(1) df.loc[0] = new_row

df.shift(n) will shift the rows n times, filling the first n rows with na and getting rid of last n rows. The number of rows of df will not change with df.shift.

I hope this is helpful.