25
votes

I am looking to write a quick script that will run through a csv file with two columns and provide me the rows in which the values in column B switch from one value to another:

eg:

dataframe:

# |  A  |  B  
--+-----+-----
1 |  2  |  3
2 |  3  |  3
3 |  4  |  4
4 |  5  |  4
5 |  5  |  4

would tell me that the change happened between row 2 and row 3. I know how to get these values using for loops but I was hoping there was a more pythonic way of approaching this problem.

3
Sorry are you asking for when 'A' does not equal 'B' on the same row? - EdChum
No sorry I am just looking to be able to tell for which rows the value in B changes. The values in A are what I want to look at when B changes but I have that part working. (I have updated the DF to hopefully make this clearer) - badrobit

3 Answers

42
votes

You can create a new column for the difference

> df['C'] = df['B'].diff()
> print df
   #  A  B   C
0  1  2  3 NaN
1  2  3  3   0
2  3  4  4   1
3  4  5  4   0
4  5  5  4   0

> df_filtered = df[df['C'] != 0]
> print df_filtered
   #  A  B  C
2  3  4  4  1

This will your required rows

6
votes

You can do the following which also works for non numerical values:

>>> import pandas as pd
>>> df = pd.DataFrame({"Status": ["A","A","B","B","C","C","C"]})
>>> df["isStatusChanged"] = df["Status"].shift(1, fill_value=df["Status"].head(1)) != df["Status"]
>>> df
  Status  isStatusChanged
0      A            False
1      A            False
2      B             True
3      B            False
4      C             True
5      C            False
6      C            False
>>> 

Note the fill_value could be different depending on your application.

2
votes

you can use this it is much faster, hope it helps!!

my_column_changes = df["MyStringColumn"].shift() != df["MyStringColumn"]