2
votes

With pandas 0.19.2 python 3.6.0 DataFrame.replace with a dictionary acts on substrings (like "find"), and so does Series.replace. Pandas 0.24.0 python 3.6.8 seems to act on the entire string (like "match") for DataFrames, and still act on substrings for Series (like "find").

df = pd.DataFrame({'c1':['AD','BD'],'c2':['AD','BD']})
print(df)
print(df.replace(to_replace={'c1':{r'D': ''}, 'c2':{r'BD': ''}},regex=True))
print(df.replace(to_replace={r'D': ''},regex=True))
print(df['c1'].replace(to_replace=r'D', value='',regex=True))

Pandas 0.19.2 produces (I added some blank lines for legibility):

   c1  c2
0  AD  AD
1  BD  BD

  c1  c2
0  A  AD
1  B    

  c1 c2
0  A  A
1  B  B

0    A
1    B
Name: c1, dtype: object

With Pandas 0.24.0:

   c1  c2
0  AD  AD
1  BD  BD

   c1  c2
0  AD  AD
1  BD    

   c1  c2
0  AD  AD
1  BD  BD

0    A
1    B
Name: c1, dtype: object

Looks like a pandas bug to me, or am I missing something?

1
Its reported here - Vaishali
Vaishali, the thread you link is concerning a somewhat different issue, though it's clearly related. As you can see my example does not use integers at all, which is what the linked issue is about. This issue here is about regex now matching the whole string ("match") vs matching a substring ("find"). All this is for strings, no integers involved. - Uwe Mayer

1 Answers

1
votes

The bug is listed among the Fixed regressions for Pandas 0.24.2:

Fixed regression in DataFrame.replace() where regex=True was only replacing patterns matching the start of the string (GH25259)

As you see, only

print(df.replace(to_replace={'c1':{r'D': ''}, 'c2':{r'BD': ''}},regex=True))
print(df.replace(to_replace={r'D': ''},regex=True))

did not work correctly. Now, the issue is fixed.