Python: numpy/pandas change values on condition

Question

I would like to know if there is a faster and more "pythonic" way of doing the following, e.g. using some built in methods. Given a pandas DataFrame or numpy array of floats, if the value is equal or smaller than 0.5 I need to calculate the reciprocal value and multiply with -1 and replace the old value with the newly calculated one. "Transform" is probably a bad choice of words, please tell me if you have a better/more accurate description.

Thank you for your help and support!!

Data:

import numpy as np
import pandas as pd
dicti = {"A" : np.arange(0.0, 3, 0.1), 
         "B" : np.arange(0, 30, 1),
         "C" : list("ELVISLIVES")*3}
df = pd.DataFrame(dicti)

my function:

def transform_colname(df, colname):
    series = df[colname]    
    newval_list = []
    for val in series:
        if val <= 0.5:
            newval = (1/val)*-1
            newval_list.append(newval)
        else:
            newval_list.append(val)
    df[colname] = newval_list
    return df

function call:

transform_colname(df, colname="A")

**--> I'm summing up the results here, since comments wouldn't allow to post code (or I don't know how to do it).**

Thank you all for your fast and great answers!!

using ipython "%timeit" with "real" data:

my function: 10 loops, best of 3: 24.1 ms per loop

from jojo:

def transform_colname_v2(df, colname):
    series = df[colname]        
    df[colname] = np.where(series <= 0.5, 1/series*-1, series)
    return df

100 loops, best of 3: 2.76 ms per loop

from FooBar:

def transform_colname_v3(df, colname):
    df.loc[df[colname] <= 0.5, colname]  = - 1 / df[colname][df[colname] <= 0.5]
    return df

100 loops, best of 3: 3.32 ms per loop

from dmvianna:

def transform_colname_v4(df, colname):
    df[colname] = df[colname].where(df[colname] <= 0.5, (1/df[colname])*-1)
    return df

100 loops, best of 3: 3.7 ms per loop

Please tell/show me if you would implement your code in a different way!

One final QUESTION: (answered) How could "FooBar" and "dmvianna" 's versions be made "generic"? I mean, I had to write the name of the column into the function (since using it as a variable didn't work). Please explain this last point! --> thanks jojo, ".loc" isn't the right way, but very simple df[colname] is sufficient. changed the functions above to be more "generic". (also changed ">" to be "<=", and updated timing)

Thank you very much!!

@jojo: Thank you!! Your solution is more pythonic, but slower. My function: 10000 loops, best of 3: 164 µs per loop. Your version: 1000 loops, best of 3: 395 µs per loop. — tryptofame
How does your speed test look like? It's hard to be faster than numpy... at least in python. ;) — jojo
@tryptofame just try to benchmark with a non-trivial sized frame, both solns below will be MUCH faster) — Jeff

FooBar FooBar · Accepted Answer · 2014-08-15T12:54:03

The typical trick is to write a general mathematical operation to apply to the whole column, but then use indicators to select rows for which we actually apply it:

df.loc[df.A < 0.5, 'A']  = - 1 / df.A[df.A < 0.5] 

In[13]: df
Out[13]: 
            A   B  C
0        -inf   0  E
1  -10.000000   1  L
2   -5.000000   2  V
3   -3.333333   3  I
4   -2.500000   4  S
5    0.500000   5  L
6    0.600000   6  I
7    0.700000   7  V
8    0.800000   8  E
9    0.900000   9  S
10   1.000000  10  E
11   1.100000  11  L
12   1.200000  12  V
13   1.300000  13  I
14   1.400000  14  S
15   1.500000  15  L
16   1.600000  16  I
17   1.700000  17  V
18   1.800000  18  E
19   1.900000  19  S
20   2.000000  20  E
21   2.100000  21  L
22   2.200000  22  V
23   2.300000  23  I
24   2.400000  24  S
25   2.500000  25  L
26   2.600000  26  I
27   2.700000  27  V
28   2.800000  28  E
29   2.900000  29  S

Python: numpy/pandas change values on condition

3 Answers