42
votes

I am creating a groupby object from a Pandas DataFrame and want to select out all the groups with > 1 size.

Example:

     A  B
0  foo  0
1  bar  1
2  foo  2
3  foo  3

The following doesn't seem to work:

grouped = df.groupby('A')
grouped[grouped.size > 1]

Expected Result:

A
foo 0
    2
    3
4
give us a concrete example, and show what you have tried. - root
Hopefully some help: grouped.size().apply(lambda x: x>1), but I'm not sure how to do this - Andy Hayden
this is interesting ..for a change I have hit a area where a feature needed by me is missing in Pandas ..for long it was my understanding of it that was missing ..great library for what I do.. - Abhi

4 Answers

45
votes

As of pandas 0.12 you can do:

>>> grouped.filter(lambda x: len(x) > 1)

     A  B
0  foo  0
2  foo  2
3  foo  3
11
votes

I have found transform to be much more efficient than filter for very large dataframes:

element_group_sizes = df['A'].groupby(df['A']).transform('size')
df[element_group_sizes>1]

Or, in one line:

df[df['A'].groupby(df['A']).transform('size')>1]
7
votes

If you still need a workaround:

In [49]: pd.concat([group for _, group in grouped if len(group) > 1])
Out[49]: 
     A  B
0  foo  0
2  foo  2
3  foo  3
0
votes

You can use the method filter and the property shape:

df.groupby('A').filter(lambda x: x.shape[0] > 1)