Subset common rows from multiple data frames

Question

I have multiple dataframes like mentioned below with unique id for each row. I am trying to find common rows and make a new dataframe which is appearing at least in two dataframes.

example- row with Id=2 is appearing in all three dataframes. similarly row with Id= 3 is there in df1 and df3.

I want to make a loop which can find common rows and create a new dataframe with common rows.

df1 <- data.frame(Id=c(1,2,3,4),a=c(0,1,0,2),b=c(1,0,1,0),c=c(0,0,4,0)) 
df2 <- data.frame(Id=c(7,2,5,9),a=c(4,1,9,2),b=c(1,0,1,5),c=c(3,0,7,0))
df3 <- data.frame(Id=c(5,3,2,6),a=c(9,0,1,5),b=c(1,1,0,0),c=c(7,4,0,0)) 

> df1                   > df2 
 Id | a | b | c |         Id | a | b | c |
 ---|---|---|---|         ---|---|---|---|                  
  1 | 0 | 1 | 0 |          7 | 4 | 1 | 3 |                           
 ---|---|---|---|         ---|---|---|---|                  
  2 | 1 | 0 | 0 |          2 | 1 | 0 | 0 |
 ---|---|---|---|         ---|---|---|---|
  3 | 0 | 1 | 4 |          5 | 9 | 1 | 7 |
 ---|---|---|---|         ---|---|---|---|
  4 | 2 | 0 | 0 |          9 | 2 | 5 | 0 |

 > df3
 Id | a | b | c |
 ---|---|---|---|
  5 | 9 | 1 | 7 |
 ---|---|---|---|
  3 | 0 | 1 | 4 |
 ---|---|---|---|
  2 | 1 | 0 | 0 |
 ---|---|---|---|
  6 | 5 | 0 | 0 |

> expected_output
 Id | a | b | c |
 ---|---|---|---|
  5 | 9 | 1 | 7 |
 ---|---|---|---|
  3 | 0 | 1 | 4 |
 ---|---|---|---|
  2 | 1 | 0 | 0 |
 ---|---|---|---|

Note:- ID is unique. Also, i want to remove rows from original dataframes which are duplicated and I am using it to create new dataframe.

Frank Frank · Accepted Answer · 2017-07-28T19:58:38

I have multiple dataframes like mentioned below with unique id for each row. I am trying to find common rows and make a new dataframe which is appearing at least in two dataframes.

Since no ID appears twice in the same table, we can tabulate the IDs and keep any found twice:

library(data.table)

DTs = lapply(list(df1,df2,df3), data.table)

Id_keep = rbindlist(lapply(DTs, `[`, j = "Id"))[, .N, by=Id][N >= 2L, Id]

DT_keep = Reduce(funion, DTs)[Id %in% Id_keep]

#    Id a b c
# 1:  2 1 0 0
# 2:  3 0 1 4
# 3:  5 9 1 7

Your data should be in an object like DTs to begin with, not a bunch of separate named objects.

How it works

To get a sense of how it works, examine intermediate objects like

list(df1,df2,df3)
lapply(DTs, `[`, j = "Id")
Reduce(funion, DTs)

Also, read the help files, like ?lapply, ?rbindlist, ?funion.

Subset common rows from multiple data frames

2 Answers