R Create new data frame for each unique id

2
votes

I have created a feature vector (data.frame) that has an id, feat1, feat2, feat3, boolean, but in this data frame there are duplicates of ids, which is done purposefully. What I want to do is as I iterate over this data frame build new data frame per id.

For simplicity lets assume I have following two columns.

          X1         X2      X3
1   000000001 -1.4061361     1
2   000000001 -0.1973846     1
3   000000002 -0.4385071     1
4   000000001 -0.6593677     0
5   000000001 -1.2592415     0
6   000000001 -0.5463655     1
7   000000002  0.4231117     0
8   000000002 -0.1640883     1
9   000000002  0.7157506     0
10  000000002  2.3234110     1

I want to build different data frame based on X1 basically I want to get all the same X1 into their own data frames. I wrote using multiple for loops but It takes super long time since this is a large data set. What is the best way to do this?

3
rdataframe
Note that creating all these copies will double your memory usage, at least. So if you plan to do some analysis on each chunk and save only a small set of summary results, check out function by(). - Ferdinand.kraft
@Ferdinand.kraft Yes I plan on doing analysis infact reason I am doing this I want run randomforest on each so I was actually worried about the memory consumption. How do you suggest I use by on this case? - add-semi-colons
@find-missing-semicolon sorry, I don't use randomforest... But by() accepts any function that works on a dataframe chunk and returns summarized data. - Ferdinand.kraft

3 Answers

3
votes

As suggested in the comments, use split. If you really want to have new objects created, use split in conjunction with list2env as follows:

## What is in the workspace presently?
ls()
# [1] "mydf"

## This is where most R users would probably stop
split(mydf, mydf$X1)
# $`1`
#   X1         X2 X3
# 1  1 -1.4061361  1
# 2  1 -0.1973846  1
# 4  1 -0.6593677  0
# 5  1 -1.2592415  0
# 6  1 -0.5463655  1
# 
# $`2`
#    X1         X2 X3
# 3   2 -0.4385071  1
# 7   2  0.4231117  0
# 8   2 -0.1640883  1
# 9   2  0.7157506  0
# 10  2  2.3234110  1

The above command creates a list, which is a very convenient format to have if you are going to be doing similar calculations on each list item. Most R users would stop there. If you really need separate objects in your workspace, use list2env:

list2env(split(mydf, mydf$X1), envir=.GlobalEnv)
# <environment: R_GlobalEnv>

## How many objects do we have now?
ls()
# [1] "1"    "2"    "mydf"

Note that these names are not syntactically valid, so you need to use backticks (</code>) to access them. (Or, alternatively,get("1")`).

`1`
#   X1         X2 X3
# 1  1 -1.4061361  1
# 2  1 -0.1973846  1
# 4  1 -0.6593677  0
# 5  1 -1.2592415  0
# 6  1 -0.5463655  1
`2`
#    X1         X2 X3
# 3   2 -0.4385071  1
# 7   2  0.4231117  0
# 8   2 -0.1640883  1
# 9   2  0.7157506  0
# 10  2  2.3234110  1
1
votes

This uses one for loop - better?

ids <- unique(df$X1)

for(i in 1:length(ids)){
    id <- ids[i]
    mini.df <- data.frame(df[df$X1 == id, ])
    assign(paste("mini.df", i, sep="."), mini.df)
    # or alternatively, if you wanted the data.frames to be assigned by id, 
    # assign(id, mini.df)
}
0
votes

It sounds like you want to be able to fit models to each subset of data (and likely extract summaries of the models). You can use broom, dplyr, purrr and tidyr to do this functionally. Here's an example:

library(broom)
library(dplyr)
library(purrr)
library(tidyr)

mtcars %>%
  group_by(cyl) %>%
  nest() %>%
  mutate(model = map(data, lm, formula = mpg ~ disp + hp),
         results = map(model, tidy)) %>%
  unnest(results)