How to build a data frame from a nested list in R

1
votes

I have a function that queries a database and returns a list of two data frames (df1 and df2). If I lapply iteratively over that function, I return a list of nested lists with the two data frames.

The resultant list is structured as below:

#e.g. sample list of lists of 2 data frames
A1 <- data.frame(Value =c("A","B","C"))
A2 <- data.frame(Value =c("1","2","3"))
B1 <- data.frame(Value =c("D","E","F"))
B2 <- data.frame(Value =c("4","5","6"))
C1 <- data.frame(Value =c("G","H","I"))
C2 <- data.frame(Value =c("7","8","9"))
myList <- list( list(df1 = A1, df2 = A2),
            list(df1 = B1, df2 = B2),
            list(df1 = C1, df2 = C2))

I then want to combine the data frames into their own separate big data frames - df1_All and df2_All.

How can I extract all of the df1 data frames from the list and combine them into a larger data frame? I am thinking it would be to use make use of a do.call(rbind) construct with an apply or map function applied to myList?

3
rlistlapplypurrr
What is your expected output? Would every list contain 2 dataframes only and you want to combine them alternately always ? do.call("rbind", lapply(myList, "[[", 1)) and do.call("rbind", lapply(myList, "[[", 2)) is what you need ? – Ronak Shah
purrr::map_dfr(myList, 'df1'), or get both the first and second at once with purrr::pmap(myList, dplyr::bind_rows) – alistaire
Expected output would be a data frame that is made up of A1, B1 and C1 combined (bound rows) and a separate data frame of A2, B2 and C2. – Brisbane Pom
Can you change your database query to return the tables you want? – SymbolixAU
@RonakShah - similarly do.call(Map, c(rbind, myList)) to put it all back into a length-2 list. – thelatemail

3 Answers

1
votes

Based on Ronak Shah's comment to my question, this is the answer I went with:

dfX1 <- data.frame(do.call("rbind",lapply(myList,"[[","df1")))
dfX2 <- data.frame(do.call("rbind",lapply(myList,"[[","df2")))
0
votes
myList %>% 
   pmap(.,bind_rows) %>% 
   bind_cols()
  Value Value1
1     A      1
2     B      2
3     C      3
4     D      4
5     E      5
6     F      6
7     G      7
8     H      8
9     I      9
0
votes

Edit: the following code does not create the required output (OP clarified the intended output after I drafted this)

Let's create a custom function. Your data frames seem to be in the same position, so let's exploit that regularity:

getDataFrame <- function(mylist, wantx) {
  df <- sapply(myList, `[[`, wantx)
  names(df) <- paste0("Name", seq(1:length(mylist)))
  df <- as_tibble(df)
  return(df)
}

So,

getDataFrame(myList, 1)

returns:

# A tibble: 3 x 3
  Name1 Name2 Name3
  <fct> <fct> <fct>
1 A     D     G    
2 B     E     H    
3 C     F     I

And similarly:

> getDataFrame(myList, 2)
# A tibble: 3 x 3
  Name1 Name2 Name3
  <fct> <fct> <fct>
1 1     4     7    
2 2     5     8    
3 3     6     9

If you don't want them to be factors, you'll have to convert them afterwards. Hope this helps.