How to bind data tables from lists in nested list?

Question

I have a nested list dt_list, which contains in itself lists: dt_list1, dt_list2, dt_list3. Each of these three lists contains data tables with same names: dt1, dt2, dt3. So I can get dt1 of dt_list1 like this:

dt_list[["dt_list1"]][["dt1"]]

It will give data table:

ID    timestamp     type
AA   2020-10-01    user
A1   2020-10-01    administrator
BA   2020-10-01    user

and

dt_list[["dt_list2"]][["dt1"]]

will give:

ID    timestamp     type
KB   2020-10-02    new_user
AA   2020-10-02    administrator
BB   2020-10-02    user

I want to bind columns ID and type to get:

ID    type
AA    user
A1    administrator
BA    user
KB    new_user
AA    administrator
BB    user

I want to bind like that all "dt1" data tables from all three sublists dt_list1, dt_list2, dt_list3 using data.table::rbindlist() function. Binding two of them can be done like this:

data.table::rbindlist(list(dt_list[["dt_list1"]][["dt1"]], dt_list[["dt_list2"]][["dt1"]]))

It will bind two necessary dataframes. But I want to bind all three with iteration, not one by one (because there can be not three sublists, but 20 and one by one is not efficient in that case). I think a function which will bind all data tables dt1 from all sublists is the best solution. data.table library must be used for that

Answer 1

You can use [[ in lapply and rbind the result and remove column 2 with [-2].

do.call(rbind, lapply(dt_list, "[[", "dt1"))[-2]
#           ID type
#dt_list1.1  1    a
#dt_list1.2  2    a
#dt_list2.1  1    b
#dt_list2.2  2    b
#dt_list3.1  1    c
#dt_list3.2  2    c

Data:

dt_list <- list(dt_list1 = list(dt1 = data.frame(ID=1:2, time=2, type="a"))
          , dt_list2 = list(dt1 = data.frame(ID=1:2, time=2, type="b"))
          , dt_list3 = list(dt1 = data.frame(ID=1:2, time=2, type="c")))

Answer 2

Another approach using rrapply() in the rrapply-package that easily extends to deeper nested lists:

## extract data.frames from nested list
(dt_flat <- rrapply::rrapply(dt_list, classes = "data.frame", how = "flatten"))
#> $dt1
#>   ID time type
#> 1  1    2    a
#> 2  2    2    a
#> 
#> $dt1
#>   ID time type
#> 1  1    2    b
#> 2  2    2    b
#> 
#> $dt1
#>   ID time type
#> 1  1    2    c
#> 2  2    2    c

## bind together
data.table::rbindlist(dt_flat)
#>    ID time type
#> 1:  1    2    a
#> 2:  2    2    a
#> 3:  1    2    b
#> 4:  2    2    b
#> 5:  1    2    c
#> 6:  2    2    c

---

If you want to filter only the dt1 data.frames, you can add an additional condition argument:

## extract dt1 data.frames from nested list
rrapply::rrapply(dt_list, classes = "data.frame", condition = function(x, .xname) .xname == "dt1", how = "flatten")
#> $dt1
#>   ID time type
#> 1  1    2    a
#> 2  2    2    a
#> 
#> $dt1
#>   ID time type
#> 1  1    2    b
#> 2  2    2    b
#> 
#> $dt1
#>   ID time type
#> 1  1    2    c
#> 2  2    2    c

Data

dt_list <- list(dt_list1 = list(another_list1 = list(dt1 = data.frame(ID=1:2, time=2, type="a"))),               
                dt_list2 = list(another_list2 = list(dt1 = data.frame(ID=1:2, time=2, type="b"))),                
                dt_list3 = list(another_list3 = list(dt1 = data.frame(ID=1:2, time=2, type="c"))))