Why is the `object.size` of an environment smaller than the `object.size` of the objects in the environment?

Question

I ran into a confusing situation while writing a function factory. I have a tibble with columns of identifying information, and a list column containing data.

tib <-
    tibble(
        id_1 = letters[1:3],
        id_2 = LETTERS[1:3],
        data = list(mtcars, iris, volcano)
    )

I created a function factory to make it easy to get the datasets stored in this tibble.

data_getter <- function(data) {
    force(data)
    function(id_1, id_2) {
        where <- data$id_1 == id_1 & data$id_2 == id_2
        data[where, ]$data %>% 
            purrr::flatten_df()
    }
}

get_from_tib <- data_getter(tib)

In reality, the data frame I am working with is very large. My understanding is that get_from_tib captures an environment that includes tib. I was worried when I did this that by duplication of this large dataset, I would start to run out of memory. However, I was very surprised when the object.size of my function was smaller than the dataset that is supposedly captured by it!

object.size(tib)
21752 bytes

object.size(get_from_tib)
8416 bytes

This works even If I delete tib from the working environment

rm(tib)

object.size(get_from_tib)
8416 bytes

Why is the environment so much smaller? Is object.size simply the wrong function for getting the size of all objects contained in an environment?

Reprex below

library(magrittr)

tib <-
    dplyr::tibble(
        id_1 = letters[1:3],
        id_2 = LETTERS[1:3],
        data = list(mtcars, iris, USArrests)
    )


data_getter <- function(data) {
    force(data)
    function(id_1, id_2) {
        where <- data$id_1 == id_1 & data$id_2 == id_2
        data[where, ]$data %>% 
            purrr::flatten_df()
    }
}

get_from_tib <- data_getter(tib)

get_from_tib('c', 'C')
#> # A tibble: 50 x 4
#>    Murder Assault UrbanPop  Rape
#>     <dbl>   <int>    <int> <dbl>
#>  1   13.2     236       58  21.2
#>  2   10       263       48  44.5
#>  3    8.1     294       80  31  
#>  4    8.8     190       50  19.5
#>  5    9       276       91  40.6
#>  6    7.9     204       78  38.7
#>  7    3.3     110       77  11.1
#>  8    5.9     238       72  15.8
#>  9   15.4     335       80  31.9
#> 10   17.4     211       60  25.8
#> # … with 40 more rows


object.size(tib)
#> 21752 bytes
object.size(get_from_tib)
#> 8416 bytes


rm(tib)
object.size(get_from_tib)
#> 8416 bytes

^{Created on 2019-04-30 by the reprex package (v0.2.1)}

MrFlick MrFlick · Accepted Answer · 2019-04-30T19:13:01

This has to do with the part in the ?object.size help page that says

Associated space (e.g., the environment of a function and what the pointer in a EXTPTRSXP points to) is not included in the calculation.

The data variable is captured in a closure; it's not really "in" the code of the function that's returned from data_getter.

The pyry::object_size() funciton does a better of job of taking the enclosing environment into consideration as well.

pryr::object_size(tib)
# 21 kB
pryr::object_size(get_from_tib)
# 26.9 kB

Why is the `object.size` of an environment smaller than the `object.size` of the objects in the environment?

Reprex below

1 Answers