How to cache intermediate results when rendering with parameters?

Question

With RMarkdown, I try to render a parametrized report for different values of a parameter. The Rmd file use caching.

The caching works as intended if I knit in RStudio, with the knit button : cache built at first, then used at each successive knitting, even if I change the parameter value in the YAML header.

But when looping with my parameters values and using rmarkdown::render() the cache is rebuilt at each iteration.

The test.Rmd file

---
title: "Untitled"
author: "Author"
params:
  id: 0
date: "23/10/2019"
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

## Test `r params$id`

```{r cars, cache=TRUE}
## open and work on large file (simulate)
test <- mtcars
Sys.sleep(10)
```

And the rendering script : render.R

library(rmarkdown)
library(tidyverse)

1:5 %>% 
  walk(function(x) render("test.Rmd",
                          params = list(id = x),
                          output_file = paste0("file", x, ".html")))

The script takes 5 * 10 seconds to run instead of about 10 seconds.

What did I do wrong? How to use the cache?

I’m guessing that changing the parameters invalidates the caches because it’s impossible to prove in general that the cached content isn’t affected by the parameters. This is a fundamental property. Maybe Knitr has a way of indicating that a chunk is independent of parameters but, short of that, this is unsolvable. — Konrad Rudolph
It seems different parameters should not invalidate the cache ; see github.com/yihui/knitr/issues/1624 — mdag02

Yihui Xie Yihui Xie · Accepted Answer · 2019-11-18T23:25:36

It has nothing to do with parameters, which can be shown by the minimized reprex below (test.Rmd) by taking out the parameters (and the irrelevant tidyverse):

---
title: "Untitled"
---

```{r, cache=TRUE}
Sys.sleep(10)
```

Then run

for (i in 1:5) rmarkdown::render(
  "test.Rmd", output_file = paste0("file", i, ".html")
)

The problem comes from output_file, which changes in each iteration. For R Markdown documents, the output filename determines the knitr chunk option fig.path. For example, when output_file = "file1.html", fig.path is set to file1_files/html/.

When any chunk option of a code chunk changes, knitr will invalidate its cache. In your case, fig.path invalidated the cache each time. To avoid that, you have to stabilize this option, e.g.,

---
title: "Untitled"
---

```{r, cache=TRUE, fig.path='test_files/html/'}
Sys.sleep(2)
```

How to cache intermediate results when rendering with parameters?

1 Answers