Loop through list of file names in a file R

Question

list.files() can help find files in a directory, but how can I loop through a list of files already in text file? Where all_my_files.txt lists the path to each file one per row:

file.txt
file2.txt
file3.txt

library(data.table)

files<- read.csv(all_my_files.txt)
for (i in 1:length(files))
{
df<-fread(files[i])
x<-mean(df$V1)
}

read.csv returns a data frame. If you don't have multiple columns in all_my_files.txt, you might want readLines to return a character vector, which would make the rest of your syntax correct. However you'll also probably want to use i in your outputs as well, so you don't just overwrite x each iteration of the loop. (And you probably need quotes around "all_my_files.txt") — Gregor Thomas
@mxttgen31 answer updated with a loop solution similar to one in question — bs93

DaveTurek DaveTurek · Accepted Answer · 2020-05-21T12:36:22

You can use lapply to loop through your file names.

I use iris like @bs93 but split into 3 separate data.frames.

iris1=iris[1:50,]   
iris2=iris[51:100,] 
iris3=iris[101:150,]

# write them to text files
write.table(iris2,file="iris2.txt",row.names=FALSE)
write.table(iris3,file="iris3.txt",row.names=FALSE)
write.table(iris1,file="iris1.txt",row.names=FALSE)

# create the text file containing the filenames
filenames <- paste0("iris", 1:3, ".txt")
writeLines(filenames,"filenames.txt")

# Now solve the problem
# read the filenames into a character vector
fn <- readLines("filenames.txt")

# apply `read.table` over that vector of filenames
Ilist <- lapply(fn,read.table,header=TRUE)

# Ilist is a list containing 3 data.frames
str(Ilist)

# Get the mean Sepal.Length from each data.frame in Ilist
x <- sapply(Ilist,function(z) mean(z$Sepal.Length))
x

# if you want to use `data.table` 
library(data.table)

# then you can use `fread` instead of `read.table`
Ilist <- lapply(fn,fread)

# Then Ilist will be a list of 3 data.tables

Loop through list of file names in a file R

2 Answers