reading data from a number text files in R to a single text file

Question

I have 198 txt files in a single directory. The filenames are stored in a list called filelist. Each file has 7 columns and a different number of rows (100-200). One file appears to have 8 columns. I used the following code

dat<-data.frame()
for (i in 1:n)
{
    dat<-rbind.fill(dat,read.table(filelist[i],sep=",",header=TRUE))
}

I get the error

"Error in read.table(filelist[i], sep = ",", header = TRUE) : duplicate 'row.names' are not allowed".

Some posts suggest I should add rownames=NULL but I tried this and get the error message

"Error in read.table(filelist[i], sep = ",", header = TRUE, rownames = NULL) : unused argument (rownames = NULL)"

All guidance would be much appreciated.

Are you sure you have the right separator? Were you expecting your files to have row names? Chances are something isn't formatted the way you are expecting. Maybe try read.csv instead since you seem to be setting sep="," — MrFlick
MrFlick, the seperator works fine, I don't think this is the issue as I can get the program to work for the first 78 files. It's only once I get to the 79th file that it stops. The 79th file has 8 columns instead of 7. — Melinda Hodkiewicz
What do you want to do with the extra column in the last file? You can't really stack files with different numbers of columns. That doesn't make a lot of sense. — MrFlick
Thanks for this. I did see a post where someone looked at two files, realised that one had an extra column and created a dummy column in the smaller file full of NAs and then did the rbind. That seemed like a good idea but I can't figure out how to do it for so many files. — Melinda Hodkiewicz

Md. Sahidul Islam Md. Sahidul Islam · Accepted Answer · 2015-08-16T05:34:36

Create three *.txt dataset (each have 100 rows and 2 columns) and save it to working directory according to filelist....

setwd("C:\\Users\\Sahidul.Islam\\Desktop\\R")
#[1] "C:/Users/Sahidul.Islam/Desktop/R"
getwd()

data<-matrix(rnorm(200),100,2)
colnames(data)<-c("var1","var2")

filelist<-c("dat1.txt","dat2.txt","dat3.txt")

for (i in 1:length(filelist)){
write.table(data,filelist[i],row.names=F)
    }

Call three *.txt datasets (created above) and bind the rows to make a compact data...

dat<-data.frame(read.table("dat1.txt",header=T))

for (i in 2:length(filelist))
    {
    dat<-rbind(dat,read.table(filelist[i],header=TRUE))
    }

Check the dimension of final dataset.

dim(dat)
#[1] 300   2

Hope this will work.

reading data from a number text files in R to a single text file

1 Answers