maximum number of files/directories handled by dirent struct

Question

I have a directory (let's call it directory A) which contains ~15000 directories (B1,B2,..,B15000), each directory of the B's includes a file(let's call it "raw") which I want to read. so I want to read: A/B1/raw A/B2/raw ...

I don't know the B's directories names in advance, but I know the name of directory A and that it each B's directory there is a file called raw.

So I used dirent in order to opendir A and move over all the directories in it - extracting the name which contained in d_name field, but the run is stopped after I read the names of 264 directories!!

I wonder if the struct dirent cannot hold more than 264 directories ? Can anyone help me ?

Thanks in advanced.

[Code from comment]

DIR* dRoot = opendir(argv[1]);
assert(dRoot != NULL);
struct dirent* dir;

while((dir = readdir(dRoot)))
{
    //suppose dirName is a string "argv[1]/dir->d_name/raw"
    FILE* file = fopen(dirName,"r");

    //do something

    fclose(file);
}

closedir(dRoot);

How do you read the directories? Can you please show us some code? Preferably a Minimal, Complete, and Verifiable example. — Some programmer dude
DIR* dRoot = opendir(argv[1]); assert(dRoot != NULL); struct dirent* dir; while((dir = readdir(dRoot))) { //suppose dirName is a string "argv[1]/dir->d_name/raw" FILE* file = fopen(dirName,"r"); //do something fclose(file); } closedir(dRoot); what I see is that I get out this loop although I didn't go over all the directories that rely in dRoot! seems like "dir->d_name" contains NULL after the directory number 264! — user3527261
I assume you check for the current and parent directories (the ones named "." and "..")? Also, what is the argument to the program (argv[1])? You check that the file opened properly (i.e. file is not NULL)? You check that it's a directory and not a file? And how do you create and initialize dirName? — Some programmer dude
The problem isn't the number of directories. You might have an issue closing the files; you might be running out of space in your program; you might not be checking that you've successfully opened the file; you might be running into all sorts of issues. But there is no specific upper limit on the size of directory that readdir() can read. — Jonathan Leffler

EOF EOF · Accepted Answer · 2014-04-12T22:05:52

readdir() allocates memory for every struct dirent it returns a pointer to. The dirents are deallocated automatically on closedir(), so you can't free() that memory when you no longer need any particular dirent, and you can't use any dirent after closedir(). I suppose it's possible (though not likely) that your program can't allocate more memory for dirents.
You could check errno after readdir() returns NULL to find out.

If that is the problem, you can instead use readdir_r(), which uses a dirent* to an allocation you provide (which could even be a local variable on the stack), and you could reuse it for all dirents, one after the other. One caveat with readdir_r() is that the size of struct dirent depends on the file system, and you need to ensure your allocation is big enough.

maximum number of files/directories handled by dirent struct

1 Answers