Segmentation fault multithreading with file open

Question

I created a program to get all files in a directory, find the individual checksums and then find the total checksums using multithreading.

I am receiving a segmentation fault so I ran gdb and saw that the error is on line 60 where open() is. After researching the seg fault on SO, and on other forums, I attempted to implement a few different approaches such as changing open() to fopen() with a FILE *handle rather than an int. That change proved incorrect.

After hours of debugging and searching, I am clueless and would greatly appreciate any insight.

#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <stdarg.h>
#include <fcntl.h>
#include <time.h>
#include <sys/types.h>
#include <dirent.h>
#include <pthread.h>   ///Compile with -pthread or -lpthread
#include <sys/stat.h>

#define BUFFER_SIZE (1<<16)

void cleanup();
void get_filenames();
void* get_checksum();

char **filenames;
int file_cnt;
DIR *dir;

//int handle;
FILE *handle;
unsigned int checksum;
unsigned char* ptr;
int length;
int count;
unsigned char* buffer;
int* sum;
unsigned int total = 0;

int main(int argc, char *argv[]){

        int i;
        pthread_t* file;

        atexit(cleanup);
        get_filenames();

        printf("There are %d files:\n", file_cnt);

        file = calloc(sizeof(pthread_t), file_cnt);
        sum = calloc(sizeof(int), file_cnt);
        for(i=0; i<file_cnt; i++){

                printf("%s\n", filenames[i]);

                pthread_create(&(file[i]), NULL, get_checksum, (void*)&filenames[i]);
        }
                for(i=0; i<file_cnt; i++){
                        total += sum[i];
                }
                printf("total is: %u\n", total);
        return EXIT_SUCCESS;
}

void* get_checksum(void* a){

        int b = *((int *)a);

        //handle = open(filenames[b], O_RDONLY); //SEG FAULT HERE
                handle = fopen(filenames[b], "r"); //SEG FAULT HERE
                 if( handle == NULL ){
                        printf( "Can't open file: %s\n", filenames[b]);
                        exit(1);
        }

                buffer = malloc(BUFFER_SIZE);
        if( buffer == NULL ){
                        printf( "Can't get enough memory\n" );
                        exit(1);
                }

                checksum = 0;

                 do{
                        //length = read( handle, buffer, BUFFER_SIZE );
                        length = read( handle, buffer, (sizeof(char)));

                        if( length == -1 ){
                                printf( "Error reading file: %s\n", filenames[b]);
                                        //return NULL;
                                        exit(1);
        }

                        ptr = buffer;
                        count = length;
                        while( count-- ){
                                checksum = checksum + (unsigned int)( *ptr++ );
                                sum[b] = checksum;
                                }
        } while( length );
                printf("Checksum= %d\nTimes at: %d\n", checksum, (int)clock());
}


void cleanup() {

        if(filenames && file_cnt > 0) {
                while(file_cnt-- > 0) {
                        if(filenames[file_cnt]) {
                                free(filenames[file_cnt]);
                        }
                }
                free(filenames);
        }

        if(dir) {
                closedir(dir);
        }

        return;
}


void get_filenames() {

        struct dirent *dir_entry;

        if((dir = opendir(".")) == NULL) {
                fprintf(stderr, "Couldn't open the directory entry for reading\n");
                exit(1);
        }

        errno = 0;
        file_cnt = 0;
        while((dir_entry = readdir(dir)) != NULL) {
                char **new_filenames = filenames;
                static int realative_dirs = 0;

                if(realative_dirs < 2 &&
                   (strcmp(".", dir_entry->d_name) == 0 || strcmp("..", dir_entry->d_name) == 0)
                  ) {
                        realative_dirs++;
                        continue;
                }

                new_filenames = (char **)realloc(filenames, sizeof(char **) * (file_cnt + 1));
                if(new_filenames == NULL) {
                        free(filenames[file_cnt]);
                        fprintf(stderr, "Could not allocate reference for filename[%d]\n", file_cnt);
                        exit(1);
                }

                filenames = new_filenames;
                filenames[file_cnt] = (char *)calloc(strlen(dir_entry->d_name) + 1, sizeof(char));
                if(filenames[file_cnt] == NULL) {
                        fprintf(stderr, "Could not allocate memory for filename[%d]'s string: \"%s\"\n",
                                file_cnt, dir_entry->d_name);
                        exit(1);
                }

                strcpy(filenames[file_cnt], dir_entry->d_name);
                file_cnt++;
        }

        if(errno != 0) {
                fprintf(stderr, "An error occured getting the filenam list\n");
                exit(1);
        }

        return;
}

Below is the output and gdb debugging:

There are 24 files:
.windows
.xscreensaver
.alias
.cshrc
Segmentation fault

(gdb) run
Starting program: /home/nolooking/a.out
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
There are 24 files:
.windows
[New Thread 0x7ffff781e700 (LWP 15957)]
.xscreensaver
[New Thread 0x7ffff701d700 (LWP 15958)]

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff781e700 (LWP 15957)]
0x0000000000400d53 in get_checksum (a=0x60b610) at checksum.c:60
60              handle = open(filenames[b], O_RDONLY);
(gdb) backtrace
#0  0x0000000000400d53 in get_checksum (a=0x60b610) at checksum.c:60
#1  0x00007ffff7bc6374 in start_thread () from /lib64/libpthread.so.0
#2  0x00007ffff7907c3d in clone () from /lib64/libc.so.6
(gdb) quit
A debugging session is active.

UPDATE: I took the advice of one user in the comments who suggested that I use: handle=fopen((char*)a, "r");. I can successfully print out the file names when the if statement if(handle==NULL) is commented out. When I include that if statement I receive the following output:

There are 24 files:
.windows
.xscreensaver
.alias
.cshrc
Can't open file: p▒`



#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <stdarg.h>
#include <fcntl.h>
#include <time.h>
#include <sys/types.h>
#include <dirent.h>
#include <pthread.h>
#include <sys/stat.h>

#define BUFFER_SIZE (1<<16)

void cleanup();
void get_filenames();
void* get_checksum();

char **filenames;
int file_cnt;
DIR *dir;

//int handle;
FILE *handle;
unsigned int checksum;
unsigned char* ptr;
int length;
int count;
unsigned char* buffer;
int* sum;
unsigned int total = 0;

int main(int argc, char *argv[]){

        int i;
        pthread_t* file;

        atexit(cleanup);
        get_filenames();

        printf("There are %d files:\n", file_cnt);

        file = calloc(sizeof(pthread_t), file_cnt);
        sum = calloc(sizeof(int), file_cnt);
        for(i=0; i<file_cnt; i++){

                printf("%s\n", filenames[i]);

                pthread_create(&(file[i]), NULL, get_checksum, (void*)&filenames[i]);
        }
                for(i=0; i<file_cnt; i++){
                        total += sum[i];
                }
                printf("total is: %u\n", total);
        return EXIT_SUCCESS;
}

void* get_checksum(void* a){

        int b = *((int *)a);

                handle = fopen(((char*)a), "r");
                if( handle == NULL ){
                        printf( "Can't open file: %s\n", ((char*)a));
                        exit(1);
       }

                buffer = malloc(BUFFER_SIZE);
        if( buffer == NULL ){
                        printf( "Can't get enough memory\n" );
                        exit(1);
                }

                checksum = 0;

                 do{
                        length = read( handle, buffer, BUFFER_SIZE );

                        if( length == -1 ){
                                printf( "Error reading file: %s\n",  ((char*)a));
                                        //return NULL;
                                        exit(1);
        }

                        ptr = buffer;
                        count = length;
                        while( count-- ){
                                checksum = checksum + (unsigned int)( *ptr++ );
                                //sum[a] = checksum;
                                }
        } while( length );
                printf("Checksum= %d\nTimes at: %d\n", checksum, (int)clock());
}


void cleanup() {

        if(filenames && file_cnt > 0) {
                while(file_cnt-- > 0) {
                        if(filenames[file_cnt]) {
                                free(filenames[file_cnt]);
                        }
                }
                free(filenames);
        }

        if(dir) {
                closedir(dir);
        }

        return;
}


void get_filenames() {

        struct dirent *dir_entry;

        if((dir = opendir(".")) == NULL) {
                fprintf(stderr, "Couldn't open the directory entry for reading\n");
                exit(1);
        }

        errno = 0;
        file_cnt = 0;
        while((dir_entry = readdir(dir)) != NULL) {
                char **new_filenames = filenames;
                static int realative_dirs = 0;

                if(realative_dirs < 2 &&
                   (strcmp(".", dir_entry->d_name) == 0 || strcmp("..", dir_entry->d_name) == 0)
                  ) {
                        realative_dirs++;
                        continue;
                }

                new_filenames = (char **)realloc(filenames, sizeof(char **) * (file_cnt + 1));
                if(new_filenames == NULL) {
                        free(filenames[file_cnt]);
                        fprintf(stderr, "Could not allocate reference for filename[%d]\n", file_cnt);
                        exit(1);
                }

                filenames = new_filenames;
                filenames[file_cnt] = (char *)calloc(strlen(dir_entry->d_name) + 1, sizeof(char));
                if(filenames[file_cnt] == NULL) {
                        fprintf(stderr, "Could not allocate memory for filename[%d]'s string: \"%s\"\n",
                                file_cnt, dir_entry->d_name);
                        exit(1);
                }

                strcpy(filenames[file_cnt], dir_entry->d_name);
                file_cnt++;
        }

        if(errno != 0) {
                fprintf(stderr, "An error occured getting the filenam list\n");
                exit(1);
        }

        return;
}

Why I am receiving that output once I uncomment the if statement?

(void*)&filenames[i] is not int b = *((int *)a); or I'm missing something? — Adriano Repetti
Yes. You're passing the wrong thing as the arg to pthread_create. get_checksum is expecting an integer, and you are passing in the filename. — bruceg
@AdrianoRepetti I'm not quite understanding. check_sum() is called and need a void* a which is (void*)&filenames[i]. However, i is not available in the function check_sum() so I created an integer index. Please explain what you mean if possible. — SamSmith
You're passing a char* but you're casting to an int* and then deferencing to obtain an index for char**. You already have filename, just handle = fopen((char*)a, "r"); — Adriano Repetti

Harry Harry · Accepted Answer · 2016-03-25T17:48:15

Don't use &i. I'll explain in a bit. The argument you're passing to the thread is wrong a is not an integer. It's meant to be a pointer to a string...

Change the thread create to this...

pthread_create(&(file[i]), NULL, get_checksum, filenames[i]);

then print the string as follows...

void* get_checksum(void *a){

  char *file_name = (char *)a;
  printf("filename=%s\n", file_name);

You're passing the string as a pointer to the called function. In your code you're trying to use this as an index into the array.

If you want to pass the index as an integer beware... this won't work..

pthread_create(&(file[i]), NULL, get_checksum, &i);

This is multithreaded and the value pointed to by &i is changing as the loop runs. Pass the pointer to the string and do not under any circumstances change filenames as the threads run.

Segmentation fault multithreading with file open

3 Answers