retrieving words from a single line in a file using fgets and strtok

Question

I have the functions below which reads words from a file and outputs each word using fgets and strtok where the words in the file are split by newline:

word1
word2
word3

I'm trying to mimic the functionality where the words in the file are on a single line split only by a space:

word1 word2 word3

However I seem to only be able to get the first word whilst changing the strtok char to " " and attempting to read a single line. I'm unsure what i'm missing.

#include <string.h>
#include <malloc.h>

int readLines;
char *output[255];
char *filename = "commands.txt";

char fileRead(const char *filename, char *output[255])
{
  int count = 0;
  char input[255];
  char *line;
  FILE *file = fopen(filename, "r");

  if (file == NULL) {
    printf("Cannot open file: %s\n", filename);
  } else {
    while(count < 255 && fgets(input, sizeof(input), file)) {
      line = strtok(input, "\n");
      if (line) {
        output[count++] = strdup(line); /* Store replica */
      }
    }
    fclose(file);
  }
  return count;
}

char *strdup(const char *str)
{
  char *ret = malloc(strlen(str)+1);
  if (ret) {
    strcpy(ret, str);
  }
  return ret;
}

int main(int argc, char *argv[])
{
  readLines = fileRead(filename, output);
  /* read from array and pass into flag function */
  for (int x = 0; x < readLines; ++x) {
    printf("%s\n", output[x]);
    free(output[x]);
  }
  return 0;
}

You need to loop on strtok(). The first call will supply input as the first argument to strtok(); the subsequent ones NULL. — Jonathan Leffler
an error message should be output to stderr, not stdout and when the error is from a system function, should use perror() which will both output the enclosed text AND the appropriate system error message. Usually the best thing to do is call exit() after displaying the message rather than allowing the program to continue — user3629249
it is poor programming practice to have a local function those name (and overall signature) is the same as a system function. Note that strdup() is a system function exposed when the header file string.h is included — user3629249
the posted code has a massive memory leak. (or, to put it differently, the code should always cleanup after itself.) The code should be calling free() for each of the char arrays obtained via strdup() — user3629249

David C. Rankin David C. Rankin · Accepted Answer · 2017-07-30T06:18:02

If I understand your question -- that you want to separate words (tokens) that are all contained on one line in a file, then you are using strtok incorrectly. In your code you have line = strtok(input, "\n"); where the delimiter is the newline. If you want to separate space separated words, then you need a space included in your delimiter as well, e.g. char *delim = " \n";

Further, on the first call to strtok you use the variable name of the buffer (or pointer to the buffer) holding the text you want to tokenize. For all remaining calls to strtok (e.g. for words 2, 3, 4...) you use NULL in its place and check the return.

Boiling your example down, you could do something like:

#include <stdio.h>
#include <string.h>

#define MAXC 255

int main (int argc, char **argv) {

    char buf[MAXC] = "",
        *delim = " \n";
    FILE *fp = argc > 1 ? fopen (argv[1], "r") : stdin;

    if (!fp) {  /* validate file open for reading */
        fprintf (stderr, "error: file open failed '%s'.\n", argv[1]);
        return 1;
    }

    if (!fgets (buf, MAXC, fp)) {  /* read one line from file */
        fprintf (stderr, "error: file read failed.\n");
        return 1;
    }

    /* tokenize line with strtok */
    for (char *p = strtok (buf, delim); p; p = strtok (NULL, delim))
        printf ("%s\n", p);

    if (fp != stdin) fclose (fp);     /* close file if not stdin */

    return 0;
}

Example Input File

$ cat dat/strtok.dat
my dog has fleas

Example Use/Output

$ ./bin/strtokoneline <dat/strtok.dat
my
dog
has
fleas

If I misunderstood, then drop a comment and I'm happy to help further. If you have any other questions about the answer, just ask.

If you prefer writing your strtok loop as a while loop instead of a for (it can be easier to look at), you can do something like the following:

    char buf[MAXC] = "",
        *p = buf,
        *delim = " \n";
    ...
    p = strtok (buf, delim);        /* get first token (word) */

    while (p) {
        printf ("%s\n", p);
        p = strtok (NULL, delim);   /* get remaining tokens */
    }

retrieving words from a single line in a file using fgets and strtok

1 Answers