Piping output between N processes in C

Question

I'm trying to write a C program to simulate the piping of two or more processes together, like ls | sort | wc so running my code as ./driver ls sort wc should show the same result. I think I'm really close, but I can't seem to find the bug in my code below. Any help would be appreciated, I'm really stumped here.

I think I understand what is supposed to happen, but Im crossing my wires somehow in making it happen. The parent should fork child processes who in turn reroute their STDOUT to the write end of a pipe(a). Any child who is created beyond the first child should consider the read end of this pipe(a) as its STDIN, as well as redirect it's own output to a pipe(b) of it's own.

Say a third process is piped. It should consider the read end of the pipe(b) as STDIN, and again pipe its output to the write end of a new pipe(c) before executing the requested command.

The last case is the when the final process is passed to the pipe. In this example, a fourth process would consider the read end of the pipe(c) but should not need to redirect the STDOUT, just send it to STDOUT as normal.

#include <errno.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/wait.h>

#define FORK_CHILD 0

static void error_and_exit(void) {
  fprintf(stderr, "Error: %s\n", strerror(errno));
  exit(EXIT_FAILURE);
}

static int fork_or_die(void) {
  int pid = fork();
  if (pid == -1) {
    error_and_exit();
  }
  return pid;
}

int main(const int argc, const char * argv[])
{
  int processes = argc - 1;
  int apipe[argc - 1][2];
  int pid;

  int result = -1;
  for (int i = 0; i < processes; i++) {
    result = pipe(apipe[i]);
    if (result == -1) {
      error_and_exit();
    }

  }

  for (int i = 0; i < processes; i++) {

    pid = fork_or_die();

    // Child process executes process
    if (pid == FORK_CHILD) {

      // If we are not the first program in the pipe
      if (i > 1) {
        // Use the output from the previous program in the pipe as our input

        // Check the read end of the pipe and STDIN are different descriptors
        if (apipe[i - 1][0] != STDIN_FILENO) {
          // Send the read end of the pipe to STDIN
          if (dup2(apipe[i - 1][0], STDIN_FILENO) == -1) {
            error_and_exit();
          }
        }
      }

      // Before we execute a process, bind the write end of the pipe to STDOUT
      // Don't do this to the last process in the pipe, just send output to STDOUT as normal
      if (i < processes - 1) {
        // Check the write end of the pipe and STDOUT are different descriptors
        if (apipe[i][1] != STDOUT_FILENO) {

          // Send the write end of the pipe to STDOUT
          if (dup2(apipe[i][1], STDOUT_FILENO) == -1) {
            error_and_exit();
          }
        }
      }

      // Child executes requested process
      if (execlp(argv[i + 1], argv[i + 1], (char *)NULL) == -1) {
        error_and_exit();
      }

      wait(NULL);
    }
    // Parent does nothing until loop exits (waits for children)

  }

  return 0;
}

Answer 1

I've spotted three issues with your code:

1. As You have decided to index children starting from 0, not one but two processes will skip this part of code. The simplest fix that comes to my mind right now is to change the 1 to 0 or > to >=.

   // If we are not the first program in the pipe
   if (i > 1) {
   

2. You are calling wait in code that is not executed by parent. Moving the call outside the if (pid == FORK_CHILD) branch won't help as parent will wait for one child to finish before another one is started. Child process's write operation may wait for next child to consume some data and make place in buffer. Simplest solution that comes to my mind right now is to move wait calls to another loop.

3. You keep all pipe's descriptor open in parent and child processes. You should close it before wait loop in parent and before execlp in forked processes. The programs like grep, sort won't finish unless they receive EOF in their incoming streams. The pipe won't send EOF as long as at least one write descriptor is still open.

The code with minimal changes applied:

#include <errno.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/wait.h>

#define FORK_CHILD 0

static void error_and_exit(void) {
  fprintf(stderr, "Error: %s\n", strerror(errno));
  exit(EXIT_FAILURE);
}

static int fork_or_die(void) {
  int pid = fork();
  if (pid == -1) {
    error_and_exit();
  }
  return pid;
}

int main(const int argc, const char * argv[])
{
  int processes = argc - 1;
  int apipe[argc - 1][2];
  int pid;

  int result = -1;
  for (int i = 0; i < processes; i++) {
    result = pipe(apipe[i]);
    if (result == -1) {
      error_and_exit();
    }

  }

  for (int i = 0; i < processes; i++) {

    pid = fork_or_die();

    // Child process executes process
    if (pid == FORK_CHILD) {

      // If we are not the first program in the pipe
      if (i > 0) {
        // Use the output from the previous program in the pipe as our input

        // Check the read end of the pipe and STDIN are different descriptors
        if (apipe[i - 1][0] != STDIN_FILENO) {
          // Send the read end of the pipe to STDIN
          if (dup2(apipe[i - 1][0], STDIN_FILENO) == -1) {
            error_and_exit();
          }
        }
      }

      // Before we execute a process, bind the write end of the pipe to STDOUT
      // Don't do this to the last process in the pipe, just send output to STDOUT as normal
      if (i < processes - 1) {
        // Check the write end of the pipe and STDOUT are different descriptors
        if (apipe[i][1] != STDOUT_FILENO) {

          // Send the write end of the pipe to STDOUT
          if (dup2(apipe[i][1], STDOUT_FILENO) == -1) {
            error_and_exit();
          }
        }
      }

      for (int j = 0; j < processes; j++) {
         if(close(apipe[j][0]) == -1)
         error_and_exit();
         if(close(apipe[j][1]) == -1)
         error_and_exit();
      }

      // Child executes requested process
      if (execlp(argv[i + 1], argv[i + 1], (char *)NULL) == -1) {
        error_and_exit();
      }
    }

  }
   
  // Parent does nothing until loop exits (waits for children)
  for (int i = 0; i < processes; i++) {
         if(close(apipe[i][0]) == -1)
         error_and_exit();
         if(close(apipe[i][1]) == -1)
         error_and_exit();
  }

  for (int i = 0; i < processes; i++) {
      wait(NULL);
  }

  return 0;
}