3
votes

It seems for big enough stdout output node does not fire data events:

Example bash script:

#!/usr/bin/env sh

for i in {1..100000}; do
    echo $i
done

Running the above script from bash should prints all numbers from 1 to 100k one number per line

However when proxying it via node's exec:

#!/usr/bin/env node --harmony --harmony_destructuring --harmony_default_parameters

const {exec} = require('child_process');

let stdout = '';

const child = exec('./tmp.sh');

child.stdout.on('data', data => {
    stdout += data;
});

child.stdout.on('close', code => {
    if (code) process.exit(code);
    console.log(stdout.split('\n').slice(0, 3).join('\n'));
    console.log('...');
    console.log(stdout.split('\n').slice(-3).join('\n'));
});

For the above script, I'd expect to get:

1
2
3
...
99998
99999
100000

However when run It returns:

1
2
3
...
35984
35985

About a third of the output is truncated. Logging inside the data handler shows that only 1 data event is fired with 1 chunk

I've tried listening to the child.on('exit') event instead with the same result (also stdout.on('exit'))

It works the same when replacing exec with spawn

node --version # v5.4.1
osx 10.10
1

1 Answers

4
votes

child_process.exec() buffers the standard output into a fixed size buffer. You are likely filling the buffer and thus you don't get the rest of the output.

There is a maxBuffer option you can pass to .exec() to make the buffer larger or you can use .spawn() and receive an unlimited stream of output.

See Stdout buffer issue using node child_process for related info.

Also worth reading: Difference between spawn and exec of Node.js child_process and Stdout of Node.js child_process exec is cut short.