Friday, 10 May 2019

Shell output is buffered into one message, despite separate echo statements?

I have a shell script with three echo statements:

echo 'first message'

echo 'second message'

echo 'third message'

I then run this script in node and collect the output via this code:

var child = process.spawn('./test.sh');
child.stdout.on('data', data => {
   data = JSON.stringify(data.toString('utf8'));
   console.log(data);
});

But the output is "first message\nsecond message\nthird message\n", which is a problem. I expected three outputs, not one smushed together due to some form of buffering. And I can't just split on newlines, because the individual outputs may contain newlines.

Is there any way to distinguish the messages of individual echo statements? (or other output commands, i.e. printf, or anything that causes data to be written to stdout or stderror)

Edit: I have tried unbuffer and stdbuf, neither work for this, as simple testing can demonstrate. Here is an example of the stdbuf attempt, which I tried with a variety of different argument values, essentially all possible options.

 var child = process.spawn('stdbuf', ['-i0', '-o0', '-e0', './test.sh']);

To be clear, this problem happens when I run a python script from node, too, with just three simple print statements. So it's language-agnostic, it's about correctly processing the output of a script in any language.



from Shell output is buffered into one message, despite separate echo statements?

No comments:

Post a Comment