gnuplot5 - sh: 1: Syntax error: "(" unexpected with awk

Question

I am trying to use paste in combination with awk inside gnuplot so that I can take certain columns of two different files and plot the result. However, I am getting this error:

sh: 1: Syntax error: "(" unexpected

I understand that it's unhappy with that part of my gnuplot script:

system("paste <(awk '{print $1,$2,$3,$4,$5,$14}' ".filename(i).") <(awk '{print $14}' ".filename_without.")  > Tim_res_".i)

And more specifically the brackets before awk. Replacing the filenames without using a variable name doesn't work either while all the command works perfectly fine outside of gnuplot. If I remove the brackets, then it says it doesn't recognise awk, and the command doesn't work outside of gnuplot either.

I've read that adding #!/usr/bin/bash might help, but it actually doesn't solve the problem nor does it give another error message. I've never had this kind of problem using such commands within gnuplot before. I have a lot of files to plot in a similar way, hence why I'd like to make it automatic directly in gnuplot, the system command being only here to make sure it works before I can replace it directly after the plot command of gnuplot.

Thanks for any help

Answer 1

mklement0 has explained what the issue is - a workaround would be to avoid process substitution entirely:

awk 'NR == FNR { col[NR] = $14; next } { print $1,$2,$3,$4,$5,$14,col[FNR] }' file2 file1

That is, read the second file first and save the column, then do the "paste" yourself in awk.

Moving the awk command to gnuplot:

system("awk 'NR == FNR { col[NR] = $14; next } \
{ print $1,$2,$3,$4,$5,$14,col[FNR] }' " . filename_without . " " . filename(i))

Answer 2

Your command, as the error message indicates, is executed with sh (/bin/sh), which is what gnuplot's system command uses.

sh, which can only be assumed to have POSIX-mandated features, doesn't support process substitutions (<(...)), which are a nonstandard Bash feature.

Tom Fenech's helpful answer shows a reformulation of your command that bypasses the need for process substitutions altogether.

If you do need (the equivalent of) process substitutions:

• (a) Either: call bash -c as part of your system() command string, passing it the original command (which presents quoting challenges).

• (b) Or: Use temporary files instead of process substitutions; in your case, create at least 1 temporary file containing awk output that you then pass to paste.

Simplified examples:

(a) Using an embedded bash -c call:

system("bash -c \"paste <(cat '".filename1."') <(cat '".filename2."')\"")

Note how the inner command is enclosed in \" so it can be passed to bash -c as a single argument.

(b) Using a temporary file:

system("cat '".filename1."'>>/tmp/$$; cat '".filename2."' | paste /tmp/$$ -; rm /tmp/$$")

Note how 1 awk command can provide input to paste via the pipeline, but the other requires a temporary file.