calling awk from perl does not work with unzip redirect

Question

The below command works fine when I run it manually but when I call it from perl script using backticks or system command, it gives this error

sh: -c: line 0: syntax error near unexpected token `('

Script snapshot:

#Find contents of myFile in zipfile and output the matched records to output.txt
$cmd = "awk -F\"|\" 'NR==FNR{hash[\$0]=1;next} \$237 in hash' $myFILE <(unzip -p $zipfile *XYZ*) >> output.txt";
$result=`$cmd`;

It seems that we cannot call a subshell i.e. (unzip ...) within a system call through perl. Please advise as I have been struggling since a couple of days.

Good grief ... no wonder it's a struggle. Who's going to get all those escapes right. Why that awk? This way you shell out (with backticks), then run a command (awk, for which you need to escape things), run another program in a subshell of that shell (unzip, for which you need to escape things), with redirections ... why not just do all processing (of unzip's return) in Perl, if the script is in Perl? — zdim
If you reframe your question as to exactly what your inputs and outputs are, we'll give you a perl only answer that's both simpler and less brittle. But why not instead use one of the various perl modules do to the job — Sobrique
Thanks for your answers. I do not want to add packages in perl — Nainesh Vashi
@NaineshVashi I didn't suggest to use packages. To repeat: call unzip out of your (Perl) script and process its output in that (Perl) script. Use Perl's (superb) native text-processing capabilities, instead of shelling out to use awk. (I get the idea: you have a command-line with awk that works and you'd like to "just use it" in the script; but that makes it really much harder, error-prone, and brittle. And awkward.) — zdim

niry niry · Accepted Answer · 2017-10-13T02:31:31

This is because the command line is most likely /bin/bash and perl's shell is /bin/sh which does not support this mean. Put the zip command before the awk with pipe (|). Something like this:

#Find contents of myFile in zipfile and output the matched records to output.txt
$cmd = "unzip -p $zipfile *XYZ* | awk -F\"|\" 'NR==FNR{hash[\$0]=1;next} \$237 in hash' $myFILE >> output.txt";
$result=`$cmd`;

calling awk from perl does not work with unzip redirect

1 Answers