Can't run shell script via oozie

Question

I'm trying to run a shell script that just create a directory via mkdir command via oozie workflow.

I'm using HDP 2.6.5 | Oozie 4.2.0

The error message is always :

java.io.IOException: Cannot run program "test.sh" (in directory "/hadoop/yarn/local/usercache/whorchani/appcache/application_1547225966242_3390/container_e111_1547225966242_3390_01_000002"): error=2, No such file or directory

Does this answer your question? Running shell script from oozie through Hue — Petro

grantler grantler · Accepted Answer · 2019-08-06T14:49:04

It's hard to answer your question if you don't at least post your Oozie XML for the relevant action. My guess is that you haven't used the <file> tag.

From Apache Oozie by Mohammad Kamrul Islam and Aravind Srinivasan

Because the shell command runs on any Hadoop node, you need to be aware of the path of the binary on these nodes. The executable has to be either available on the node or copied by the action via the distributed cache using the <file> tag. For the binaries on the node that are not copied via the cache, it’s perhaps safer and easier to debug if you always use an absolute path.

Here is a simple example of a shell action:

    <action name="shell_action">
        <shell xmlns = "uri:oozie:shell-action:0.1">
            <job-tracker>${jobTracker}</job-tracker>
            <name-node>${nameNode}</name-node>
            <exec>sh</exec>
            <argument>my_shell_script.sh</argument>
            <file>/full/hdfs/path/to/your/script/my_shell_script.sh</file>
        </shell>
        <ok to="action2"/>
        <error to="fail"/>
    </action>

Can't run shell script via oozie

1 Answers