oozie workflow shellAction

1
votes

I am trying to run a shell action in an oozie workflow with the shell script containing the following command:

.#!/bin/bash

hadoop fs -mkdir /user/mathon_k/shell_dir

.#echo "hello "

I get the following error:

Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1]

Log

2014-03-28 15:11:31,951 INFO org.apache.oozie.command.wf.ActionStartXCommand: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@:start:] Start action [0000028-140328083804680-oozie-oozi-W@:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2014-03-28 15:11:31,952 WARN org.apache.oozie.command.wf.ActionStartXCommand: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@:start:] [0000028-140328083804680-oozie-oozi-W@:start:]Action status=DONE 2014-03-28 15:11:31,952 WARN org.apache.oozie.command.wf.ActionStartXCommand: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@:start:] [0000028-140328083804680-oozie-oozi-W@:start:]Action updated in DB! 2014-03-28 15:11:32,027 INFO org.apache.oozie.command.wf.ActionStartXCommand: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@oozie_20140328] Start action [0000028-140328083804680-oozie-oozi-W@oozie_20140328] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2014-03-28 15:11:32,170 WARN org.apache.oozie.action.hadoop.ShellActionExecutor: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@oozie_20140328] credentials is null for the action 2014-03-28 15:11:32,669 INFO org.apache.oozie.action.hadoop.ShellActionExecutor: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@oozie_20140328] checking action, external ID [job_201403280835_0020] status [RUNNING] 2014-03-28 15:11:32,772 WARN org.apache.oozie.command.wf.ActionStartXCommand: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@oozie_20140328] [*0000028-140328083804680-oozie-oozi-W@oozie_20140328*]Action status=RUNNING 2014-03-28 15:11:32,772 WARN org.apache.oozie.command.wf.ActionStartXCommand: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@oozie_20140328] [*0000028-140328083804680-oozie-oozi-W@oozie_20140328*]Action updated in DB! 2014-03-28 15:11:45,254 INFO org.apache.oozie.servlet.CallbackServlet: USER[-] GROUP[-] TOKEN[-] APP[-] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@oozie_20140328] callback for action [0000028-140328083804680-oozie-oozi-W@oozie_20140328] 2014-03-28 15:11:45,332 INFO org.apache.oozie.action.hadoop.ShellActionExecutor: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@oozie_20140328] action completed, external ID [job_201403280835_0020] 2014-03-28 15:11:45,358 WARN org.apache.oozie.action.hadoop.ShellActionExecutor: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@oozie_20140328] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.ShellMain], exit code [1] 2014-03-28 15:11:45,518 INFO org.apache.oozie.command.wf.ActionEndXCommand: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@oozie_20140328] ERROR is considered as FAILED for SLA 2014-03-28 15:11:45,563 INFO org.apache.oozie.command.wf.ActionStartXCommand: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@kill] Start action [0000028-140328083804680-oozie-oozi-W@kill] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 2014-03-28 15:11:45,563 WARN org.apache.oozie.command.wf.ActionStartXCommand: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@kill] [0000028-140328083804680-oozie-oozi-W@kill]Action status=DONE 2014-03-28 15:11:45,563 WARN org.apache.oozie.command.wf.ActionStartXCommand: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[0000028-140328083804680-oozie-oozi-W@kill] [0000028-140328083804680-oozie-oozi-W@kill]Action updated in DB! 2014-03-28 15:11:45,626 WARN org.apache.oozie.command.coord.CoordActionUpdateXCommand: USER[mathon_k] GROUP[-] TOKEN[] APP[oo_20140328] JOB[0000028-140328083804680-oozie-oozi-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100

But when i change the shell script to the following it works well.

.#!/bin/bash

.#hadoop fs -mkdir /user/mathon_k/shell_dir

echo "hello "
2
shellhadoophdfsoozie

2 Answers

1
votes

If you want to mkdir - use workflow HDFS action

if you want to run the bash - seem like it work fine

0
votes

We also experienced issues with issuing hadoop shell commands in Oozie due to security issues. Which version Hadoop are you using? Which version oozie?

For starters, try to set HADOOP_USER_NAME to name of the user you want the hadoop command to execute with prior to executing it.