Oozie Re-run the workflow from the node, where it was last killed

1
votes

I am using the MR action in oozie. And want to use the suspend/resume functionality provided by oozie on MR.

The problem - However, when we call 'suspend' on Oozie that doesn't kill the hadoop job instead it only updates the oozie state, while the hadoop job continues to run to completion. I want the hadoop job to be killed immediately when the oozie workflow is suspended. Guess that's not possible right ?

So, I figured I can kill the oozie workflow and re-run it. But now the problem is , if there are 3 action nodes in my workflow, I don't want to re-run all of them.

Am I smoking here or is it possible to access the last successful oozie node ( which is most likely stored in the mysql database) ?

Is there a better approach to my connived use-case ?

Appreciate your help in advance,

P.S. I am not posting my workflow because it's a simple workflow with several map-reduce actions, no rocket science here.

1
hadoopoozie

1 Answers

2
votes

The oozie command line interface allows for the flag "-rerun" to be used in the following context:

oozie job -oozie http://localhost:11000/oozie -config job.properties -rerun 14-20090525161321-oozie-joe

There is one issue in the automation of reruns, and this is that you must supply a list of action nodes to skip inside of the job.properties file using the oozie.wf.rerun.skip.nodes variable. This following is taken from source

Only one of following two configurations is mandatory. Both should not be defined at the same time oozie.wf.rerun.skip.nodes
oozie.wf.rerun.failnodes
Skip nodes are comma separated list of action names. They can be any action nodes including decision node.
The valid value of oozie.wf.rerun.failnodes is true or false. If secured hadoop version is used, the following two properties needs to be specified as well mapreduce.jobtracker.kerberos.principal dfs.namenode.kerberos.principal.

In order to get the automation, it might be worth you time to try appending to the properties file on completion of an action using either a shell script or implementing it directly into your MR jobs.