I've 10 sqoop commands to execute using oozie. Should I write sqoop action 10 times or is it possible to put the sqoop commands in a file and call it in the sqoop action. Thanks in advance.
0
votes
1 Answers
0
votes
is it possible to put the sqoop commands in a file and call it in the sqoop action
yes, off course its possible. You can save all sqoop commands in shell file and execute shell action.
But should you do that? probably not . If those tasks are independent of each other, executing them as shell action would mean , you are not taking advantage of oozie's scalable/parallel capabilities and all commands will execute one after another.
on the other hand, if those tasks after dependent on each other, how will you control the execution if one of commands fail for some reason.
how about mixed case where some sqoop commands can be run in parallel while other are interdependent on some previous command's result. Things get complicated fairly quickly, right?
for all these reasons, I think better to isolate them as individual actions if they are independent of each other, make a coordinator if you need to repeat them again and again.
