OOZIE HIVE Action - The workflow.xml properties are not passed on to child tasks

Question

We have a hive action in oozie and we are overriding some of the default of mapred properties in the workflow.

The child tasks triggered by the hive action has default value for the properties rather than the values configured in the workflow. The job.xml generated per child job in the jobtracker shows only default values.

How can I ensure that all the subtasks started by hive action get the properties from workflow.xml.

The below is my hive action from workflow.xml :

<action name="Hive">
    <hive>
        <job-tracker>${job_tracker}</job-tracker>
        <name-node>${name_node}</name-node>
        <script>${wf:actionData('Query')['Location']}</script>
         <param>JOBDIR=${jobDir}</param>            
        <configuration>             
            <property> 
                <name>oozie.launcher.mapred.child.java.opts</name> 
                <value>-Xmx512m</value> 
            </property>                 
            <property> 
                <name>mapred.map.child.java.opts</name> 
                <value>-Xmx1024m</value> 
            </property> 
            <property> 
                <name>mapred.reduce.child.java.opts</name> 
                <value>-Xmx1024m</value> 
            </property>
                  <name>mapred.max.split.size</name>
                  <value>2560000</value>
                  <description>Largest valid size in bytes for a file split.
                        This should be set to a small multiple of the block size in bytes
                  </description> 
            </property>             
        </configuration>            
    </hive>

You can tune Hive parameters in the HQL scripts but not in the Oozie workflow. Note that in the latest Hadoop distribution, the oozie Hive actions is deprecated and replaced by Hive2 (oozie.apache.org/docs/4.2.0/DG_Hive2ActionExtension.html). — Harold

Mohit Jain Mohit Jain · Accepted Answer · 2018-06-28T01:27:59

Try using

mapred.map.child.java.opts -> mapreduce.map.java.opts
mapred.reduce.child.java.opts -> mapreduce.reduce.java.opts
mapred.max.split.size -> mapreduce.input.fileinputformat.split.maxsize

I think "oozie.launcher.mapred.child.java.opts" is working fine.

OOZIE HIVE Action - The workflow.xml properties are not passed on to child tasks

1 Answers