Hadoop keeps on writing mapred intermediate outuput in /tmp directory

0
votes

I have limited capacity in /tmp so I want to move all the intermediate output of mapred in a bigger partition, say /home/hdfs/tmp_data .
If I understand correctly, I just need to set 

<property>
<name>mapred.child.tmp</name>
<value>/home/hdfs/tmp_data</value>

in mapred-site.xml
I restart the cluster through Ambari, I check everything is written in the conf file, however, when I run a pig script, it keeps writing in:

/tmp/hadoop-hdfs/mapred/local/taskTracker/hdfs/jobcache/job_localXXX/attempt_YY/output I have also modified hadoop.tmp.dir in core-site.xml to be /home/hdfs/tmp_data , but nothing changes.
Is there any parameter that overwrite my settings?

1
hadoopconfigurationmapreduce

1 Answers

0
votes

Try override the following property in tasktracker nodes mapred-site.xml file and restart it.

<property>
<name>mapred.local.dir/name>
<value>/home/hdfs/tmp_data</value>
</property>