Hadoop 2.5.0 on Mesos 0.21.0 with library 0.0.8 executor error

Question

The stderr logs the following while running a map-reduce job:

root@dbpc42:/tmp/mesos/slaves/20141201-225046-698725789-5050-19765-S24/frameworks/20141201-225046-698725789-5050-19765-0016/executors/executor_Task_Tracker_2/runs/latest# ls
hadoop-2.5.0-cdh5.2.0  hadoop-2.5.0-cdh5.2.0.tgz  stderr  stdout

Contents of stderr:

WARNING: Logging before InitGoogleLogging() is written to STDERR I1202 19:41:40.323521 7223 fetcher.cpp:76] Fetching URI 'hdfs://dbpc41:9000/hadoop-2.5.0-cdh5.2.0.tgz' I1202 19:41:40.323685 7223 fetcher.cpp:105] Downloading resource from 'hdfs://dbpc41:9000/hadoop-2.5.0-cdh5.2.0.tgz' to '/tmp/mesos/slaves/20141201-225046-698725789-5050-19765-S24/frameworks/20141201-225046-698725789-5050-19765-0016/executors/executor_Task_Tracker_0/runs/59c9a5ac-c376-4e9e-b9b7-63a24ae84dcd/hadoop-2.5.0-cdh5.2.0.tgz'

I1202 19:41:48.779220 7223 fetcher.cpp:64] Extracted resource '/tmp/mesos/slaves/20141201-225046-698725789-5050-19765-S24/frameworks/20141201-225046-698725789-5050-19765-0016/executors/executor_Task_Tracker_0/runs/59c9a5ac-c376-4e9e-b9b7-63a24ae84dcd/hadoop-2.5.0-cdh5.2.0.tgz' into '/tmp/mesos/slaves/20141201-225046-698725789-5050-19765-S24/frameworks/20141201-225046-698725789-5050-19765-0016/executors/executor_Task_Tracker_0/runs/59c9a5ac-c376-4e9e-b9b7-63a24ae84dcd'

sh: 1: cd: can't cd to hadoop-2.5.0-cdh5.2.0

sh: 1: ./bin/hadoop: not found

The stdout is empty.

The interesting part is, that when I run the slave on the same node where the master runs, the problem does not appear and tasks finish successfully. Only remote slaves introduce this problem. What could be the problem?

Answer 1

The script placed by the JobTracker and ran by the executor was not able to access the directory. I've solved the problem by changing permissions recursively for directory /tmp to "free for all", 777.

Answer 2

What you need to do is to give +x access wherever your mesos-work-dir is located. You don't need 777 recursively.