My problem is mapper has completed and is showing at 100% but my reducer on the other hand is stuck at 16%. My hadoop cluster consists of 2 data nodes and single namenode on which the secondary namenode and job tracker is configured. Tasktracker GUI is showing the reducer is stuck at 15% though slowly increased to 16% in a matter of 15 mins.I am using hadoop version 0.20.2
I did try some tweaks in mapred-site.xml but to no effect.
Also attaching the jobtracker and tasktracker logs to the message.
I can't figure out what is going wrong.
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>10.0.0.93:54311</value>
<description>The host and port that the MapReduce job tracker runs at.</description>
</property>
<property>
<name>mapred.child.java.opts</name>
<value>-Xmx2024m</value>
</property>
<property>
<name>mapred.reduce.slowstart.completed.maps</name>
<value>0.80</value>
</property>
<property>
<name>tasktracker.http.threads</name>
<value>80</value>
</property>
<property>
<name>mapred.reduce.parallel.copies</name>
<value>.4</value>
</property>
<property>
<name>mapred.tasktracker.shuffle.fadvise</name>
<value>false</value>
</property>
</configuration>Jobtracker log
2014-11-10 08:56:55,852 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:69 completedMapsInputSize:29478557296 completedMapsOutputSize:1518
2014-11-10 08:56:55,858 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #1 for task attempt_201411100854_0001_m_000000_0
2014-11-10 08:56:55,858 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201411100854_0001_m_000068_0' has completed task_201411100854_0001_m_000068 successfully.
2014-11-10 08:56:55,858 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:70 completedMapsInputSize:29746992753 completedMapsOutputSize:1540
2014-11-10 08:56:58,857 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201411100854_0001_m_000069_0' has completed task_201411100854_0001_m_000069 successfully.
2014-11-10 08:56:58,857 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:71 completedMapsInputSize:29831867549 completedMapsOutputSize:1562
2014-11-10 08:56:58,861 INFO org.apache.hadoop.mapred.JobTracker: Adding task (cleanup)'attempt_201411100854_0001_m_000067_1' to tip task_201411100854_0001_m_000067, for tracker 'tracker_GSI-2:GSI-2/127.0.0.1:45167'
2014-11-10 08:57:01,866 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201411100854_0001_m_000067_1' from 'tracker_GSI-2:GSI-2/127.0.0.1:45167'
2014-11-10 08:59:31,995 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #2 for task attempt_201411100854_0001_m_000000_0
2014-11-10 09:04:32,240 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #3 for task attempt_201411100854_0001_m_000000_0
2014-11-10 09:04:32,240 INFO org.apache.hadoop.mapred.JobInProgress: Too many fetch-failures for output of task: attempt_201411100854_0001_m_000000_0 ... killing it
2014-11-10 09:04:32,240 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201411100854_0001_m_000000_0: Too many fetch-failures
2014-11-10 09:04:32,242 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_201411100854_0001_m_000000_1' to tip task_201411100854_0001_m_000000, for tracker 'tracker_GSI-2:GSI-2/127.0.0.1:45167'
2014-11-10 09:04:32,242 INFO org.apache.hadoop.mapred.JobInProgress: Choosing rack-local task task_201411100854_0001_m_000000
2014-11-10 09:04:35,244 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201411100854_0001_m_000000_0' from 'tracker_GSI-1:localhost/127.0.0.1:42259'
2014-11-10 09:04:44,255 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201411100854_0001_m_000000_1' has completed task_201411100854_0001_m_000000 successfully.
2014-11-10 09:04:44,255 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:72 completedMapsInputSize:30905609374 completedMapsOutputSize:1584
2014-11-10 09:07:11,362 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #1 for task attempt_201411100854_0001_m_000001_0
2014-11-10 09:09:47,481 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #2 for task attempt_201411100854_0001_m_000001_0
2014-11-10 09:14:47,697 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #3 for task attempt_201411100854_0001_m_000001_0
2014-11-10 09:14:47,697 INFO org.apache.hadoop.mapred.JobInProgress: Too many fetch-failures for output of task: attempt_201411100854_0001_m_000001_0 ... killing it
2014-11-10 09:14:47,697 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201411100854_0001_m_000001_0: Too many fetch-failures
2014-11-10 09:14:47,698 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_201411100854_0001_m_000001_1' to tip task_201411100854_0001_m_000001, for tracker 'tracker_GSI-2:GSI-2/127.0.0.1:45167'
2014-11-10 09:14:47,698 INFO org.apache.hadoop.mapred.JobInProgress: Choosing rack-local task task_201411100854_0001_m_000001
2014-11-10 09:14:50,693 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201411100854_0001_m_000001_0' from 'tracker_GSI-1:localhost/127.0.0.1:42259'
2014-11-10 09:14:59,708 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201411100854_0001_m_000001_1' has completed task_201411100854_0001_m_000001 successfully.
2014-11-10 09:14:59,708 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:73 completedMapsInputSize:31979351199 completedMapsOutputSize:1606
2014-11-10 09:17:26,812 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #1 for task attempt_201411100854_0001_m_000002_0
2014-11-10 09:20:02,916 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #2 for task attempt_201411100854_0001_m_000002_0
2014-11-10 09:25:03,127 INFO org.apache.hadoop.mapred.JobInProgress: Failed fetch notification #3 for task attempt_201411100854_0001_m_000002_0
2014-11-10 09:25:03,127 INFO org.apache.hadoop.mapred.JobInProgress: Too many fetch-failures for output of task: attempt_201411100854_0001_m_000002_0 ... killing it
2014-11-10 09:25:03,127 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201411100854_0001_m_000002_0: Too many fetch-failures
2014-11-10 09:25:03,128 INFO org.apache.hadoop.mapred.JobTracker: Adding task 'attempt_201411100854_0001_m_000002_1' to tip task_201411100854_0001_m_000002, for tracker 'tracker_GSI-2:GSI-2/127.0.0.1:45167'
2014-11-10 09:25:03,128 INFO org.apache.hadoop.mapred.JobInProgress: Choosing rack-local task task_201411100854_0001_m_000002
2014-11-10 09:25:06,146 INFO org.apache.hadoop.mapred.JobTracker: Removed completed task 'attempt_201411100854_0001_m_000002_0' from 'tracker_GSI-1:localhost/127.0.0.1:42259'
2014-11-10 09:25:15,138 INFO org.apache.hadoop.mapred.JobInProgress: Task 'attempt_201411100854_0001_m_000002_1' has completed task_201411100854_0001_m_000002 successfully.
2014-11-10 09:25:15,138 INFO org.apache.hadoop.mapred.ResourceEstimator: completedMapsUpdates:74 completedMapsInputSize:33053093024 completedMapsOutputSize:1628
Task tracker log
2014-11-10 09:06:12,777 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 127.0.0.1:50060, dest: 127.0.0.1:43401, bytes: 22, op: MAPRED_SHUFFLE, cliID: attempt_201411100854_0001_m_000000_1
2014-11-10 09:06:17,272 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:06:23,275 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:06:26,276 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:06:32,279 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:06:38,281 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:06:41,283 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:06:47,286 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:06:53,288 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:06:56,290 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:07:02,292 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:07:08,295 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:07:11,297 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:07:17,299 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:07:23,302 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:07:26,304 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:07:32,306 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:07:38,309 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:07:41,310 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:07:47,313 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:07:53,315 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:07:56,317 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:08:02,320 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:08:08,322 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:08:11,324 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:08:17,327 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:08:23,330 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:08:26,331 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:08:32,334 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:08:38,337 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:08:41,339 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:08:47,341 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:08:53,344 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:08:56,346 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:09:02,348 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:09:08,351 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:09:11,353 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:09:17,355 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:09:23,358 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:09:26,360 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:09:32,363 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:09:38,366 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:09:41,367 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:09:47,370 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:09:53,373 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:09:56,375 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:10:02,377 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:10:08,380 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:10:11,382 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:10:17,385 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:10:23,387 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:10:26,389 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:10:32,392 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:10:38,394 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:10:41,396 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:10:47,399 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:10:53,401 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:10:56,403 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:11:02,405 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:11:08,408 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:11:11,410 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:11:17,412 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:11:23,415 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:11:26,416 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:11:32,419 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:11:38,422 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:11:41,423 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:11:47,426 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:11:53,429 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:11:56,430 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:12:02,433 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:12:08,435 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:12:11,437 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:12:17,440 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:12:23,442 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:12:26,444 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:12:32,446 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:12:38,449 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:12:41,451 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:12:47,454 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:12:53,456 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:12:56,458 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:13:02,460 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:13:08,463 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:13:11,465 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:13:17,467 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:13:23,470 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:13:26,471 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:13:32,474 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:13:38,477 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:13:41,478 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:13:47,481 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:13:53,483 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:13:56,485 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:14:02,487 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:14:08,490 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:14:11,491 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:14:17,494 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:14:23,496 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:14:26,498 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:14:32,501 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:14:38,503 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:14:41,505 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:14:47,507 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:14:53,510 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:14:56,511 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:15:02,514 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:15:08,516 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:15:11,518 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:15:26,525 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:15:32,527 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:15:38,530 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:15:41,531 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:15:47,534 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:15:53,536 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:15:56,538 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:16:02,540 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:16:08,543 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:16:10,964 INFO org.apache.hadoop.mapred.TaskTracker: LaunchTaskAction (registerTask): attempt_201411100854_0001_m_000001_1 task's state:UNASSIGNED
2014-11-10 09:16:10,964 INFO org.apache.hadoop.mapred.TaskTracker: Trying to launch : attempt_201411100854_0001_m_000001_1
2014-11-10 09:16:10,964 INFO org.apache.hadoop.mapred.TaskTracker: In TaskLauncher, current free slots : 2 and trying to launch attempt_201411100854_0001_m_000001_1
2014-11-10 09:16:10,983 WARN org.apache.hadoop.fs.FileSystem: "server093:54310" is a deprecated filesystem name. Use "hdfs://server093:54310/" instead.
2014-11-10 09:16:10,985 WARN org.apache.hadoop.fs.FileSystem: "server093:54310" is a deprecated filesystem name. Use "hdfs://server093:54310/" instead.
2014-11-10 09:16:10,992 INFO org.apache.hadoop.mapred.JvmManager: In JvmRunner constructed JVM ID: jvm_201411100854_0001_m_-136852211
2014-11-10 09:16:10,993 INFO org.apache.hadoop.mapred.JvmManager: JVM Runner jvm_201411100854_0001_m_-136852211 spawned.
2014-11-10 09:16:11,489 INFO org.apache.hadoop.mapred.TaskTracker: JVM with ID: jvm_201411100854_0001_m_-136852211 given task: attempt_201411100854_0001_m_000001_1
2014-11-10 09:16:11,544 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:16:17,547 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
2014-11-10 09:16:17,903 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_m_000001_1 0.0% hdfs://server093:54310/user/hive/warehouse/demo.db/tss_voice_svc_fct/TSS_Voice_SVC_FCT_01.csv:3221225472+1073741824
2014-11-10 09:16:20,907 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_m_000001_1 0.0% hdfs://server093:54310/user/hive/warehouse/demo.db/tss_voice_svc_fct/TSS_Voice_SVC_FCT_01.csv:3221225472+1073741824
2014-11-10 09:16:22,439 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_m_000001_1 0.0% hdfs://server093:54310/user/hive/warehouse/demo.db/tss_voice_svc_fct/TSS_Voice_SVC_FCT_01.csv:3221225472+1073741824
2014-11-10 09:16:22,440 INFO org.apache.hadoop.mapred.TaskTracker: Task attempt_201411100854_0001_m_000001_1 is done.
2014-11-10 09:16:22,440 INFO org.apache.hadoop.mapred.TaskTracker: reported output size for attempt_201411100854_0001_m_000001_1 was 0
2014-11-10 09:16:22,441 INFO org.apache.hadoop.mapred.TaskTracker: addFreeSlot : current free slots : 2
2014-11-10 09:16:22,550 INFO org.apache.hadoop.mapred.JvmManager: JVM : jvm_201411100854_0001_m_-136852211 exited. Number of tasks it ran: 1
2014-11-10 09:16:23,549 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201411100854_0001_r_000000_0 0.15962441% reduce > copy (34 of 71 at 0.00 MB/s) >
The same hive query was running fine on a single node hadoop cluster.
