I'm trying to use my Hadoop Multinode Cluster :
- 1 Namenode (master)
- 2 Datanodes (slave1 & slave2)
I would like to make some tests with MapReduce but I'm getting an issue and I don't find anywhere to solve this one.
I uploaded to my HDFS a file called data.txt
I created both files : mapper.py and reducer.py which are stored in my hadoop local repository.
I executed this command :
hadoop jar /usr/local/hadoop-2.7.5/share/hadoop/tools/lib/hadoop-streaming-2.7.5.jar
-mapper /usr/local/hadoop/mapper.py -reducer /usr/local/hadoop/reducer.py
-input hdfs://master:54310/data.txt -output hdfs://master:54310/output.txt
In my Terminal, I have this issue :
packageJobJar: [/tmp/hadoop-unjar6386556681700293769/] [] /tmp/streamjob2613722562503212451.jar tmpDir=null
18/03/16 15:45:02 INFO client.RMProxy: Connecting to ResourceManager at master/172.30.10.64:8050
18/03/16 15:45:03 INFO client.RMProxy: Connecting to ResourceManager at master/172.30.10.64:8050
18/03/16 15:45:05 INFO mapred.FileInputFormat: Total input paths to process : 1
18/03/16 15:45:05 INFO mapreduce.JobSubmitter: number of splits:2
18/03/16 15:45:05 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1521211085961_0001
18/03/16 15:45:06 INFO impl.YarnClientImpl: Submitted application application_1521211085961_0001
18/03/16 15:45:06 INFO mapreduce.Job: The url to track the job: http://master:8088/proxy/application_1521211085961_0001/
18/03/16 15:45:06 INFO mapreduce.Job: Running job: job_1521211085961_0001
18/03/16 15:45:21 INFO mapreduce.Job: Job job_1521211085961_0001 running in uber mode : false
18/03/16 15:45:21 INFO mapreduce.Job: map 0% reduce 0%
18/03/16 15:45:39 INFO mapreduce.Job: Task Id : attempt_1521211085961_0001_m_000001_0, Status : FAILED
Error: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:450)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1754)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
... 9 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:38)
... 14 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
... 17 more
Caused by: java.lang.RuntimeException: configuration exception
at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:222)
at org.apache.hadoop.streaming.PipeMapper.configure(PipeMapper.java:66)
... 22 more
Caused by: java.io.IOException: Cannot run program "/usr/local/hadoop/mapper.py": error=2, Aucun fichier ou dossier de ce type
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at org.apache.hadoop.streaming.PipeMapRed.configure(PipeMapRed.java:209)
... 23 more
Caused by: java.io.IOException: error=2, Aucun fichier ou dossier de ce type
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
... 24 more
18/03/16 15:45:39 INFO mapreduce.Job: Task Id : attempt_1521211085961_0001_m_000000_0, Status : FAILED
Error: java.lang.RuntimeException: Error in configuring object
So I looked to : http://172.30.10.64:8088/proxy/application_1521211085961_0001/
HTTP ERROR 500
Problem accessing /proxy/application_1521211085961_0001/. Reason:
Connection to http://slave1:8088 refused
Caused by:
org.apache.http.conn.HttpHostConnectException: Connection to http://slave1:8088 refused
It's very strange because my both datanodes are well-configured (I can ping, I can connect thanks to ssh, ...)
My configuration :
First step :
/etc/hosts are good on each node :
- 172.30.10.64 master
- 172.30.10.72 slave1
- 172.30.10.62 slave2
Second step :
/etc/hostname are good on each node :
- master for master node
- slave1 for slave1 node
- slave2 for slave2 node
Third step :
The command jps gives me on master node :
4131 NameNode
4501 ResourceManager
5049 Jps
4347 SecondaryNameNode
and on slave nodes :
2357 DataNode
2491 NodeManager
2957 Jps
So why my namenode get refused connection to slave1 ?
EDIT:
I installed Hadoop 2 years ago on Proxmox with 3 VM's in order to simulate each node and I had NAT mode enabled.
Now, I'm working with VMWare VSphere and I don't have NAT mode.
My issue could comes from this option ??
