hadoop/bin/hadoop doesn't have the example jar

Question

I installed hadoop 2.2.0 and try to run sample wordcount program. For that, first I imported data in to hdfs using:

bin/hadoop fs -copyFromLocal /home/prassanna/Desktop/input /input

After that, I tried to run word count jar file using:

root@prassanna-Studio-1558:/usr/local/hadoop# bin/hadoop jar hadoop*examples*.jar wordcount /input -output

but it showed: Not a valid JAR: /usr/local/hadoop/hadoop*examples*.jar

I checked in usr/local/hadoop/bin/hadoop directory and there is no hadoop example jar.

Answer 1

The Jar file you are looking for is in this directory:

hadoop_root/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar

and should be run with a command like this:

$ yarn jar hadoop_root/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar wordcount /input /output

Answer 2

Hadoop example jar is no more present here.

usr/local/hadoop/bin/hadoop

From hadoop version 2.x, as SAM has rightly indicated in his answer, the jar you are looking for is

$HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar

You can run it like,

hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar wordcount /input /output

Make sure /input folder has an input file to be counted in the hdfs. Also note that /output should not exist. This is for the hadoop framework to create.

Also please refer to this document to use the Hadoop2.2.0 Shell Commands. It is always a good practice to not use the deprecated version.

http://hadoop.apache.org/docs/r2.2.0/hadoop-project-dist/hadoop-common/FileSystemShell.html

Answer 3

In my hadoop, I am working in 2.4.1 so the command is

hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.4.1.jar wordcount /input /output

Answer 4

You have to compile the WordCount.java and then JAR it as follows below. I had to dig around for the lib paths, but in the end I was able to use this to compile the examples Class

[apesa@localhost ~]$ javac -classpath $HADOOP_HOME/apesa/hadoop/common/hadoop-common-2.2.0.jar:$HADOOP_HOME/apesa/hadoop/mapreduce/hadoop-mapreduce-client-core-2.2.0.jar:$HADOOP_HOME/apesa/hadoop/common/lib/commons-cli-1.2.jar -d wordcount_classes WordCount.java

Then JAR it as follows

[apesa@localhost ~]$ jar -cvf wordcount.jar -C wordcount_classes/ . 

I have not run this in a while but you may have to verify the lib files are in the same place if you get an error