Is there a way I can develop and test Map reduce program on windows development box

0
votes

I would like to develop and test Map reduce program on windows 7 machine before deploying to Hadoop cluster.

If it is possible, can any body point me to some good resources.

I have eclipse installed on my windows machine, what else i need to develop and test map reduce program.

2
javaeclipsehadoopmapreduce
Personally I'd consider a VM or buying another machine with a real OS. srccodes.com/p/article/38/… Good luck! - Dave Newton

2 Answers

1
votes

You can test localy with MRUnit http://mrunit.apache.org/

0
votes

Linux is the only OS supported for production. I don't know whether you can install Hadoop on Windows or not (I never tried that) but I would suggest to go with a "Hadoop Distribution".

You can for example get a VirtualBox image from HortonWorks or Cloudera (major Hadoop players) and run it on your Windows machine (for development and testing purposes).

The distribution usually include the Hadoop ecosystem: Hadoop, HDFS, Pig, Hive, HCatalog, Impala ...

Everything is configured to be compatible with each other (real time saver).