How to install Hadoop 3 on AWS EMR?

1
votes

Hadoop 3 is already 15 months old, and EMR official release labels are still supporting only Hadoop 2.

I couldn't find a quick documentation on how to set up Hadoop 3.1.2 on EMR. Are most people not using it? Seems more difficult than it should be, what am I missing?

1
amazon-web-serviceshadoopamazon-emrhadoop3
It's currently not supported, so I'm afraid you're stuck with 2. - Ben Watson
EMR does not work this way. You get only those versions that Amazon installed for you. - Viacheslav Rodionov

1 Answers

1
votes

EMR did come out with the official support for hadoop 3.1 in September as part of EMR6-beta release.[1] Also, it includes support for Amazon Linux 2, and Amazon Corretto JDK 8.

[1]EMR6-beta: https://aws.amazon.com/about-aws/whats-new/2019/09/simplify-your-spark-application-dependency-management-with-docker-and-hadoop-3-with-emr-6-0-0-beta/