EMR configuration json

Question

According to http://docs.aws.amazon.com/cli/latest/reference/emr/create-cluster.html# the new EMR release, 4.0.0, allows for a configuration json file to configure components. Also looking at http://docs.aws.amazon.com//ElasticMapReduce/latest/ReleaseGuide/emr-configure-apps.html, I have the json structured:

[
 {
   "Classification": "mapred-site",
   "Properties": {
      "mapreduce.map.memory.mb": "4096",
      "mapreduce.map.java.opts": "-Xmx4096M"
   }
 },
 {
   "Classification": "hadoop-env",
   "Properties": {},
   "Configurations": [
       {
         "Classification": "export",
         "Properties": {
             "HADOOP_CLIENT_OPTS": "-Xmx4096M"
         }
       }
   ]
 }
]

The Hadoop env property works, but the mapred-site properties are not reflected when a create an EMR cluster. I can always set these properties from Hive, but any ideas how to make it work with configuration file?

Answer 1

I experienced a similar issue, but with the "hadoop-env" classification.

From this post: https://github.com/aws/aws-cli/issues/1502 I found that making the keywords (Classification, Properties, ...) lower case solved the problem.

Note: I am using AWS console, not CLI.