Hive with emrfs

0
votes

I am importing tables from Amazon RDS to Hive using sqoop. The process is working and the data is being stored in the hive default hdfs directory : /user/hive/warehouse.

I need to change the storage location from hdfs to emrfs s3.

It is my understanding that I need to change (in hive-site.xml on the master node) value of the property hive.metastore.warehouse.dir to the s3//bucket/warehouse-location. It appears that I don't have the permission to modify the file hive-site.xml.

I am looking for some advise on how best to do it.

Sudi

2
hadoophiveamazon-emr
Change to root user. – loneStar

2 Answers

1
votes

You will need sudo privileges to modify the hive-site.xml file on the masternode (located in /etc/hive/conf/hive-site.xml usually).

If this is not an option, try setting this property before the cluster is started. An example with CloudFormation : 

                "Configurations" : [
                {
                    "Classification" : "hive-site",
                    "ConfigurationProperties" : {
                        "hive.metastore.warehouse.dir" : "s3://your_s3_bucket/hive_warehouse/",
                    }
                }
            ],

Or through the EMR dialogue in the section for "Edit Software Settings"

0
votes
sudo vi /etc/hive/conf/hive-site

or 

sudo -su root
vi /etc/hive/conf/hive-site.xml