Running Hbase ImportTSV job remotely

1
votes

I am trying to run HBase importTSV hadoop job to load data into HBase from a TSV file. I am using the following code. 

    Configuration config = new Configuration();
    Iterator iter = config.iterator();
    while(iter.hasNext())
    {
        Object obj = iter.next();
        System.out.println(obj);
    }

    Job job = new Job(config);
    job.setJarByClass(ImportTsv.class);
    job.setJobName("ImportTsv");
    job.getConfiguration().set("user", "hadoop");
    job.waitForCompletion(true);

I am getting this error

ERROR security.UserGroupInformation: PriviledgedActionException as:E317376 cause:org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=E317376, access=WRITE, inode="staging":hadoop:supergroup:rwxr-xr-x

I dont know how user name E317376 is being set. This is my windows machine user from where I am trying to run this job in a remote cluster. My haddop user account in linux machine is "hadoop"

when i run this in linux machine which is part of Hadoop cluster under hadoop user account, everything works well. But I want to programatically run this job in a java web application. Am I doing anything wrong. Please help...

2
javahadoophbase

2 Answers

2
votes

you should have a property like bellow in your mapred-site.xml file

<property>
<name>mapreduce.jobtracker.staging.root.dir</name>
<value>/user</value>
<property>

and maybe it is necessary to chmod the /user folder of your dfs file system to 777

do not forget to stop/start your jobtrackers and tasktrackers (sh stop-mapred.sh and sh start-mapred.sh)

0
votes

I havent tested these solutions, but try adding something like this in your job configuration

conf.set("hadoop.job.ugi", "hadoop");

The above may be abolete so you can also try the following with user set as hadoop (code from http://hadoop.apache.org/common/docs/r1.0.3/Secure_Impersonation.html) :

UserGroupInformation ugi = 
                     UserGroupInformation.createProxyUser(user, UserGroupInformation.getLoginUser());
             ugi.doAs(new PrivilegedExceptionAction<Void>() {
               public Void run() throws Exception {
                 //Submit a job
                 JobClient jc = new JobClient(conf);
                 jc.submitJob(conf);
                 //OR access hdfs
                 FileSystem fs = FileSystem.get(conf);
                 fs.mkdir(someFilePath); 
               }
             }