marklogic content pump yarn support

Question

We are running mlcp.sh in distributed mode on cdh5.2.4, the job is always running in local its not submitting to yarn/resource manager. does anyone successfully implement mlcp on cdh5+?

we are using marklogic-contentpump-1.0.5.jar

 bin/mlcp.sh export 
-host xxx.xx.xx.xxx 
-port xxxx 
-username <user> 
-password xxxxx 
-output_type sequence 
-compress_type record 
-output_file_path /tmp 
-mode distributed 
-job_queue cp11 
-query_type unfiltered 
-max_split_size 500 
-query_config file.properties 
-after_ts 2015-01-01T16:55:05-04:00 
-before_ts 2015-04-10T17:55:37-04:00  
-perm_path /data/mlcp

Answer 1

Fixed after changing from client-0.20 to client for yarn

Using JAR Files Provided in the hadoop-client Package Make sure you add to your project all of the JAR files provided under /usr/lib/hadoop/client-0.20 (for MRv1 APIs) or /usr/lib/hadoop/client (for YARN).

For example, you can add this location to the JVM classpath:

$ export CLASSPATH=/usr/lib/hadoop/client-0.20/\*