How to make Hue - Oozie workflow run a java job which has config file?

Question

I have a buildModel.jar, and a folder "conf" which contain a configuration file named config.properties.

The command line running it look like this:

hadoop jar /home/user1/buildModel.jar -t fp-purchased-products -i hdfs://Hadoop238:8020/user/user2/recommend_data/bought_together

After doing some analyze, it use the db information in "config.properties" file to store data to a mongo db.

Now i need to run it with Hue Oozie workflow, so I used Hue to upload the jar file and folder "conf" to hdfs then created a workflow. I also added "config.properties" file in workflow

This is the workflow.xml

<workflow-app name="test_service" xmlns="uri:oozie:workflow:0.4">
<start to="run_java_file"/>
<action name="run_java_file">
    <java>
        <job-tracker>${jobTracker}</job-tracker>
        <name-node>${nameNode}</name-node>
        <main-class>xxx.xxx.recommender.buildModel.Application</main-class>
        <arg>-t=fp-purchased-products</arg>
        <arg>-i=hdfs://Hadoop238:8020/user/user2/recommend_data/bought_together</arg>
        <file>/user/user2/service/build_model/conf/config.properties#config.properties</file>
    </java>
    <ok to="end"/>
    <error to="kill"/>
</action>
<kill name="kill">
    <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name="end"/>

And this is the workflow-metadata.json

{"attributes": {"deployment_dir": "/user/hue/oozie/workspaces/_user2_-oozie-31-1416890719.12", "description": ""}, "nodes": {"run_java_file": {"attributes": {"jar_path": "/user/user2/service/build_model/buildModel.jar"}}}, "version": "0.0.1"}

After doing analyze, it got error when save data to mongo db. It seem that the java file can't see the config.properties.

Can anyone guide me how to use Hue Oozie run java which has config file ?

The problem is that config.properties is copied in the same directory of the command, not in a conf directory. Do you have a way to specify the path of config.properties directly in the Hadoop command? Not sure if doing a symlink like conf/config.properties#conf/config.properties is supported but might be worth a try. — Romain
@user1452759 yes, i changed my code, and remove the config file to the same directory with buildModel.jar. — SieuCau

SieuCau SieuCau · Accepted Answer · 2015-07-13T07:27:11

Sorry for late answer.

As Romain explained above. Hue will copy the config.properties to the same directory with the BuildModel.jar. So i changed the code to let BuildModel.jar read config file at the same directory. It worked !

How to make Hue - Oozie workflow run a java job which has config file?

1 Answers