5
votes

How can I dynamically set up an additional classpath so that jars are accessible from a hadoop tool class? I'm running my job via "hadoop jar". I need something like the "-libjars" switch which unfortunately works only for mappers and reducers, and my Tool implementations does not see jars provided by "libjars"

export "HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/path/to/my.jar && hadoop jar ..." seems making "my.jar" accessible to the Tool, but then libraries that are inside the job jar in the "/lib" folder become unaccessible.

1

1 Answers

2
votes

Additionally, if you have conflicts with some jars included into a Hadoop distribution, you can export HADOOP_USER_CLASSPATH_FIRST=true. Maybe will be useful to somebody.