15
votes

I am having some trouble running a Hadoop job that includes a newer version of Guava than the one that is included in the Hadoop distribution (CDH 5.2). This is a known problem. I try to solve it by shading the libraries using the Maven shade plugin. Therefore, I added the following lines to my pom.xml:

  <plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-shade-plugin</artifactId>
    <version>2.3</version>
    <executions>
      <execution>
        <phase>package</phase>
        <goals>
          <goal>shade</goal>
        </goals>
        <configuration>
          <relocations>
            <relocation>
              <pattern>com.google</pattern>
              <shadedPattern>thirdparty.com.google</shadedPattern>
            </relocation>
          </relocations>
        </configuration>
      </execution>
    </executions>
  </plugin>

Unfortunately, the shading seems not to work. When I extract the uber-JAR there is no folder thirdparty/com/google but still the folder com/google.

Does someone have an idea what is going wrong?

3
Have you found a solution for this? Thanks! - Zongjun
it is avaibable for me, you can try it again then check the generated jar whether it takes effect. <br/> <configuration> <relocations> <relocation> <pattern>com.google</pattern> <shadedPattern>shade.com.google</shadedPattern> </relocation> </relocations> </configuration> - Jiayu Wang

3 Answers

10
votes

This worked for me:

<relocations>
   <relocation>
     <pattern>com.google.</pattern>
     <shadedPattern>thirdparty.com.google.</shadedPattern>
   </relocation>
 </relocations>

note the dot at the end of the pattern.

3
votes

You may need to specify explicit artifactSet::includes under your <configuration> section:

    <configuration>
        <artifactSet>
            <includes>
                <include>com.google.guava:*</include>
                ...
            </includes>
        </artifactSet>
        <relocations>
        ...
3
votes

Seems like you need to refer to the package name in the relocation rule, not the maven groupId, I'm using rx v1 in my lib and don't want to pollute the user's namespace with it, the pom.xml section below rewrites the bytecode, so the final uberjar will have rx, but renamed (shaded.rx).

shade relocation doc

<plugin>
  <groupId>org.apache.maven.plugins</groupId>
  <artifactId>maven-shade-plugin</artifactId>
  <version>3.0.0</version>
  <executions>
    <execution>
      <phase>package</phase>
        <goals>
          <goal>shade</goal>
        </goals>
        <configuration>
          <relocations>
            <relocation>
              <pattern>rx</pattern>
              <shadedPattern>shaded.rx</shadedPattern>
            </relocation>
          </relocations>
        </configuration>
      </execution>
    </executions>
  </plugin>