Ivy can not resolve the scope of a dependency which is a dependency of a transitive dependency

1
votes

I add a dependency(let's name it as A) to ivy.xml which has a pom file in maven central. Ivy uses ibiblio for resolving the maven dependencies. The dependency(A) which is added to ivy.xml has a transitive dependency(B). So far so good till here. The dependency(C) of transitive dependency(B) can not be resolved by ivy.

I defined A in ivy.xml like this:

<dependency org="Z" name="A" rev="0.6-SNAPSHOT" conf="*->default"/>

In pom file of B, C is defined both in compile and test scopes like below:

<dependency>
      <groupId>X</groupId>
      <artifactId>C</artifactId>
    </dependency>
    <dependency>
      <groupId>X</groupId>
      <artifactId>C</artifactId>
      <type>test-jar</type>
      <scope>test</scope>
</dependency>

When I look the xml file of B which is resolved by ivy in ivy's cache file(~/.ivy2/cache/X/C/ivy-0.98.8-hadoop2.xml), it looks like this:

<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)"/>
<dependency org="X" name="C" rev="0.98.8-hadoop2" force="true" conf="test->runtime(*),master(*)">
  <artifact name="C" type="test-jar" ext="jar" conf="" m:classifier="tests"/>
</dependency>

For this reason, ivy can not define C scopes correctly. For the record, I don't have permissions to modify the pom files as they are third party projects. How can I fix it ?

1
javamavenivytransitive-dependency
I'm quite new to Maven, but in my environment it won't pull the dependency down if the version element isn't included with the group and artifact ID's in the pom.xml - does that help? - Michael
pom file of B is a child pom. Because of that it has not version tag. BTW if I use A in a maven project there is not problem. I think ivy can not map correctly sub dependency's scope of B. - Talat
If the module is in Maven Central why not just give it as an example? As it stands I don't understand what the problem is. - Mark O'Connor
You are right @mark-oconnor . I guess I gave example because I wanted to tell problem. For concretization I give real names. I develop Nutch's Hadoop 2 support. Nutch 2 use Apache Gora for reach to Apache Hbase. Apache Nutch use ant+ivy. When I build Nutch, gora-hbase dependency and its transitive dependency hbase-client is fetched. However hbase-client's hbase-common and hbase-annotation dependencies are resolved in test scope. But they are runtime scope at the same time. Their confs is "test->runtime(),master()". When I try to run Nutch did not find and throw exception. Is it clear ? - Talat
BTW hbase-protocol is depended by hbase-client too. But it resolved and fetched. Its conf is compile, master(), runtime, compile(), runtime(*), master. It do not have any test scope defination on hbase-client pom. I think this cause the problem. @MarkO'Connor - Talat

1 Answers

1
votes

I reviewed the ivy usage of the nutch project and apologies but my conclusion is that it's overly complex for the following reasons:

  • "compile" and "test" targets are issuing separate calls to the resolve task
  • Each plugin is also calling an ivy resolve task
  • Complex logic for maintaining classpaths. Could be simplified using the cachepath task and ivy configurations.
  • Build plugins are not managed by ivy (Sonar, eclipse, rat)

I started to refactor the build, but had to stop when I realised that I didn't understand the relationship between the main nutch artifact and the plugins... (I discovered NUTCH-1515 the hard way... big time-waster The feed plugin has missing dependencies).

I also noticed issue NUTCH-1371 calling for the removal of ivy. This would be a tricky refactoring without significant change to the current codebase. I suspect it would have to be a multi-module build with each plugin listing its own dependencies.

In conclusion, this work does not answer your question, but thought I needed to at least document the result of a few hours analysis :-) In light of NUTCH-1371 I don't know if your project will tolerant major ivy refactoring?

Refactoring ivy

Here follows what I achieved so far:

Benefits:

Impacts the following Nutch issues

  • NUTCH-1881 : This new approach removes resolve-test and resolve-default targets and manages the classpaths using ivy instead of the ${build.lib.dir}
  • NUTCH-1805 : Can easily setup a separate configuration for the job target with it's own dependencies.
  • NUTCH-1755 : I think this one is fixed by assigning a name to the the build.xml (see: diff)