How to configure settings.xml to pull from Artifactory private virtual repository?

5
votes

I'm an Apache Archiva veteran who has recently switched over to Artifactory Pro. I've been attempting to replicate, in Artifactory, a setup where I implemented virtual, private repositories within Archiva. The end result should be that different teams have access to their own team's artifacts, remote repos, the default local repos, and a company-wide repository.

However, I'm having trouble downloading only some dependencies, that are available on and pulled from Maven Central, whenever I configure my Maven settings.xml to use my private virtual repository. If I use Artifactory's default settings.xml (libs-release, libs-snapshot, plugins-release, plugins-snapshot and no virtual repository), then I can download all dependencies that are available on Maven Central and that my projects require.

I'm running Artifactory v4.4.2 on their provided Tomcat within a Windows server 2012 virtual server. I am not running behind a proxy. I have no such issues with Archiva.

In regard to my private virtual repository setup, I have a virtual repository named "test" that is a "generic" Package Type. That repository includes the following repos:

  • remote-repos
  • libs-release
  • libs-snapshot
  • plugins-release
  • plugins-snapshot
  • test-release-local
  • test-snapshot-local
  • company-release-local
  • company-snapshot-local

The first 5 repos listed are the default virtual repository repos provided Artifactory. The last 4 repos listed are the local repos, and they are of a Generic Package Type.

I think I should use a mirror.

Here is an example settings.xml that I am using (my username and password are getting populated within the generated settings.xml that I save to my Maven conf folder):

<?xml version="1.0" encoding="UTF-8"?>
<settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" xmlns="http://maven.apache.org/SETTINGS/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <servers>
    <server>
      <username>${security.getCurrentUsername()}</username>
      <password>${security.getEscapedEncryptedPassword()!"*** Insert encrypted password here ***"}</password>
      <id>central</id>
    </server>
    <server>
      <username>${security.getCurrentUsername()}</username>
      <password>${security.getEscapedEncryptedPassword()!"*** Insert encrypted password here ***"}</password>
      <id>snapshots</id>
    </server>
  </servers>
  <mirrors>
    <mirror>
      <mirrorOf>*</mirrorOf>
      <name>test</name>
      <url>http://localhost:8082/artifactory/test</url>
      <id>test</id>
    </mirror>
  </mirrors>
  <profiles>
    <profile>
      <repositories>
        <repository>
          <snapshots>
            <enabled>false</enabled>
          </snapshots>
          <id>central</id>
          <name>libs-release</name>
          <url>http://localhost:8082/artifactory/libs-release</url>
        </repository>
        <repository>
          <snapshots />
          <id>snapshots</id>
          <name>libs-snapshot</name>
          <url>http://localhost:8082/artifactory/libs-snapshot</url>
        </repository>
      </repositories>
      <pluginRepositories>
        <pluginRepository>
          <snapshots>
            <enabled>false</enabled>
          </snapshots>
          <id>central</id>
          <name>plugins-release</name>
          <url>http://localhost:8082/artifactory/plugins-release</url>
        </pluginRepository>
        <pluginRepository>
          <snapshots />
          <id>snapshots</id>
          <name>plugins-snapshot</name>
          <url>http://localhost:8082/artifactory/plugins-snapshot</url>
        </pluginRepository>
      </pluginRepositories>
      <id>artifactory</id>
    </profile>
  </profiles>
  <activeProfiles>
    <activeProfile>artifactory</activeProfile>
  </activeProfiles>
</settings>

An example of an artifact that I cannot pull from Maven Central through my virtual repository is poi-ooxml-schemas-3.10-FINAL.jar or many Apache Maven plugins. As soon as I remove the mirror section, I can download all dependencies from Maven Central. This makes no sense to me. My virtual repository is configured to include the remote-repos virtual repository and therefore should be able to pull everything from Maven Central.

What am I doing wrong?

3
javamavenmaven-2maven-3artifactory
I should mention that I've disabled and removed the jcenter local repository from the remote-repos virtual repository. I was having trouble with that repository downloading a handful of dependencies that exist on Maven Central. I replaced it with a new local repo that points directly to Maven Central. – Chris Harris
The best way to debug such an issue is to send a trace request. This will show you exactly what Artifactory is doing behind the scenes in order to resolve the dependency. For example localhost:8082/artifactory/test/org/apache/poi/… – Dror Bereznitsky

3 Answers

6
votes

Well, it seems that the actual thing here is that this is a bug which is related to the predefined layout of the generic virtual repository. The suggested workaround, until this will be fixed, is to create a new repository layout based on the simple-default layout. So, instead of the simple-default layout: [orgPath]/[module]/[module]-[baseRev](-[fileItegRev]).[ext]

One should create a new layout which looks like this: [orgPath]/[module]/[module]-[baseRev].[ext]

Once you create this new repository layout you can change the defined layout of the repository to the newly created one.

This will fix the behaviour you mentioned.

5
votes

O.K. I figured out part of the answer myself and another part of the answer via JFrog support's help.

Here was the first set of issues that I discovered:

  1. I noticed that if I use a virtual repository that's of a Generic type with a local repository of a Generic type, then artifact retrieval would download most of my pom's dependencies but not all of the dependencies.
  2. I noticed that if I use a virtual repository that's of a Generic type with a local repository of a Maven type, then artifact retrieval would download most of my pom's dependencies but not all of the dependencies.

Those issues seem like bugs to me.

So, I ended up defining a virtual repository (named test-maven in the config below) and two local repositories, all of which were a Maven type. The two local repositories are included within the virtual repository.

I came up with a settings.xml that worked, but it was using a Mirror, which JFrog support advised me that is not a best practice. Bearing that advice in mind, here is the final settings.xml that works for me:

<?xml version="1.0" encoding="UTF-8"?>
<settings xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.1.0 http://maven.apache.org/xsd/settings-1.1.0.xsd" xmlns="http://maven.apache.org/SETTINGS/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <servers>
    <server>
      <username>USERNAME</username>
      <password>PASSWORD</password>
      <id>test-maven</id>
    </server>
  </servers>
  <profiles>
    <profile>
      <repositories>
        <repository>
          <snapshots />
          <id>test-maven</id>
          <name>test-maven</name>
          <url>http://localhost:8082/artifactory/test-maven</url>
        </repository>
      </repositories>
      <pluginRepositories>
        <pluginRepository>
          <snapshots />
          <id>test-maven</id>
          <name>test-maven</name>
          <url>http://localhost:8082/artifactory/test-maven</url>
        </pluginRepository>
      </pluginRepositories>
      <id>artifactory</id>
    </profile>
  </profiles>
  <activeProfiles>
    <activeProfile>artifactory</activeProfile>
  </activeProfiles>
</settings>

Using this config, I verified that I can download all artifacts published to Maven Central, further remote repositories, snapshots from the local snapshot Maven repository, and releases from the local snapshot Maven repository. This setup allows me to utilize private repositories per development team.

0
votes

As soon as I remove the mirror section, I can download all dependencies from Maven Central.

You cannot access the mirror because you have not specified the credentials to use. (Also you don't have anonymous access enabled)

From: https://www.jfrog.com/confluence/display/RTF/Maven+Repository#MavenRepository-ConfiguringAuthentication

Each <repository> and <mirror> element specified in the file must have a corresponding <server> element with a matching <id> that specifies the username and password.

Your mirror's id is test. So you need to add another server entry for it:

<servers>
    <server>
        <username>USERNAME</username>
        <password>PASSWORD</password>
        <id>test-maven</id>
    </server>
    <server>
        <!-- credentials for mirror -->
        <username>USERNAME</username>
        <password>PASSWORD</password>
        <id>test</id>
    </server>
</servers>

It's a tad annoying that Artifactory's "Set me up" feature for creating settings.xml files doesn't add a <server> entry for mirrors automatically.