How do I use Artifactory to mirror linux distributions?

Question

I configured external yum and apt repositories for Artifactory, for CentOS, Debian and Ubuntu and it seems as working but Artifactory does not cache/mirror them in advance. It seems that the artifacts are cached the first time they are requested and I do want to be sure that I pre-cache them.

I imagined that this would be done by replication option but somehow it seems that this option require an Artifactory server on the other side, which I obviously do not have as these are just public http mirrors, like:

• http://mirror.bytemark.co.uk/centos/
• http://ftp.uk.debian.org/debian/

• http://mirror.bytemark.co.uk/ubuntu/

  How do I perform the caching/mirroring?

Answer 1

All your observations and assumptions are correct. Arifactory remote repositories are lazy proxies and download the artifacts only on demand. Replication can pre-populate the caches, but it requires Artifactory instances on both sides (because of the checksum-based replication algorithm it uses).

If you're sure you want to pre-populate Artifactory with all the artifacts from those repositories (we don't see this demand justified usually), the easiest way will be to use a web crawler on build the list of all the packages and then issue a HEAD request to those packages via Artifactory.