Collect all rpm dependencies to deploy a project

Question

I need to deploy a software project (packaged as an rpm) from a developer machine into a server. I'm using Fedora 23, along with the dnf package manager. I have to collect all dependencies of my rpm before I deploy to the server. The server can't be connected to the internet due to internal regulation (But I can ssh onto it). Running repository mirrors, etc. is not an option. I'm afraid I just have to collect all dependencies on the developer machine, scp (or ansible) them to the server and install them on the server.

I hoped that --installroot option in dnf could be of much help, as I could retrieve all rpms that would get installed into what dnf thinks is an empty system. This however doesn't work.

mkdir foo && sudo dnf install --installroot=$PWD/foo golang

gives an error:

Failed to synchronize cache for repo 'fedora'

Why does this fail? What are my options?

I'd like to see an elegant and robust solution. I'd prefer not to install anything on the server (I'd be most happy to do a single scp followed by one or two commands over ssh). A combination of rpm + yum/dnf magic would be great, but other solutions, including apt + deb are also of interest. I'd prefer not to use docker, and I'm strongly against running any additional infrastructure (docker registry, rpm mirror, etc.)

Yup, seen these restrictions before and I've concluded that the necessary work-arounds make this the least secure way to install software.... Would be a lot better to enable signed package repositories, one could even control the repository contents, if worried about the packages being installed. In the absence of such, I've been forced to copy all necessary RPMs up and just install them one at a time. Lastly you could also consider the use of Docker containers, which could be built offline, signed and then pushed into a deployment registry. Likely to be far to radical an idea. Hope that helps — Mark O'Connor
@MarkO'Connor any idea why the install-root switch is not working? This sounds like the cleanest way to collect all packages. — Adam Kurkiewicz
You want a chroot installed with all prerequisites of a package called "golang"? Poky-Yocto uses RPM5 with a "solvedb" (which is nothing more than an rpmdb of "everything"). Given a solvedb (configured with a macro), then "rpm -Uvh --root=$PWD/foo +golang" will install golang and all its prereq's into a chroot in $PWD/foo. I can add further details, but be forewarned: the operation is only as good as the metadata contained in packages. — Jeff Johnson
@JeffJohnson this sounds fantastic! Could you please give me more details? Do I add this poky-yocto's solvedb the way I would add any other repository (in such way it will appear in dnf's repolist?) or do I run it locally? I'm anxious to learn more! — Adam Kurkiewicz
Creating a solvedb is a single "rpm -Uvh --justdb --dbpath /tmp/solvedb .rpm" command. The hard part is in the ".rpm" download of "everything". The solvedb contains all the package metadata, including the path to the package. The path to the solvedb is configured in RPM5 using the macro %_solvedb_dbpath. When configured, all unresolved dependencies are looked up in the solvedb to find RPMTAG_PACKAGEORIGIN (i.e. the path to the *.rpm) which is then added to the install transaction, which is then continued, finding all prereqs recursively. A complete transaction is then attempted. Make sense? — Jeff Johnson

Jeff Johnson Jeff Johnson · Accepted Answer · 2016-06-23T22:30:00

Here is a (ad hoc, lightly tested) script (assuming you have an already installed rpm system) to generate the list of all the rpm package names needed to install a given package (the script assumes goal="bash", edit to taste).

Feed the output names to dnf/yum to install.

#!/bin/sh
goal=bash
deps=$(rpm -q --qf '[%{REQUIRENAME}\n]'  $goal | egrep -v '^(rpmlib|rtld|config|/)')
goals=
while true; do
  subs=$(rpm -q --qf '%{NAME}\n' --whatprovides $deps | sort -u | tr '\n' ' ')
  if [ ."$subs" = ."$goals" ]; then
    echo "--- packages needed"
    echo "$goals" | tr ' ' '\n'
    exit 0
  fi
  goals=$(echo $goals $subs | tr ' ' '\n' | sort -u | tr '\n' ' ')
  for sub in $subs; do
    subdeps=$(rpm -q --qf '[%{REQUIRENAME}\n]' $sub | egrep -v '^(rpmlib|rtld|config|/)')
    deps=$(echo $deps $subdeps | sort -u)
  done
done

Collect all rpm dependencies to deploy a project

1 Answers