We have a distributed subsystem consisting of multiple components, each component is deployed in it's own RPM package onto various different RHEL/CentOS environments. For example the components might be called:
- JL_batman,
- JL_superman, and
- JL_wonderwoman.
And they are deployed as follows:
- host1:
- JL_batman
- JL_wonderwoman
- host2:
- JL_superman
We do periodic system releases and also maintenance releases. So the initial few System Releases might look like this:
- SR1:
- JL_batman-1.0-hg123.rpm
- JL_superman-2.7-hg651.rpm
- JL_wonderwoman-1.1-hg101.rpm
- SR2:
- JL_batman-2.0-hg137.rpm
- JL_superman-2.7-hg651.rpm
- JL_wonderwoman-1.1-hg101.rpm
- SR3:
- JL_batman-2.0-hg137.rpm
- JL_superman-2.8-hg655.rpm
- JL_wonderwoman-1.1-hg101.rpm
So in each System Release not all packages are updated. Currently we use symbolic links on the YUM repo for the packages that are not updated between releases:
- SR1:
- JL_batman-1.0-hg123.rpm
- JL_superman-2.7-hg651.rpm
- JL_wonderwoman-1.1-hg101.rpm
- SR2:
- JL_batman-2.0-hg137.rpm
- JL_superman-2.7-hg651.rpm --> ../SR1/superman-2.7-hg651.rpm
- JL_wonderwoman-1.1-hg101.rpm --> ..SR1/wonderwoman-1.1-hg101.rpm
- SR3:
- JL_batman-2.0-hg137.rpm --> ../SR2/batman-2.0-hg137.rpm
- JL_superman-2.8-hg655.rpm
- JL_wonderwoman-1.1-hg101.rpm --> ..SR1/wonderwoman-1.1-hg101.rpm
Each release directory (SR1, SR2, SR3, ...) is a YUM repository. We also use symlinks to link to the following rolling repositories:
- JL-old-stable --> SR1
- JL-stable --> SR2
- JL-testing --> SR3
This is all managed on the YUM repo server using some home grown scripts to pull packages from Jenkins and put them into the JL-testing repo (replacing any old versions there). When SR3 testing is complete and it is promoted to stable, we jiggle the symlinks as follows:
- JL-old-stable --> SR2
- JL-stable --> SR3
- JL-testing --> SR4
Each production environment has the yum .repo file installed for JL-old-stable.repo and JL-stable.repo. Test environments also have a JL-testing.repo file. Then yum upgrade 'JL_*' is used on each environment to keep it up to date. Works OK but has some issues, mainly:
When SR3 is promoted to stable, but we need to rollback to SR2 we need to use something like
yum --disablerepo='JL-*' --enablerepo='JL-old-stable' downgrade 'JL-*'.There is no way to easily rollback from SR3 (stable) to SR1 apart from installing a new JL-SR1.repo file and then using
yum --disablerepo='JL-*' --enablerepo='JL-SR1' downgrade 'JL-*'.
Is there a better way?