1
votes

We have a deployment in AWS where we have a single PuppetMaster box that services hundreds of other servers within the AWS ecosystem. Starting yesterday, we noticed that puppet changes were not applying to the agents. At first we thought it was only newly provisioned boxes, but now we see that we simply aren't getting any error message on any of the machines where puppet agent runs.

# puppet agent --test --verbose
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Caching catalog for blarg-follower-0e5385bace7e84fe2
Info: Applying configuration version '1529498155'
Notice: Finished catalog run in 0.24 seconds

I have access to the PuppetMaster and have validated that the code there is up to date. Need help figuring out how to get better logging out of this and debugging what is wrong between the agent and the puppet master.

1
/var/log/puppetlabs on the master and -d for an additional flag to the agent command. - Matt Schuchard
Usually when the agent finishes so quickly it means that it received an empty catalog from the master, so that it has nothing to do. When it receives a nonempty catalog it has to spend time checking the statuses of the resources within, even if it ultimately determines that no changes are required. The most recent catalog for each node is cached on both sides, so you can check that. - John Bollinger
If there was any change in manifests, you might want to download the catalog on any agent node and see if changes are applicable to that node. puppet catalog download or puppet catalog find --terminus rest. - Tejas Sarade

1 Answers

1
votes

In this case the issue was that our Puppet Master's /etc/puppet/puppet.conf file had been modified, and indeed the agents weren't getting the full catalog from Puppet Master. We found a backup copy of the file, restored it, and we were back in business.