Chef-solo installs MySql and Apache2 out of order

3
votes

I'm working with chef and vagrant to better automate some machine build processes and I've run into a problem I can't seem to figure out. The issue is that no matter what we do, as soon as we include our php server cookbook it pops apache2 to the top of the run list somehow. The same goes for mysql. We'd like to use remi repos rather than being stuck with PHP 5.3.3 but I'm not able to as apache and mysql are getting installed first. IE, the expected install order is something like:

  1. run yum/apt update and install baseline packages (the
  2. baseline-linux-server recipes) install apache2 / php install mysql
  3. server

But I'm seeing:

  1. install apache2
  2. install mysql server
  3. start installing everything in the expected order -- including seeing php / apache2 / mysql server again.

The run list in the vagrantfile looks like:

chef.add_recipe "company-baseline-linux-server::enterprise-linux"
chef.add_recipe "company-baseline-linux-server"
chef.add_recipe "company-php-server"
chef.add_recipe "company-mysql-server"
chef.add_recipe "company-php-server::setup-website"
chef.add_recipe "company-wordpress-app"
chef.add_recipe "company-wordpress-app::copy-assets"
chef.add_recipe "company-wordpress-app::load-wordpress-db"

the company-php::default recipe is:

#set apache ports
node.default['apache']['listen_ports'] = node['listen_ports']

#setup apache default modules
node.default['apache']['default_modules'] = %w(status alias rewrite headers deflate dir env mod_proxy mod_proxy_http mime negotiation setenvif authz_default authz_host log_config logio)
include_recipe "apache2"
include_recipe "apache2::mod_ssl"

#set apache to autostart
execute "auto start apache" do
    command "chkconfig httpd on"
end

#add php
include_recipe "php"
include_recipe "apache2::mod_php5"
include_recipe "php::module_mysql"

#disable iptables for now
include_recipe "iptables::disabled"

#kill default site
apache_site "default" do
  enable false
end

the company-mysql::default recipe is:

node.default["mysql"]["remove_anonymous_users"] = true

include_recipe "mysql::server"

#setup remote database user
include_recipe "database"
include_recipe "database::mysql"

mysql_connection_info = {
  :host     => node['dbserver'],
  :username => 'root',
  :port     => node['mysql']['port'],
  :password => node['mysql']['server_root_password']
}

dbuser = node['remote_root_user']
dbpassword = node['remote_root_password']

mysql_database_user dbuser do
  connection    mysql_connection_info
  password      dbpassword
  host          '%'
  grant_option  true
  action        :grant
  only_if { node['remote_root_user'] }
end

mysql_database_user dbuser do
  connection    mysql_connection_info
  password      dbpassword
  host          'localhost'
  grant_option  true
  action        :grant
  only_if { node['remote_root_user'] }
end

When I run all of this I get the following output where one can see httpd installing first and then mysql and then the baseline-linux-cookbooks which start with the EPEL install:

[2013-11-29T00:53:48+00:00] INFO: Forking chef instance to converge...
[2013-11-29T00:53:48+00:00] INFO: *** Chef 11.6.0 ***
[2013-11-29T00:53:48+00:00] INFO: Setting the run_list to ["recipe[company-baseline-linux-server::enterprise-linux]", "recipe[company-baseline-linux-server]", "recipe[company-php-server]", "recipe[company-mysql-server]", "recipe[company-php-server::setup-website]", "recipe[company-wordpress-app]", "recipe[company-wordpress-app::copy-assets]", "recipe[company-wordpress-app::load-wordpress-db]"] from JSON
[2013-11-29T00:53:48+00:00] INFO: Run List is [recipe[company-baseline-linux-server::enterprise-linux], recipe[company-baseline-linux-server], recipe[company-php-server], recipe[company-mysql-server], recipe[company-php-server::setup-website], recipe[company-wordpress-app], recipe[company-wordpress-app::copy-assets], recipe[company-wordpress-app::load-wordpress-db]]
[2013-11-29T00:53:48+00:00] INFO: Run List expands to [company-baseline-linux-server::enterprise-linux, company-baseline-linux-server, company-php-server, company-mysql-server, company-php-server::setup-website, company-wordpress-app, company-wordpress-app::copy-assets, company-wordpress-app::load-wordpress-db]
[2013-11-29T00:53:48+00:00] INFO: Starting Chef Run for localhost
[2013-11-29T00:53:48+00:00] INFO: Running start handlers
[2013-11-29T00:53:48+00:00] INFO: Start handlers complete.
[2013-11-29T00:53:49+00:00] WARN: Cloning resource attributes for service[apache2] from prior resource (CHEF-3694)
[2013-11-29T00:53:49+00:00] WARN: Previous service[apache2]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/apache2/recipes/default.rb:24:in `from_file'
[2013-11-29T00:53:49+00:00] WARN: Current  service[apache2]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/apache2/recipes/default.rb:210:in `from_file'
[2013-11-29T00:53:49+00:00] WARN: Cloning resource attributes for file[/etc/httpd/conf.d/ssl.conf] from prior resource (CHEF-3694)
[2013-11-29T00:53:49+00:00] WARN: Previous file[/etc/httpd/conf.d/ssl.conf]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/apache2/recipes/default.rb:84:in `block in from_file'
[2013-11-29T00:53:49+00:00] WARN: Current  file[/etc/httpd/conf.d/ssl.conf]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/apache2/recipes/mod_ssl.rb:28:in `from_file'
[2013-11-29T00:53:49+00:00] WARN: Cloning resource attributes for template[/etc/httpd/ports.conf] from prior resource (CHEF-3694)
[2013-11-29T00:53:49+00:00] WARN: Previous template[/etc/httpd/ports.conf]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/apache2/recipes/default.rb:185:in `from_file'
[2013-11-29T00:53:49+00:00] WARN: Current  template[/etc/httpd/ports.conf]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/apache2/recipes/mod_ssl.rb:34:in `from_file'
[2013-11-29T00:53:49+00:00] WARN: Cloning resource attributes for execute[a2dissite default] from prior resource (CHEF-3694)
[2013-11-29T00:53:49+00:00] WARN: Previous execute[a2dissite default]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/apache2/definitions/apache_site.rb:34:in `block in from_file'
[2013-11-29T00:53:49+00:00] WARN: Current  execute[a2dissite default]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/apache2/definitions/apache_site.rb:34:in `block in from_file'
[2013-11-29T00:53:49+00:00] WARN: Cloning resource attributes for directory[/var/lib/mysql] from prior resource (CHEF-3694)
[2013-11-29T00:53:49+00:00] WARN: Previous directory[/var/lib/mysql]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/mysql/recipes/server.rb:117:in `block in from_file'
[2013-11-29T00:53:49+00:00] WARN: Current  directory[/var/lib/mysql]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/mysql/recipes/server.rb:117:in `block in from_file'
[2013-11-29T00:53:49+00:00] WARN: Cloning resource attributes for template[/etc/my.cnf] from prior resource (CHEF-3694)
[2013-11-29T00:53:49+00:00] WARN: Previous template[/etc/my.cnf]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/mysql/recipes/server.rb:134:in `from_file'
[2013-11-29T00:53:49+00:00] WARN: Current  template[/etc/my.cnf]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/mysql/recipes/server.rb:194:in `from_file'
[2013-11-29T00:53:49+00:00] INFO: Could not find previously defined grants.sql resource
[2013-11-29T00:54:36+00:00] INFO: package[mysql] installing mysql-5.1.69-1.el6_4 from updates repository
[2013-11-29T00:54:47+00:00] INFO: package[mysql-devel] installing mysql-devel-5.1.69-1.el6_4 from updates repository
[2013-11-29T00:55:18+00:00] WARN: Cloning resource attributes for mysql_database_user[wordpress-dba] from prior resource (CHEF-3694)
[2013-11-29T00:55:18+00:00] WARN: Previous mysql_database_user[wordpress-dba]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/company-mysql-server/recipes/default.rb:27:in `from_file'
[2013-11-29T00:55:18+00:00] WARN: Current  mysql_database_user[wordpress-dba]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/company-mysql-server/recipes/default.rb:36:in `from_file'
[2013-11-29T00:55:18+00:00] WARN: Cloning resource attributes for link[/etc/httpd/sites-enabled/cabletechtalk] from prior resource (CHEF-3694)
[2013-11-29T00:55:18+00:00] WARN: Previous link[/etc/httpd/sites-enabled/cabletechtalk]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/company-php-server/recipes/setup-website.rb:14:in `from_file'
[2013-11-29T00:55:18+00:00] WARN: Current  link[/etc/httpd/sites-enabled/cabletechtalk]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/company-php-server/recipes/setup-website.rb:19:in `from_file'
[2013-11-29T00:55:18+00:00] WARN: Cloning resource attributes for link[/srv/websites/cabletechtalk/webroot/wp-config.php] from prior resource (CHEF-3694)
[2013-11-29T00:55:18+00:00] WARN: Previous link[/srv/websites/cabletechtalk/webroot/wp-config.php]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/company-wordpress-app/recipes/default.rb:13:in `from_file'
[2013-11-29T00:55:18+00:00] WARN: Current  link[/srv/websites/cabletechtalk/webroot/wp-config.php]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/company-wordpress-app/recipes/default.rb:18:in `from_file'
[2013-11-29T00:55:18+00:00] WARN: Cloning resource attributes for service[httpd] from prior resource (CHEF-3694)
[2013-11-29T00:55:18+00:00] WARN: Previous service[httpd]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/company-php-server/recipes/setup-website.rb:23:in `from_file'
[2013-11-29T00:55:18+00:00] WARN: Current  service[httpd]: /tmp/vagrant-chef-1/chef-solo-1/cookbooks/company-wordpress-app/recipes/default.rb:22:in `from_file'
[2013-11-29T00:55:18+00:00] INFO: Adding RPM-GPG-KEY-EPEL-6 GPG key to /etc/pki/rpm-gpg/
[2013-11-29T00:55:18+00:00] INFO: remote_file[/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6] created file /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
[2013-11-29T00:55:23+00:00] INFO: remote_file[/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6] updated file contents /etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6
[2013-11-29T00:55:23+00:00] INFO: remote_file[/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6] mode changed to 644
[2013-11-29T00:55:23+00:00] INFO: remote_file[/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-6] sending run action to execute[import-rpm-gpg-key-RPM-GPG-KEY-EPEL-6] (immediate)
[2013-11-29T00:55:24+00:00] INFO: execute[import-rpm-gpg-key-RPM-GPG-KEY-EPEL-6] ran successfully
[2013-11-29T00:55:24+00:00] INFO: Adding epel repository to /etc/yum.repos.d/epel.repo
[2013-11-29T00:55:24+00:00] INFO: template[/etc/yum.repos.d/epel.repo] created file /etc/yum.repos.d/epel.repo
[2013-11-29T00:55:24+00:00] INFO: template[/etc/yum.repos.d/epel.repo] updated file contents /etc/yum.repos.d/epel.repo
[2013-11-29T00:55:24+00:00] INFO: template[/etc/yum.repos.d/epel.repo] mode changed to 644
[2013-11-29T00:55:24+00:00] INFO: template[/etc/yum.repos.d/epel.repo] sending run action to execute[yum-makecache-epel] (immediate)

I have tried most of the obvious tricks such as pushing the calls to this directly into our wordpress-app recipe and declaring the run list as a run_list instead of a bunch of includes. I suspect it is something simple and stupid but I can't seem to come up with an answer.

2
chef-infravagrantchef-solo
The "php" recipe you provided is "company-php-server", isn't it? Apache is installed before php because you include it earlier: include_recipe "apache2" is before include_recipe "php". Not very clear what is the problem actually. – Draco Ater
@DracoAter -- thanks for the feedback, I've edited the question a bit to clear things up. The issue is more that the apache and php kicks off before the recipes listed above them so I can't do something like select repos before getting into that. – Wyatt Barnett
May be "company-baseline-linux-server" recipes also include apache? Otherwise, no idea, sorry. – Draco Ater
@DracoAter thanks for looking, unfortunately it isn't that simple -- I just checked again and no apache in that cookbook at all. – Wyatt Barnett
May be "apache2" recipe installs apache in "compile phase" (using run_action)? – Draco Ater

2 Answers

3
votes

I have had limited (read: next to none) exposure to Chef, so I can't see your problem. However, I would make some suggestions in case you haven't tried:

  • Look for any override or force-override attribute types that might be causing weirdness.

  • Look for a depends apache2 in any of the metadata.rb files.

  • Just as one does writing code, build your Vagrantfile gradually and look at the first thing that causes issues to try to localize your problem.

Good luck!

1
votes

Don't have enough reputation to write comment, though it would be more appropriate then this. Think that guys are right. Here's what you might try:

Just kick "include_recipe" calls outside of your custom recipes, and include it as "chef.add_recipe" in Vagrant file. It would probably look something like this:

chef.add_recipe "company-baseline-linux-server::enterprise-linux"  
chef.add_recipe "company-baseline-linux-server"  
chef.add_recipe "company-php-server_PART1"  
#following kicked out of company-php-server::default recipe #
chef.add_recipe "apache2"                                   #
chef.add_recipe "apache2::mod_ssl"                          #
#---------------------------------------------------------- #
chef.add_recipe "company-php-server_PART2"
#following kicked out of company-php-server::default recipe # 
chef.add_recipe "php"                                       #
chef.add_recipe "apache2::mod_php5"                         #
#---------------------------------------------------------- # 
chef.add_recipe "php::module_mysql"  
chef.add_recipe "iptables::disabled"
chef.add_recipe "company-php-server_PART3"
# one line recipe that sets 'remove_anonymous_users' attribute
chef.add_recipe "mysql-remove_anonymous_users_set_true"
#following kicked out of company-mysql-server::default recipe #
chef.add_recipe "mysql::server"                               #
chef.add_recipe "database"                                    #
chef.add_recipe "database::mysql"                             #
#------------------------------------------------------------ #
chef.add_recipe "company-mysql-server"  
chef.add_recipe "company-php-server::setup-website"  
chef.add_recipe "company-wordpress-app"  
chef.add_recipe "company-wordpress-app::copy-assets"  
chef.add_recipe "company-wordpress-app::load-wordpress-db"

Don't forget to delete/comment 'include_recipe' lines from your recipes ;)

If this invocation I wrote is not in the order you wanted, then there's the problem. I just extracted your calls to Vagrantfile. Reorganise it as you wish and you'll have the problem solved. You surely included those (apache etc.) recipes somehow - we have very similar structure on our boxes, and it works just fine.

PS. If there are puritans among you, please don't judge this Vagrantfile restructuring. I made it just as an example of what I meant.