Chef cookbook organisation

1
votes

Background

Our company recently decided to finally automate our deployment processes using Chef and I have been tasked to implement this. Being more of a developer, I am currently struggeling with the best approach on how to organize and structure the approach and cookbooks in a way that I can also use them in our development environments with Vagrant.

We have ~ 10 applications that we need to deploy to several environments (development, staging, acceptance, production, demo). Each application is running on the same basic tech stack and platform. Some of these applications are actually subservices of other applications.

After reading several tutorials, learn-chef as well as watching endless videos, I have started to create several cookbooks using berkshelf (each cookbook is in it's own GIT repo):

  • 1 x cookbook to setup the basic infrastructure (setup admin user, install basic system tools, sudo, sshd, etc)
  • 1 x cookbook to setup the platform (in our case: LAMP with a fixed PHP version)
  • 10 x cookbook to setup each application

Question/s

Is this the correct approach to organize cookbooks? None of these cookbooks depend on each other (for now), even though the application cookbook require the infrastructure and platform cookbooks to be run first in order to setup the basic system. Each application has the same tech stack requirement for now - so seems kind of weird to add the same dependancies into each application cookbook. Should the exact dependancies of the applications (for packages that are installed via the platform cookbook, e.g. apt) still be explictly mentioned? How is this all then tied together to deploy it to the target machines? Do I need another "container" cookbook (single chef-repo for all machines and cookbooks?) which holds the specific applications together? Do I need roles to make this work?

2
chef-infra

2 Answers

2
votes

One of the frustrations with chef is that there are about 100 ways to do this, and the community still has not formed a full consensus on which is best. That said, if your introduction to Chef is learn chef, and you're using Berkshelf, then you should probably look to The Berkshelf Way of doing things. In this case:

  • Each library cookbook should depend on and include_recipe your platform cookbook and your infrastructure cookbook
  • Each "app cookbook" should install and configure 1 and only 1 app
  • You should create "role cookbooks" that provide default recipes and attributes required to install each logical group of applications

I would also recommend that you look into the 'silverware' cookbook by infochimps (now part of ironfan-pantry). It is a great way to tie cookbooks together without needing explicit dependencies between them. For example, it lets you declare the host and port of your database service in one cookbook, and then search for that in other cookbooks. This way, your applications don't need dependencies on your database cookbook in order to find your database service.

As for the physical layout of your cookbooks, you're doing great. One repo per cookbook is a very solid approach.

2
votes

In addition I would recommand one cookbook per use case (for LAMP, one for apache, one for mysql, one for php) to allow flexibility on upgrading each part.

With only one cookbook you may end up with conflicting versions i.e.

  • Base stack v1.0.0
  • Testing a new php version v1.1.0
  • Needing a minor mysql upgrade for a security fix => v1.0.1

Now you have a problem on v1.1.0, you need to report the change on mysql.

It sounds easy at first, but once you have 3 or 4 part moving through testing/review and you wish t promote to prod some and not the other it become a nightmare if you can't allow one part only.

If you're absolutely sure you will always have sequential update you can include all the parts in only one cookbook.