Failing To Find Chef Community Cookbooks Using Chef 12 On AWS Opsworks

Question

The Problem

When running custom cookbooks on AWS OpsWorks a setup_failed status appears against the instance and the following in the failure log:

[2016-03-26T22:53:48+00:00] INFO: Started chef-zero at chefzero://localhost:8889 with repository at /var/chef
One version per cookbook
data_bags at /var/chef/runs/62832572-cb67-421a-8309-d831140d7b98/data_bags
nodes at /var/chef/runs/62832572-cb67-421a-8309-d831140d7b98/nodes
 
[2016-03-26T22:53:48+00:00] INFO: Forking chef instance to converge...
[2016-03-26T22:53:48+00:00] INFO: *** Chef 12.7.2 ***
[2016-03-26T22:53:48+00:00] INFO: Chef-client pid: 17842
[2016-03-26T22:53:49+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found: chefzero://localhost:8889/nodes/test1.localdomain
[2016-03-26T22:53:49+00:00] INFO: Setting the run_list to ["recipe[my_cookbook::default]"] from CLI options
[2016-03-26T22:53:49+00:00] INFO: Run List is [recipe[my_cookbook::default]]
[2016-03-26T22:53:49+00:00] INFO: Run List expands to [my_cookbook::default]
[2016-03-26T22:53:49+00:00] INFO: Starting Chef Run for test1.localdomain
[2016-03-26T22:53:49+00:00] INFO: Running start handlers
[2016-03-26T22:53:49+00:00] INFO: Start handlers complete.
[2016-03-26T22:53:49+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found: 
[2016-03-26T22:53:49+00:00] INFO: HTTP Request Returned 412 Precondition Failed: No such cookbook: apt
 
================================================================================
Error Resolving Cookbooks for Run List:
================================================================================
 
Missing Cookbooks:
------------------
No such cookbook: apt
 
Expanded Run List:
------------------
* my_cookbook::default

The OpsWorks stack is configured

for Chef 12
to use custom Chef cookbooks
to use a Git repository for the custom cookbooks

The Custom Cookbook Repo

For the sake of finding the cause of the problem, a cookbook was created with a single recipe that runs apt-get update (which runs as expected under kitchen converge). The files were created using chef generate cookbook my_cookbook:

.
└── my_cookbook
    ├── Berksfile
    ├── chefignore
    ├── metadata.rb
    ├── README.md
    ├── recipes
    │   └── default.rb
    ├── spec
    │   ├── spec_helper.rb
    │   └── unit
    │       └── recipes
    │           └── default_spec.rb
    └── test
        └── integration
            ├── default
            │   └── serverspec
            │       └── default_spec.rb
            └── helpers
                └── serverspec
                    └── spec_helper.rb

The only altered files are:

my_cookbook/Berksfile

source 'https://supermarket.chef.io'    
metadata    
cookbook 'apt'

my_cookbook/metadata.rb

...
depends 'apt', '~> 3.0.0'

my_cookbook/recipes/default.rb

include_recipe 'apt::default'

Also, a Berksfile was added to the root of the repo

source 'https://supermarket.chef.io'
cookbook 'apt'

The Current Workaround

The current workaround involves downloading the dependencies using berks vendor and then copying the downloaded cookbooks into the root of the repo ie:

.
├── Berksfile
├── apt
└── my_cookbook

Is there a better solution (using AWS OpsWorks with Chef 12) that does not require explicitly storing and distributing community cookbooks with custom cookbooks?

coderanger coderanger · Accepted Answer · 2016-03-27T01:17:36

I'm pretty sure this is how you are supposed to do it now. They removed a bunch of the automated berks integration between the Chef 11 and Chef 12 stacks for reasons I've never understood.