Github to AWS EC2 using CodeDeploy best practice

Question

We are a relatively inexperienced development team trying to do things 'the right way'. We are using Github along with AWS and CodeDeploy for multiple PHP based web applications. We are utilising Github's auto-deployment with CodeDeploy when the master branch is updated.

We have two production EC2 web servers in separate AZ's along with a single EC2 staging server.

It currently works as follows:

We write code in a branch, we push to GitHub, we merge into 'master' which then kicks off CodeDeploy to write to our staging server where we can test it. Once we have tested it we then manually kick off CodeDeploy to write to production (with the same commit ID).

The problem is, if testing brings up issues, and we have another branch waiting to be merged and tested, everything becomes backed up?

We are obviously doing something wrong. We are writing to the master branch to utilise GitHub's autodeploy, but I assumed master was only to be written to when it was ready to be deployed?

Can someone please help us and put us straight?

Thanks

Answer 1

Make another branch called 'livecandidate' this branch will have each of the new feature branches merged into it

Each time a feature branch is merged into 'livecandidate' pull 'livecandidate' into your Code Deploy process and install to the test machine.

If the tests pass then merge 'livecandidate' into 'master' and kick off the install to production

If the tests do not pass then unwind the merge into 'livecandidate' (assuming no dependencies on chains of changes etc)

After doing a production install or a un-merge, try the next feature

General idea is to never ever have a broken master

All problems in computer science can be solved by another level of indirection - David Wheeler