AWS CodePipeline build lacks Git history

Question

Context:

I have a CodePipeline set up that uses CodeCommit and CodeBuild as its source and build phases.

My build includes a plugin (com.zoltu.git-versioning) that uses the Git commit history to dynamically create a build version number.

Issue:

This fails on the AWS pipeline because of it cannot find any Git information in the source used to perform the build.

Clearly the action used to checkout the source uses an export which omits the Git metadata and history.

Question:

How do I configure CodeCommit or CodePipeline to do a proper git clone? I've looked in the settings for both these components (as well as CodeBuild) and cannot find any configuration to set the command used by the checkout action.

Has anyone got CodePipeline builds working with a checkout containing full Git metadata?

Answer 1

This is currently not possible with the CodeCommit action in CodePipeline.

https://forums.aws.amazon.com/thread.jspa?threadID=248267

Answer 2

CodePipeline supports git full clone as of October: https://aws.amazon.com/about-aws/whats-new/2020/09/aws-codepipeline-now-supports-git-clone-for-source-actions/

In your console, go to the source stage and edit. You will have a new option to fully clone your git history. full clone option

In Terraform you will have to add it to the source action's configuration:

      configuration = {
           RepositoryName       = var.repository_name
           BranchName           = "master"
           OutputArtifactFormat = "CODEBUILD_CLONE_REF"
         }

More info:

https://docs.aws.amazon.com/codepipeline/latest/userguide/tutorials-codecommit-gitclone.html

Answer 3

Yes, CodePipeline supports now a Git Full Clone. You just need to do some extra steps: https://docs.aws.amazon.com/codepipeline/latest/userguide/troubleshooting.html#codebuild-role-connections

However, CodePipeline does not currently support dynamic branches, Pull Requests. See Dynamically change branches on AWS CodePipeline

Therefore, if you need to extend your pipeline for Pull Requests, I'd recommend the approach posted by Timothy Jones above.

There's one more related thing that's worth mentioning. CodeBuild has the Full Clone option as well.

As long as you do not use the Local Source cache option, the Git history is there.

When I tried to use the above mentioned cache option, I noticed that .git is not a directory. It's a file containing one line of text, e.g.:

gitdir: /codebuild/local-cache/workspace/9475b907226283405f08daf5401aba99ec6111f966ae2b921e23aa256f52f0aa/.git

I don't know why it's currently implemented like this but, it's confusing (at least for me) and I don't consider it to be the expected behavior.

Answer 4

Although CodePipeline doesn't natively support this, you can get the information by cloning the repository in CodeBuild.

To do this, you need to set the permissions correctly, then carefully clone the repository.

Permissions

To give the permissions to clone the repository you need to:

1. Give your CodeBuild role the codecommit:GitPull permission, with the resource ARN of your CodeCommit repository
2. Put git-credential-helper: yes in the env part of your buildspec file

Cloning the repo

To clone the repo, you'll need to:

1. know the clone URL and branch (CodeBuild doesn't know this information)

2. git reset back to the commit that CodeBuild is building (otherwise you'll have a race condition between commits and builds).

   git reset "$CODEBUILD_RESOLVED_SOURCE_VERSION"
   

---

If you'd like examples, I've made a detailed writeup of the process, and published an example CodePipeline stack showing it in action.