AWS Pass in variable into buildspec.yml from CodePipeline

Question

I have an AWS CodePipeline that invokes CodeBuild in the Build Stage.

The question is how do I pass in an environment variable from CodePipeline that can be read in the CodeBuild's buildspec.yml?

I know I can set environment variables in CodeBuild, but I want to use the same CodeBuild project for dev, qa, and prod environments. I don't see how I can pass an environment variable from CodePipeline that makes it all the way to the buildspec.yml

Example buildspec.yml

version: 0.1

phases:   
  build:
    commands:
      - npm install
      - npm build -- --env ${CURRENT_ENVIRONMENT}

Where CURRENT_ENVIRONMENT would be the variable I set in the CodePipeline Stage action.

Answer 1

As of today, you can set environment variables for CodeBuild build jobs in your pipeline. This improvement makes it possible to reuse the same build project for multiple actions and simplify deployments to staging and production environments in your pipeline.

name: Build
actions:
  - name: MyBuildJob
    actionTypeId:
      category: Build
      owner: AWS
      provider: CodeBuild
      version: '1'
    runOrder: 1
    configuration:
      ProjectName: my-build-project
      PrimarySource: MyApplicationSource1
      EnvironmentVariables: '[{"name":"CURRENT_ENVIRONMENT","value":"Production","type":"PLAINTEXT"},
                              {"name":"UseParamStore","value":"CURRENT_ENVIRONMENT","type":"PARAMETER_STORE"}]'

https://docs.aws.amazon.com/codepipeline/latest/userguide/action-reference-CodeBuild.html#action-reference-CodeBuild-config

Answer 2

You can actually pass environment variables in CodeBuild cloudformation as below:

Build:
    Type: AWS::CodeBuild::Project
    Properties:
      Name:
        !Sub Build-${AWS::StackName}
      Description: Build your project
      Environment:
        Type: LINUX_CONTAINER
        ComputeType: BUILD_GENERAL1_SMALL
        Image: node8
        EnvironmentVariables:
          - Name: CURRENT_ENVIRONMENT
            Type: PLAINTEXT
            Value: staging

And in your buildspec.yml you can reference the environment like this,

version: 0.2

phases:
  install:
    commands:
      - npm install

  build:
    commands:
      - npm build -- --env ${CURRENT_ENVIRONMENT}

Answer 3

This feature isn't available today.

A workaround would be to create a different CodeBuild project for each stage with different environment variables.

You can find details on using an environment variable in your builspec.yml commands here: http://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html

Answer 4

If you don't want to use CF, you can set ENV vars on your CodeBuild project in the AWS UI.

In AWS, go to your code builder project, in the top right corner click "edit" and select "environment." On the edit environment page, click the "Additional Configuration" drop down. In there you will see inputs for "name" and "value." "Name" is where you set your ENV and "value" is where you set your value for that variable.

Example: set API_ENV in "name" and development in "value." Then, in your buildspec.yml, you can use $API_ENV.

Answer 5

You can use build environment variables to conditionally pass from buildspec to npm build by detecting which CodeBuild job or CodePipeline job is running. If you have one CodePipeline listening to /dev commits and another listening to /master commits, for example, this works perfectly.

Here's an example which runs a different PROD vs DEV build:

  build:
    commands:
   - |
     if expr "${CODEBUILD_BUILD_ARN}" : ".*build/MyProjectDev-" >/dev/null; then
       yarn run build-dev;
     fi
   - |
     if expr "${CODEBUILD_BUILD_ARN}" : ".*build/MyProject-" >/dev/null; then
       yarn run build-prod;
     fi

Answer 6

I am sure there are many answers posted above but putting some extra info which I found useful.

CodePipeline started supporting the environment variable at the Pipeline level where some of have native support via Pipeline (CodePipeline-generated pipeline variables)

So for example if we need to get the CommitId in any stage then we can declare the environment variable as per below.

1. Give any name to variable namespace on source stage for example "SourceVariables"
2. Use that namespace in any stage as "#{SourceVariables.CommitId}" in this case its getting use in build stage.

Also we can generate our own environment variable from any stage which will be available to use in next stages. More info is available here.

Going back to now original question, we can set the environment variable on build action which will help us to use it along with build spec file. But still it will not allow us to dynamically pass it during every push of code from source. And due to that it require two steps like update pipeline and then trigger the Pipeline so that it can go to respective stage.

Answer 7

1. Create a Codebuild project.
2. Create a pipeline project inside Codepipline.
3. Add a CodeBuild action and define your variables in this action.
   For example:
   Variable name is: TAG
   Variable value is: staging

Using the variable you defined in the Codepipline Buildspec file in the Codebuild project:

the logging results:

Answer 8

I've created a lambda function that updates an existing codebuild project's environment variables. You can the start the build (codebuild.start) after updating the variables. Looks something like this (nodejs):

var params = {
    "name": "Build-project-name",
    "description": "Build this project.",

    "environment": {
        "type": "LINUX_CONTAINER",
        "image": "aws/codebuild/standard:1.0",
        "computeType": "BUILD_GENERAL1_LARGE",
        "environmentVariables": [
        {
            "name": "MY_ENVIRONMENT_VARIABLE",
            "value": "VALUE_OF_ENVIRONMENT_VARIABLE",
            "type": "PLAINTEXT"
        }]
    }
}

codebuild.updateProject(params, function(err, data) {
    if (err) {
        console.log(err, err.stack); // an error occurred
    }
    else     {
        console.log(data);           // successful response
    }
});

Answer 9

The CodeBuild initiator is an environment variable in CodeBuild and this can be used to read in the CodePipeline name.

So if you include your environment in your CodePipeline name as a suffix (-dev or -prod, for instance), then you can parse it out like so.

version: 0.2
phases:
  build:
    commands:
      - CURRENT_ENVIRONMENT=`echo $CODEBUILD_INITIATOR | cut -d '-' -f2 | tr '[:upper:]' '[:lower:]'`
      - echo "My env is $CURRENT_ENVIRONMENT"

Answer 10

I created a small python script to parse the $CODEBUILD_INITIATOR variable that is passed. Below is the buildspec.yml and below that is the python script that I include with the build and call.

      build:
        commands:
          - |
            PIPELINE_ENV=$(python3 codebuild_env_parser.py $CODEBUILD_INITIATOR)
            OUTPUT_STATUS=$?
            if [ "$OUTPUT_STATUS" = "0" ]; then
              echo "Success finding a valid environment from codebuild_env_parser.py."
            else
              echo "Failure finding a valid environment from codebuild_env_parser.py. Check the script to see if the codepipeline env was passed correctly."
            fi

Python script (codebuild_env_parser.py):

    import sys
    
    
    def main():
        args = sys.argv
        if len(args) == 2:
            env_list = ["dev", "prod"]
            pipeline_invoker = args[1].lower()
            code_pipeline_name = pipeline_invoker.split("codepipeline/")[1]
            env_name = code_pipeline_name.split("-project-name")[0]
            if env_name in env_list:
                print("{}".format(env_name))
                sys.exit(0)
            else:
                sys.exit(1)
        else:
            sys.exit(1)
    
    
    main()

You'll have to tweak some variable values here if you want this to work. Namely, "-project-name".