AWS codepipeline, dockerfile is not able to access env variable CODEBUILD_SRC_DIR and CODEBUILD_SRC_DIR_SecondarySource

Question

I am using aws codepipeline. I have 2 codecommit repo say source1 and source2. I am using codepipeline for CI/CD.

Codepipeline that i have created, is using both the codecommit repo i.e. source1 and source2 in codepipeline's source. Now Codebuild is also using both the input source i.e source1 and source2 in its Input Artifacts. Source1 is primary and source2 is secondary Input artifact

I have a buildspec.yml file which is using dockerfile stored in the root directory of source1 to build the code.

Now issue is, dockerfile is not able to copy source2 code in the container. i.e say source1 has folder abc in that and source2 has folder xyz in that

I am doing below in docker file

COPY ./abc /source1/abc/ --working

COPY ./xyz /source2/xyz/ --Not working,getting below error

COPY failed: stat /var/lib/docker/tmp/docker-builder297252497/xyz: no such file or directory.

then i tried below in dockerfile

COPY ./abc /source1/abc/ --working

COPY $CODEBUILD_SRC_DIR_source2/xyz /source2/xyz/ --Not working,getting same error

also tried to CD in $CODEBUILD_SRC_DIR_source2 and than run COPY command, but same error.

Afterwards, I tried printing PWD,CODEBUILD_SRC_DIR,CODEBUILD_SRC_DIR_source2 in both yaml as well as in dockerfile. it yields below output

in yaml file

echo $CODEBUILD_SRC_DIR prints --> /codebuild/output/src886/src/s3/00

echo CODEBUILD_SRC_DIR_source2 --> /codebuild/output/src886/src/s3/01

echo $PWD --> /codebuild/output/src886/src/s3/00

in dockerfile

echo $CODEBUILD_SRC_DIR prints --> prints nothing

echo CODEBUILD_SRC_DIR_source2 --> prints nothing

echo $PWD --> print '/'

Seems like dockerfile doesn't have access to CODEBUILD_SRC_DIR and CODEBUILD_SRC_DIR_source2 env variables.

Anyone have any idea how can i access CODEBUILD_SRC_DIR_source2 or source2 in dockerfile so that I can copy them in container and make codebuild successful.

Thanks in Advance !!!

Answer 1

Adding answer for anyone else who is facing the same issue. Hope this will help someone !

The issue was regarding build context passed to docker when there is only one repo as input source, then codebuild uses this directory to build as pwd --> CODEBUILD_SRC_DIR=/codebuild/output/src894561443/src The source in first repo (in case only one repo) is present in the same directory i.e. CODEBUILD_SRC_DIR=/codebuild/output/src894561443/src

and in buildspec.yml file we had following command to build the image docker build -t tag . (uses the dockerfile present in root directory of first source)

But when we have multiple source then code build stores the input artifacts like this

CODEBUILD_SRC_DIR=/codebuild/output/src886/src/s3/00
CODEBUILD_SRC_DIR_source2=/codebuild/output/src886/src/s3/01

instead of CODEBUILD_SRC_DIR=/codebuild/output/src886/src/ where CODEBUILD_SRC_DIR is first input artifact(1st codecommit repo) and CODEBUILD_SRC_DIR_source2 is second input artifact (2nd codecommit repo)

In this case codebuild was using directory -> CODEBUILD_SRC_DIR=/codebuild/output/src886/src/s3/00 as pwd

So below command where context was passed as dot '.' (pwd) docker build -t tag . As a result only first source was passed to the docker as CODEBUILD_SRC_DIR was PWD and docker was failed to refer to the second source.

To fix this we passed the parent directory of CODEBUILD_SRC_DIR=/codebuild/output/src886/src/s3/00 i.e /codebuild/output/src886/src/s3/ in docker build command like this.

docker build -t tag -f $CODEBUILD_SRC_DIR/Dockerfile /codebuild/output/src886/src/s3/

and in dockerfile reffered the source1 and source2 as below

source1=./00
source2=./01

and it worked !!!