Upload jest code coverage results to codecov during google cloud build

Question

I'm trying to figure out how to upload my jest code coverage reports to codecov. From there documentation:

bash <(curl -s https://codecov.io/bash) -t token

So I tried to run the bash script from a cloud build step witth the following cloudbuild.yaml

steps:
  - name: node:10.15.1
    entrypoint: npm
    args: ["install"]
  - name: node:10.15.1
    entrypoint: npm
    args: ["test", "--", "--coverage"]
  - name: 'gcr.io/cloud-builders/curl'
    entrypoint: bash
    args: ['<(curl -s https://codecov.io/bash)', '-t', '$_CODECOV_TOKEN']
  - name: node:10.15.1
    entrypoint: npm
    args: ["run", "build:production"]

I get the following error:

Step #2: bash: <(curl -s https://codecov.io/bash): No such file or directory

Obviously because the <(curl -s https://codecov.io/bash) is interpreted as string while I want it to be executed.

Edit:

I've changed my build step to the following:

  - name: "gcr.io/cloud-builders/curl"
    entrypoint: bash
    args: ["./scripts/codecov-upload.bash", "$_CODECOV_TOKEN"]

And added a file codecov-upload.bash

bash <(curl -s https://codecov.io/bash) -t $1

When running my cloud build the codecov bash uploader succesfully starts. However, I doesn't manage to upload to reports to clodecov.

Here's the logs from codecov bash uploader:

Step #2: Test Suites: 1 passed, 1 total
Step #2: Tests:       1 passed, 1 total
Step #2: Snapshots:   1 passed, 1 total
Step #2: Time:        28.981s
Step #2: Ran all test suites.
Finished Step #2
Starting Step #3
Step #3: Already have image (with digest): gcr.io/cloud-builders/curl
Step #3: /dev/fd/63: option requires an argument -- t
Step #3: 
Step #3:   _____          _
Step #3:  / ____|        | |
Step #3: | |     ___   __| | ___  ___ _____   __
Step #3: | |    / _ \ / _` |/ _ \/ __/ _ \ \ / /
Step #3: | |___| (_) | (_| |  __/ (_| (_) \ V /
Step #3:  \_____\___/ \__,_|\___|\___\___/ \_/
Step #3:                               Bash-tbd
Step #3: 
Step #3: 
Step #3: x> No CI provider detected.
Step #3:     Testing inside Docker? http://docs.codecov.io/docs/testing-with-docker
Step #3:     Testing with Tox? https://docs.codecov.io/docs/python#section-testing-with-tox
Step #3:     project root: .
Step #3: /dev/fd/63: line 897: git: command not found
Step #3: /dev/fd/63: line 897: hg: command not found
Step #3:     Yaml not found, that's ok! Learn more at http://docs.codecov.io/docs/codecov-yaml
Step #3: ==> Running gcov in . (disable via -X gcov)
Step #3: ==> Python coveragepy not found
Step #3: ==> Searching for coverage reports in:
Step #3:     + .
Step #3:     -> Found 3 reports
Step #3: ==> Detecting git/mercurial file structure
Step #3: ==> Reading reports
Step #3:     + ./coverage/clover.xml bytes=163786
Step #3:     + ./coverage/coverage-final.json bytes=444241
Step #3:     + ./coverage/lcov.info bytes=71582
Step #3: ==> Appending adjustments
Step #3:     http://docs.codecov.io/docs/fixing-reports
Step #3:     + Found adjustments
Step #3: ==> Gzipping contents
Step #3: ==> Uploading reports
Step #3:     url: https://codecov.io
Step #3:     query: branch=&commit=&build=&build_url=&name=&tag=&slug=&service=&flags=&pr=&job=
Step #3:     -> Pinging Codecov
Step #3: https://codecov.io/upload/v4?package=bash-tbd&token=secret&branch=&commit=&build=&build_url=&name=&tag=&slug=&service=&flags=&pr=&job=
Step #3:     -> Uploading
Step #3:     X> Failed to upload
Step #3:     -> Sleeping for 30s and trying again...
Step #3:     -> Pinging Codecov
Step #3: https://codecov.io/upload/v4?package=bash-tbd&token=secret&branch=&commit=&build=&build_url=&name=&tag=&slug=&service=&flags=&pr=&job=
Step #3:     -> Uploading
Step #3:     X> Failed to upload
Step #3:     -> Sleeping for 30s and trying again...
Step #3:     -> Pinging Codecov
Step #3: https://codecov.io/upload/v4?package=bash-tbd&token=secret&branch=&commit=&build=&build_url=&name=&tag=&slug=&service=&flags=&pr=&job=
Step #3:     -> Uploading
Step #3:     X> Failed to upload
Step #3:     -> Sleeping for 30s and trying again...
Step #3:     -> Pinging Codecov
Step #3: https://codecov.io/upload/v4?package=bash-tbd&token=secret&branch=&commit=&build=&build_url=&name=&tag=&slug=&service=&flags=&pr=&job=
Step #3:     -> Uploading
Step #3:     X> Failed to upload
Step #3:     -> Sleeping for 30s and trying again...
Step #3:     -> Uploading to Codecov
Step #3:     HTTP 400
Step #3: missing required properties: ['commit']
Finished Step #3
Starting Step #4
Step #4: Already have image: node:10.15.1
Step #4: 

I've noticed two things in the logs:

1. Step #3: /dev/fd/63: option requires an argument -- t
2. Step #3: missing required properties: ['commit']

When searching to fix number 2, I found the following on SO: codecov.io gives error in combination with Bitbucket pipelines

Where the answer seems to be to that git is not installed in my container.

So I've tried to make a custom container image with docker:

Dockerfile:

FROM gcr.io/cloud-builders/curl
RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y git

So I build the image:

build -t "gcr.io/[PROJECT_ID]/builder .

And update my build step to use this image instead:

• name: "gcr.io/$PROJECT_ID/builder" entrypoint: bash args: ["./scripts/codecov-upload.bash"]

But using the image created with that dockerfile returns the same errors.

Maybe the Dockerfile for that custom image isn't correct? Or I'm missing something else?

My code is available on github: https://github.com/thdk/timesheets/tree/feat/112-1

Answer 1

After the answer from Ajordat , an answer on the community of codecov and looking into the source code of the codecov batch uploader I discovered that some environment variables are required for the bash uploader to work.

I've changed my build step in cloudbuild.yaml to include environmental variables. The values for these are included in the default substition variables from google cloud build.

- name: 'gcr.io/cloud-builders/curl'
  entrypoint: bash
  args: ['-c', 'bash <(curl -s https://codecov.io/bash)']
  env:
  - 'VCS_COMMIT_ID=$COMMIT_SHA'
  - 'VCS_BRANCH_NAME=$BRANCH_NAME'
  - 'VCS_PULL_REQUEST=$_PR_NUMBER'
  - 'CI_BUILD_ID=$BUILD_ID'
  - 'CODECOV_TOKEN=$_CODECOV_TOKEN' # _CODECOV_TOKEN is user user substituion variable specified in my cloud build trigger

This seems to work except for a warning from the bash uploader:

Step #3: /dev/fd/63: line 897: git: command not found
Step #3: /dev/fd/63: line 897: hg: command not found

So I had to use my own build image starting from the curl image and add git to it.

Dockerfile:

FROM gcr.io/cloud-builders/curl
RUN apt-get update && \
    apt-get upgrade -y && \
    apt-get install -y git

And build the image:

docker build -t "gcr.io/[PROJECT_ID]/builder .

So my final cloudbuild.yaml file is:

steps:
  - name: node:10.15.1
    entrypoint: npm
    args: ["install"]
  - name: node:10.15.1
    entrypoint: npm
    args: ["test", "--", "--coverage"]      
  - name: node:10.15.1
    entrypoint: npm
    args: ["run", "build:production"]
  - name: "gcr.io/$PROJECT_ID/builder"
    entrypoint: bash
    args: ['-c', 'bash <(curl -s https://codecov.io/bash)']
    env:
    - 'VCS_COMMIT_ID=$COMMIT_SHA'
    - 'VCS_BRANCH_NAME=$BRANCH_NAME'
    - 'VCS_PULL_REQUEST=$_PR_NUMBER'
    - 'CI_BUILD_ID=$BUILD_ID'
    - 'CODECOV_TOKEN=$_CODECOV_TOKEN'

Answer 2

As stated in the answers of a previous question, the commands are not executed in a shell, so operations such as pipes and redirections are not available.

In this accepted answer there's an example on how you can do it to use a redirection. Adapted to your needs I believe it should be something like this:

- name: 'gcr.io/cloud-builders/curl'
  entrypoint: bash
  args: ['-c', 'bash <(curl -s https://codecov.io/bash) -t $_CODECOV_TOKEN']

I'm not sure that you will be able to retrieve the $_CODECOV_TOKEN from there though, but it's an option you should try.

Regarding your second try, the error /dev/fd/63: option requires an argument -- t suggests me that the value in $_CODECOV_TOKEN is not being retrieved and thus it complains about the lack of a value in argument -t. Regardless, in this case it seems strange to me that /dev/fd/63 complains about that since it isn't executable.

Maybe a viable workaround would be to download the file on your repository and execute it from there. I know that this way the downloaded script wouldn't be up to date every time you deploy, but it would work.