Google managed Cloud run container fails to start on deploy from CLI but the same image works when manually deploying via dashboard

Question

So I have this issue, I have a (currently) only local devops process which is just a series of commands in bash building a docker container for a nodejs application and uploading to google container registry and then deploying it to google cloud run from there.

The issue I'm having is the deployment step always fails throwing:

ERROR: (gcloud.beta.run.services.replace) Cloud Run error: Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable. and there's nothing in the logs when I follow the link or manually try to access the log for that service in cloud run.

At some point I had a code issue which was preventing the container from starting and I could see that error in the cloud run logs.

I'm using the following command & yaml to deploy:

gcloud beta run services replace .gcp/cloud_run/auth.yaml

and my yaml file:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: auth-service
spec:
  template:
    spec:
      containers:
        - image: gcr.io/my_project_id/auth-service
      serviceAccountName: abc@my_project_id.iam.gserviceaccount.com

EDIT:

I have since pulled the yaml file configuration for the service that I manually deployed, and it looks something like this:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  annotations:
    client.knative.dev/user-image: gcr.io/my_project_id/auth-service
    run.googleapis.com/ingress: all
    run.googleapis.com/ingress-status: all
    run.googleapis.com/launch-stage: BETA
  labels:
    cloud.googleapis.com/location: europe-west2
  name: auth-service
  namespace: "1032997338375"
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/maxScale: "2"
        run.googleapis.com/client-name: cloud-console
        run.googleapis.com/sandbox: gvisor
      name: auth-service-00002-nux
    spec:
      containerConcurrency: 80
      containers:
        - image: gcr.io/my_project_id/auth-service
          ports:
            - containerPort: 3000
          resources:
            limits:
              cpu: 1000m
              memory: 512Mi
      serviceAccountName: abc@my_project_id.iam.gserviceaccount.com
      timeoutSeconds: 300
  traffic:
    - latestRevision: true
      percent: 100

I've changed the name to the service I'm trying to deploy from the command line and deployed it as a new service just like before, and it worked right away without further modifications.

Although I'm not sure which of the configurations I'm missing in my initial file as the documentation on the YAML for cloud run deployments doesn't specify a minimum configuration.

Any ideas which configs I can keep & which can be filtered out?

Are you using Cloud Run managed or Knative on GKE? What are the other flags you're passing to the gcloud command? Why not use gcloud run deploy - no YAML needed? — sethvargo
@sethvargo I thought long-term would be more useful to have all the configuration inside the yaml rather than commands. I'm running 2 commands to set platform as 'managed' and region as 'europe-west2' but that's pretty much it. I'll give it a go with just the run deploy command. — SebastianG
@sethvargo I've also edited my initial post to add a few more details. — SebastianG

Jan Hernandez Jan Hernandez · Accepted Answer · 2021-03-09T23:35:10

If you check both yaml files, you can find the property containerPort in the file generated by the console

By default cloud run performs a healtcheck test and expects listen something in the port 8080 or in this example the dockerfile will run over the port that Docker/Cloud Run sent to the container

In your case you are running a container that runs over the port 3000, if you don't declare the port, cloud run can't run your image because is not detecting anything on 8080

You can define the yaml as this:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: auth-service
spec:
  template:
    spec:
      containers:
        - image: gcr.io/myproject/myimage:latest
          ports:
            - containerPort: 3000
      serviceAccountName: abc@my_project_id.iam.gserviceaccount.com

Google managed Cloud run container fails to start on deploy from CLI but the same image works when manually deploying via dashboard

1 Answers