How to run a Cloud Scheduler job with App Engine HTTP when protected by IAP

Question

I have a Python app on Google App Engine Standard which is secured using Google Cloud Identity Aware Proxy (IAP).

I would like to trigger a part of my app every day by using Cloud Scheduler. (It is calling an API, doing calculations, and stores results in Google Cloud Storage.)

To do so, I tried triggering a POST request on an "App Engine HTTP". example URL: "/refresh_data"

When running the job, I get the following error:

jsonPayload: {
status: "FAILED_PRECONDITION"
relativeUrl: "/refresh_data"
jobName: "..."
@type: "type.googleapis.com/google.cloud.scheduler.logging.AttemptFinished"
targetType: "APP_ENGINE_HTTP"
}

I can't find any documentation relative to the "FAILED_PRECONDITION" error so I'm feeling kind of stuck here.

As an alternative, I tried to trigger a POST request to a simple "HTTP" and by granting the Owner access level in IAP to the service account I am using for Cloud Scheduler. This time, the error message I get is the following :

 jsonPayload: {
 status: "UNAUTHENTICATED"
 @type: "type.googleapis.com/google.cloud.scheduler.logging.AttemptFinished"
 jobName: "..."
 targetType: "HTTP"
 url: "https:.../refresh_data"
 }

I really don't understand how to make this work... The Cloud Scheduler documentation barely documents the "App Engine HTTP" option, and nothing is documented relative to te use of IAP...

Any help would be much appreciated :)

Answer 1

It's the dark side of IAP. I sent this feedback to Google months ago. Same thing with Pubsub, you can't trigger and App Engine IAP protected from these serverless product, even if you use a service account with the correct authorization.

Answer 2

You can check this relevant documentation from IAP on how to authenticate with a Service Account.

Whenever you are using Cloud Scheduler, the requests will be done with its Service Account, therefore the guide to follow should be the one linked above as you are basically trying to authenticate programmatically rather than with the Google sign-in.

That being said, you would need to generate the OIDC token prior to making a request to an IAP-secured endpoint. Once you have the OIDC token, it needs to be included in the Authorization: Bearer header.

Answer 3

I had a similar problem when upgrading my GAE app from python 2.7 to python3 (standard). I got the same error message as you (status: "FAILED_PRECONDITION") when running my previous cron.yaml jobs set up as cloud scheduler jobs. And the upload of previous cron.yaml files did not work to run either. Then I found out that just adding an ending '/' on the url fixed it. So a cron like:

cron:
- description: competition participants pilot list update
  url: /daily1/
  schedule: every 8 hours from 05:00 to 21:00

worked after being uploaded with cloud SDK:

gcloud app deploy cron.yaml --project my-gae-project

(and I forgot that) I also had to:

gcloud services enable cloudscheduler.googleapis.com --project my-gae-project