HTTP Triggering Cloud Function with Cloud Scheduler

13
votes

I have a problem with a job in the Cloud Scheduler for my cloud function. I created the job with next parameters:

Target: HTTP

URL: my trigger url for cloud function

HTTP method: POST

Body: 

{
 "expertsender": {
  "apiKey": "ExprtSender API key",
  "apiAddress": "ExpertSender APIv2 address",
  "date": "YYYY-MM-DD",
  "entities": [
     {
        "entity": "Messages"
     },
     {
        "entity": "Activities",
        "types":[
           "Subscriptions"
        ]
     }
  ]
 },
 "bq": {
         "project_id": "YOUR GCP PROJECT",
         "dataset_id": "YOUR DATASET NAME",
         "location": "US"
       } 
}

The real values has been changed in this body.

When I run this job I got an error. The reason is caused by processing body from POST request.

However, when I take this body and use it as Triggering event in Testing I don't get any errors. So I think, that problem in body representation for my job but I havn't any idea how fix it. I'll be very happy for any idea.

4
pythongoogle-cloud-functionsgoogle-cloud-scheduler
What is the error? What is the body content? – Doug Stevenson
@DougStevenson The error occurred when I tried to get body data inside a cloud function (I use python). So, my function get the body but then i see in logs: 'NoneType' object is not subscriptable. It means, that function can't extract parameters in the body properly because smth wrong with the body. However, when i trigerring my function from Testing interface of the cloud function with the same body then no errors occurs. Body content I left at a question above. – Sergey Kravchenko
You might want to edit your question to show your code and point out the line where the error occurs. You should also show what you expect the body content should be. – Doug Stevenson
Can you also include the entire log message that contains " 'NoneType' object is not subscriptable"? – Dustin Ingram
@SergeyKravchenko Are you able to update the question with more details? – Dustin Ingram

4 Answers

17
votes

Disclaimer: I have tried to solve the same issue using NodeJS and I'm able to get a solution

I understand that this is an old question. But I felt like its worth to answer this question as I have spent almost 2 hours figuring out the answer for this issue.

Scenario - 1: Trigger the Cloud Function via Cloud Scheduler

  • Function fails to read the message in request body.

Scenario - 2: Trigger the Cloud Function via Test tab in Cloud Function interface

  • Function call always executes fine with no errors.

What did I find?

  • When the GCF routine is executed via Cloud Scheduler, it sends the header content-type as application/octet-stream. This makes express js unable to parse the data in request body when Cloud scheduler POSTs the data.
  • But when the exact same request body is used to test the function via the Cloud Function interface, everything works fine because the Testing feature on the interface sends the header content-type as application/json and express js is able to read the request body and parses the data as a JSON object.

Solution

I had to manually parse the request body as JSON (explicitly using if condition based on the content-type header) to get hold of data in the request body.

/**
 * Responds to any HTTP request.
 *
 * @param {!express:Request} req HTTP request context.
 * @param {!express:Response} res HTTP response context.
 */
exports.helloWorld = (req, res) => {
  let message = req.query.message || req.body.message || 'Hello World!';

  console.log('Headers from request: ' + JSON.stringify(req.headers));

  let parsedBody;

  if(req.header('content-type') === 'application/json') {
    console.log('request header content-type is application/json and auto parsing the req body as json');
    parsedBody = req.body; 
  } else {
    console.log('request header content-type is NOT application/json and MANUALLY parsing the req body as json');
    parsedBody = JSON.parse(req.body);
  }

  console.log('Message from parsed json body is:' + parsedBody.message);

  res.status(200).send(message);
};

It is truly a feature issue which Google has to address and hopefully Google fixes it soon.

Cloud Scheduler - Content Type header issue

10
votes

Another way to solve the problem is this:

request.get_json(force=True)

It forces the parser to treat the payload as json, ingoring the Mimetype. Reference to the flask documentation is here

I think this is a bit more concise then the other solutions proposed.

6
votes

Thank you @Dinesh for pointing towards the request headers as a solution! For all those who still wander and are lost, the code in python 3.7.4:

import json

raw_request_data = request.data

# Luckily it's at least UTF-8 encoded...
string_request_data = raw_request_data.decode("utf-8")
request_json: dict = json.loads(string_request_data)

Totally agree, this is sub-par from a usability perspective. Having the testing utility pass a JSON and the cloud scheduler posting an "application/octet-stream" is incredibly irresponsibly designed. You should, however, create a request handler, if you want to invoke the function in a different way:

def request_handler(request):
    # This works if the request comes in from 
    # requests.post("cloud-function-etc", json={"key":"value"})
    # or if the Cloud Function test was used
    request_json = request.get_json()
    if request_json:
        return request_json

    # That's the hard way, i.e. Google Cloud Scheduler sending its JSON payload as octet-stream
    if not request_json and request.headers.get("Content-Type") == "application/octet-stream":
        raw_request_data = request.data
        string_request_data = raw_request_data.decode("utf-8")
        request_json: dict = json.loads(string_request_data)

    if request_json:
        return request_json

    # Error code is obviously up to you
    else:
        return "500"
1
votes

One of the workarounds that you can use is to provide a header "Content-Type" set to "application/json". You can see a setup here.