Getting the response results from an asynchronous call to AWS lambda in python

Question

I have an AWS lambda function which I can call synchronously and get results back alright with below code

response = lambda_client.invoke(
        FunctionName=FUNCTION_NAME,
        InvocationType='RequestResponse',
        LogType='Tail',
        Payload=payload,
        Qualifier=$LATEST
    )

The response Payload is of type <botocore.response.StreamingBody object at 0x115fb3160> So I use below code to extract the payload which works fine.

response_body = response['Payload']
response_str = response_body.read().decode('utf-8')
response_dict = eval(response_str)

Now, I need to call my lambda asynchronously, so I change the invocation type with InvocationType='Event'

It gives me a response with payload of the same type as before, botocore.response.StreamingBody object but I am getting error with this line - response_dict = eval(response_str)

The error message says

    response_dict = eval(response_str)
  File "<string>", line 0

    ^
SyntaxError: unexpected EOF while parsing

What am I missing? If the response payload is same type as synchronous call, why is this parsing error? Any suggestion?

EDIT

For clarity, I understand that if the InvocationType='Event', then we only get the status of the invoke call, not the lambda function result. In my case though, I need both - launch the lambda async and get the result back when done. How do I do that? Is writing the result back to s3 and periodically checking that the only option?

I've answered why this doesn't work as you expected, below, but what I couldn't anticipate was what you were actually trying to accomplish by making the invocation asynchronous, since you still wanted the result returned. Please advise, and perhaps I or someone else can help you solve the original issue, if one remains. — Michael - sqlbot
There's no architected way to do what you want in Lambda. You need to design a solution. One option would be for the client invoking the initial Lambda to send a UUID along with its request, then the async Lambda does its work and writes the results somewhere (DynamoDB or S3, for example, using the client-supplied UUID as a key), and the client can then poll a second Lambda passing that same UUID in order to retrieve the results when they become available. Or the original Lambda sends a notification via SNS and your client polls SNS. — jarmod

Michael - sqlbot Michael - sqlbot · Accepted Answer · 2019-10-01T00:39:25

InvocationType='Event' means you aren't getting a response. An asynchronous Lambda invocation means you just want to invoke the function, not wait for the response. The response payload from the function is discarded by the service.

When you invoke a function asynchronously, Lambda sends the event to a queue. A separate process reads events from the queue and runs your function. When the event is added to the queue, Lambda returns a success response without additional information. (emphasis added)

https://docs.aws.amazon.com/lambda/latest/dg/invocation-async.html

Note that the queue mentioned here is a queue inside the Lambda service, not to be confused with Amazon Simple Queue Service (SQS).

Getting the response results from an asynchronous call to AWS lambda in python

1 Answers