Lambda AWS instant return logs

Question

I would like to have direct fast access to logs in AWS lambda.

I can query CloudWatch but It requires time to GET data.

Is it possible to get all logs of execution in Lambda and return this data in response?

---

UPDATE 15.06.2020

Thank you @John for answer, here I provide more information:

I am using dedicated logger for lambda in .NET

https://github.com/aws/aws-lambda-dotnet/tree/master/Libraries/src/Amazon.Lambda.Logging.AspNetCore

My app needs logging to debug if any problem occurs.

For example, what type of "logs" are you wanting to access

I log just some data in lambda function on different logging levels like:

1. logger.LogDebug("Lambda started");
2. logger.LogCritical(exceptions);
3. logger.LogInfo(businessDataForDebug);

I would like to get all logged statements, which where passed by logging level and return all those information in lambda response as fast as possible. Logs data must refer to particular lambda execution. Otherwise, it would be useless.

What type of data are you seeking from CloudWatch?

I need mostly logged string messages and exceptions if occur. If it is possible It could be nice to include time of particular log invocation. Moreover some "log" Id would be nice to have as reference - there is something like RequestId.

For now I can get all those logs doing query in CloudWatch but it takes time and it's too slow. I do API calls: StartQueryAsync and then GetQueryResultsAsync. Solution should be as fast as possible. I could make ugly custom logger where I can collect all logs to ex. List myLogs and simply return in response as JSON but I hope there are nice AWS solution to this purpose.

What do you mean by "logs of execution"?

I mean all logs generated by logger in Lambda simple function runtime.

LambdaFunction(request, context){
InitializeLoggerWithLogLevel(debug);
logger.LogInfo("Log Info"); //Not logged due to min log level Debug
logger.LogTrace("Log Trace"); //Logged, trace > debug
logger.LogDebug("Log debug"); //Logged, debug == debug

Are you referring to the output of AWS Lambda functions that are stored in Amazon CloudWatch Logs?

Yes, I can query logs from there but it takes a few seconds to get it. I would have faster fetching data.

I can believe that AWS doen't provide such solution. Workaround could be to delegate getting logs to another lambda by ex. SQS or move logs of executed particular lambda function to S3 bucket and then give user link to download. My first main lambda must be fast, processing many requests like machine gun ;).

Please advice me the best approach. I am open for further discussion.

Regards,

Answer 1

If you're hitting 50 transactions per second the CloudWatch transactions get throttled. You can ask for an increase on this limit: https://console.aws.amazon.com/support/home#/case/create?issueType=service-limit-increase&limitType=service-code-cloudwatch-logs

Ref: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/cloudwatch_limits_cwl.html

If it is possible It could be nice to include time of particular log invocation. Moreover some "log" Id would be nice to have as reference - there is something like RequestId. For now I can get all those logs doing query in CloudWatch but it takes time and it's too slow.

If you're after a FAST solution and you really are pushing the CloudWatch limits use DynamoDB.

Answer 2

You can stream the CloudWatch logs to yourself instead of getting them.

You can subscribe a Lambda function to CloudWatch Log Events. This way you can process it in real-time.

Using this approach, you can stream it to an ElasticSearch instance for easy querying and processing.

You can also have a Lambda write those events to an S3 bucket.

Reference: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Subscriptions.html

Answer 3

It may be possible with new aws lambda extension and logs API The idea is to make the extension request the logs api to get the logs than send the logs to the lambda function using ipc (didn't manage to make it work) or simply using the /tmp folder to write the logs via the extension and then read it from the function.

I tried to do an POC that return logs for a side project, but didn't manage to make it work. The extension has it's own life cycle and usually receives logs after the lambda shutdown.

Sometimes I received the logs when I run the functions multiple times, maybe the logs are from a previous execution.

A solution that I think would work is to store the logs into an s3 bucket and trigger another lambda via s3 upload event that will return the logs. Depending on the requirements this may be a solid solution compared to just subscribing to the cloud watch which will take more time.