Can't get AWS Lambda function to log (text output) to CloudWatch

Question

I'm trying to set up a Lambda function that will process a file when it's uploaded to an S3 bucket. I need a way to see the output of console.log when I upload a file, but I can't figure out how to link my Lambda function to CloudWatch.

I figured about by looking at the context object that my log group is /aws/lambda/wavToMp3 and the log stream is 2016/05/23/[$LATEST]hex_code_redacted. So I created that group and stream in CloudWatch, yet nothing is being logged to it.

Answer 1

For the lambda function to create log stream and publish logs to cloudwatch, the lambda execution role needs to have the following permissions.

{
    "Statement": [
        {
            "Action": [
                "logs:CreateLogGroup",
                 "logs:CreateLogStream",
                 "logs:PutLogEvents"
            ],
            "Effect": "Allow",
            "Resource": "arn:aws:logs:*:*:*"
        }
    ]
} 

Please refer to the following AWS documentation for more details http://docs.aws.amazon.com/lambda/latest/dg/intro-permission-model.html#lambda-intro-execution-role

Answer 2

After you update your policy, it seems that you have to update your function's settings to refresh all job instances to read new policies.

So if you just click 'test' button from Lambda console after you update your role policy in IAM, the cached Lambda instances will still have old role permissions, so you will still see no logs being written to Cloudwatch logs.

Just change your timeout by a second and click on 'save and test' button, and you will start to see logs in Cloudwatch.

Answer 3

For the lambda function to create log stream and publish logs to cloudwatch, the lambda execution role needs to have the following permissions

I already had these permissions yet it did not work.

Just change your timeout by a second and click on 'save and test' button, and you will start to see logs in Cloudwatch.

I changed the timeout, saved and logs still did not work.

I assigned another role and logs still did not work.

What ended up working for me was clicking "Create a custom role", then "Allow". This was it and logs started being generated but since I did not want to use a new role but my existing role, I simply assigned my existing role afterwards and it worked. So technically I should have returned back to original configuration that did not work but now it works. Go figure.

Answer 4

Apparently another necessity for logging to happen is the Lambda function must indicate completion; for instance in the Python context, the handler must return something other than None.

Answer 5

July 2020 Update !!

Logs may not be in us-east-1, try looking for lambda edge logs in different regions !!

Answer 6

Make sure you have the full path of your "Existing role" in your lambda function "Configuration":

Role: Choose an existing role Existing Role: service-role/yourRoleName

For some reason, typing only yourRoleName will work for some services (like SES) but not for CloudWatch.

Also, you may try creating a new role instead of using an existing one. This will create the role with the proper configuration (hopefully).

Answer 7

For the issue was I was trying to create a log group in the Cloudformation script by : AWS::Logs::LogGroup and then trying to push the Lambda log to this log group. :P Novice After careful reading , i found that Lambda creates its own log with the aforementioned format: /aws/lambda/ We just need to provide policy permission to this log group , or just a generic permission with resource as: arn:aws:logs:::*

hope this helps

Answer 8

Maybe a bit late, but for those who still struggle with seeing the lambda logs in cloudwatch. I noticed this regarding the lambda function's execution role: "You may use an existing role with this function. Note that the role must be assumable by Lambda and must have Cloudwatch Logs permissions." So in IAM i granted " CloudWatchLogsFullAccess" to the role i assigned to my function. then in cloudwatch, under logs, you'll see the logs for the functions assigned this role.

Answer 9

As other answers state you need to give lambda permission to post logs to cloud watch logs. AWS had provided AWSLambdaExecute policy just for that. It's json is -

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "logs:*"
            ],
            "Resource": "arn:aws:logs:*:*:*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::*"
        }
    ]
}

You can add this policy in your role which is assigned to your lambda and you should start seeing the logs.

NOTE: It also has S3 read/write access. If you do not want it you can create a custom policy with just the logs part.

Answer 10

It might already log, we just couldn't find the logs we expect...

e.g.

app.use(function simpleLogger (req, res, next) {
  console.info('[Logger]', req.method, req.originalUrl)
  next()
})

After performing GET /hello?world=1,

Local console: (simple and clear, nice!)

[Logger] GET /hello?world=1

CloudWatch Logs: (can you easily find the exact log below?)

START RequestId: a3552c34-f7a6-11e8-90ba-2fb886f31fb0 Version: $LATEST
2018-12-04T09:26:11.236Z  a3552c34-f7a6-11e8-90ba-2fb886f31fb0  [Logger] GET /hello?world=1
END RequestId: a3552c34-f7a6-11e8-90ba-2fb886f31fb0
REPORT RequestId: a3552c34-f7a6-11e8-90ba-2fb886f31fb0  Duration: 41.02 ms  Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 29 MB

Conclusion: too verbose to locate the original logs.

Answer 11

I encountered this problem but none of the answers above solved my issue. It turns out that the region was somehow set to Ohio when I first started CloudWatch. After I changed it to US East (N. Virginia), everything works fine.

Answer 12

CloudWatch & CloudWatch Logs are different Permissions, you need add CloudWatch Logs to the policy which attached with your role.

Answer 13

There's a writeup called How to Monitor AWS Lambda with CloudWatch with a section on "How to Use CloudWatch Logs with Lambda". Looks like you already found your answer, but for anybody without the IAM specific issues, this may help.

Answer 14

You might want add the Last Ingestion Time column to the log output. It's taking a few minutes for all my events to be written.

Answer 15

Though this got answered already, Just wanted to add my experience which might be useful for others.

Even the permissions are set appropriately so that lambda can log in cloudwatch, sometimes it is taking 3-4 hrs to reflect the log groups.

In my case when using lambda for DynamoDB events, where lambda was given all needed permissions for both cloudwatch and DynamoDB. It took 4 hrs to get the logs reflected. May be some sync issues from AWS end. After 4 hrs without any action from my end, I was able to see the Cloudwatch logs.

Also make sure that you are seacrching the logs in the same region where your function is created.