How to make a lambda listen on multiple log groups of cloudwatch?

Question

I have a lambda that process logs from a cloudwatch log group. It saves log to Elasticsearch cluster. I am using serverless to configure the log group streaming to lambda: https://www.serverless.com/framework/docs/providers/aws/events/cloudwatch-log/.

However, it only supports listen on one log group. How can I make my lambda listen on multiple log groups? I'd like to use the pattern like:

functions:
  myCloudWatchLog:
    handler: myCloudWatchLog.handler
    events:
      - cloudwatchLog: '/aws/lambda/hello*'

In above example, I'd like my lambda to be triggered whenever there is log sent to the log group with the name start withs /aws/lambda/hello. In this way, it will save all logs to Elasticsearch for analysis.

I can't add a wildcard on the log group, I will get this error if I add a *

An error occurred (InvalidParameterException) when calling the PutSubscriptionFilter operation: 1 validation error detected: Value '/aws/lambda/hello*' at 'logGroupName' failed to satisfy constraint: Member must satisfy regular expression pattern: [\.\-_/#A-Za-z0-9]+

I hope I understand the question correctly. Let you try Log subscription filter. docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/… — petrch
The filter only works within a log group. I can't filter by log group name. — Joey Yi Zhao
Not sure if you can use wildcard patterns but you should be able to add multiple - cloudwatchLog: '/aws/lambda/hello' entries under events. — Noel Llevares

Jishnu Jishnu · Accepted Answer · 2020-10-28T12:24:45

you can add multiple cloudwatchLog events

  myCloudWatchLog:
    handler: myCloudWatchLog.handler
    events:
      - cloudwatchLog: '/aws/lambda/hello1'
      - cloudwatchLog: '/aws/lambda/hello2'

and if you want to give different filter patterns to each log group, you can use:

  myCloudWatchLog:
    handler: myCloudWatchLog.handler
    events:
      - cloudwatchLog: 
          logGroup: '/aws/lambda/hello1'
          filter: 'filter1'
      - cloudwatchLog: 
          logGroup: '/aws/lambda/hello2'
          filter: 'filter2'

How to make a lambda listen on multiple log groups of cloudwatch?

1 Answers