how to connect a cloudwatch alarm to a lambda function

Question

How do you connect an aws cloud watch alarm to a lambda function invocation?

I am programmatically adding a cloud watch alarm to the ELBs that we create as part of a cloud formation stack via AWS CloudFormation Templates. I want to have the alerts sent to a lambda function that will post the message to Slack. Although the alert works, and the SNS config seems correct to me, the lambda function is never invoked.

The lambda function follows these examples:

https://medium.com/cohealo-engineering/how-set-up-a-slack-channel-to-be-an-aws-sns-subscriber-63b4d57ad3ea#.x2j9apedu

http://inopinatus.org/2015/07/13/hook-aws-notifications-into-slack-with-a-lambda-function/

The lambda function works, and I can send it test data via the aws console resulting in a message posted to Slack.

The load balancer is created with a correct-looking cloud watch alarm:

The alarm appears to be configured to send alerts to the correct SNS topic:

There is an SNS subscription to that topic, with the lambda function as it's endpoint:

Alarms are triggered and messages sent to the correct topic when the alarm fires:

But the lambda function is never invoked:

However, if I manually add the SNS topic as an "event source" on the lambda function, it is invoked when the alarm fires and Slack messages are posted.

Am I misunderstanding how to connect a cloud watch alarm to a lambda function? Or is there a small detail I am missing?

If this approach cannot work, and the only way to connect a lambda function to a cloud watch alarm is to add the SNS topic as an "event source", what is the appropriate way to do that via AWS CloudFormation Templates? I don't see an obvious way to modify an existing resource such as a fixed lambda function.

Here is my CloudFormation Template:

"GenericSlackAlertSNSTopic" : {
    "Type" : "AWS::SNS::Topic",
    "Properties" : {
        "Subscription" : [ {
            "Endpoint" : "arn:aws:lambda:us-east-1:[...]:function:snsToSlack",
            "Protocol" : "lambda"
        } ]
    }
},
"ELBNoTrafficAlarm": {
    "Type": "AWS::CloudWatch::Alarm",
    "Properties": {
        "Namespace" : "AWS/ELB",
        "AlarmDescription": "Alarm for no apparent traffic on an ELB.",
        "AlarmActions": [{
            "Ref": "GenericSlackAlertSNSTopic"
        }],
        "InsufficientDataActions": [{
            "Ref": "GenericSlackAlertSNSTopic"
        }],
        "MetricName": "RequestCount",
        "Statistic": "Sum",
        "Dimensions" : [ {
            "Name" : "LoadBalancerName",
            "Value" : { "Ref" : "ElasticLoadBalancer" }
        } ],
        "Period": "60",
        "EvaluationPeriods": "3",
        "Threshold" : "10",
        "ComparisonOperator": "LessThanOrEqualToThreshold"
    }
}

Thanks!

-neil

Answer 1

AWS released (~3 days ago) a blueprint for the slack integration with AWS Cloudwatch using lambda both in python and nodejs: https://aws.amazon.com/blogs/aws/new-slack-integration-blueprints-for-aws-lambda/

Being said that, I also had the same problem as you, following the steps mentioned in the blueprint, I do not get the alarms until I manually add the SNS topic as an "event source" on the lambda function. Further investigation lead me to this question: Can't create a SNS Event source on a Lambda function using CloudFormation

And finally reading the AWS documentation: 1) http://docs.aws.amazon.com/lambda/latest/dg/intro-core-components.html

Amazon SNS maintains the event source mapping via topic subscription configuration (there is no AWS Lambda API to configure this mapping).

2) http://docs.aws.amazon.com/sns/latest/dg/sns-lambda.html

Configuring Amazon SNS with Lambda Endpoints with the AWS Management Console

Concluded that the subscription at the moment should be done through the AWS Management console

Summary: at the moment the only way to configure Amazon SNS with Lambda Endpoints is through the AWS Management Console

Bonus: similar question with the same answer: AWS Lambda scheduled event source via cloudformation

Answer 2

CloudWatch Scheduled event now has Lambda an a native target.

Also you can add a scheduled event to cloudformation for lambda

EventListFunction:
  Type: 'AWS::Serverless::Function'
  Properties:
    ...
    Events:
      Schedule1:
        Type: Schedule
        Properties:
          Schedule: rate(1 day)

Answer 3

Make sure you give the SNS topic permission to invoke your Lambda function. The CloudFormation for the permission will look something like this:

"LambdaInvokePermission": {
    "Type": "AWS::Lambda::Permission",
    "Properties": {
        "FunctionName" : "arn:aws:lambda:us-east-1:[...]:function:snsToSlack",
        "Action": "lambda:InvokeFunction",
        "Principal": "sns.amazonaws.com",
        "SourceArn": { "Ref": "GenericSlackAlertSNSTopic" }
    }
}

Answer 4

At writing time, in order to connect a cloudwatch alarm to lambda you need a SNS topic.

The cloudwatch alarm will have the SNS topic as alarm action and the SNS topic requires to have lambda:Invoke permission in addition to have the lambda subscribed to the specified topic.

CloudFormation Template

Provided that you have the cloudwatch alarm in a template of cloudformation and the lambda arn (either the arn string or the resource in the same template) you can connect them with the resources AWS::Lambda::Permission and AWS::SNS::Topic.

For example, with this example (add the default lambda arn value):

• aws cloudformation create-stack --stack-name MyTest --template-body file://mytest.yaml

AWSTemplateFormatVersion: '2010-09-09'

Parameters:

  LambdaArn:
    Type: String
    Description: The lambda arn, if lambda resource is in template change !Ref LambdaArn with !GetAtt LogicName.Arn
    Default: "[YOUR ARN]"

Resources:

  DummyTopic:
    Type: "AWS::SNS::Topic"
    Description: "sns topic to complete this dummy template, remove it when changing the cloudwatch alarm"
    Properties:
      TopicName: "mytest"

  CloudWatchAlarmInvocationAlarm:
    Type: "AWS::CloudWatch::Alarm"
    Description: "DummyAlarm change it (mainly Namespace, MetricName and Dimensions) to achieve your goal"
    Properties:
      Namespace: "AWS/SNS"
      AlarmDescription: "Dummy alarm"
      AlarmActions:
        - !Ref NotificationTopic
      MetricName: NumberOfMessagesPublished
      Statistic: "Sum"
      Dimensions:
        - Name: TopicName
          Value: !GetAtt DummyTopic.TopicName
      Period: 60
      EvaluationPeriods: 1
      Threshold: 1
      ComparisonOperator: "GreaterThanOrEqualToThreshold"
      TreatMissingData: notBreaching   # missing data points will not trigger the alarm => maintain

  NotificationTopic:
    Type: "AWS::SNS::Topic"
    Description: "Sns topic that communicates directly with the lambda"
    Properties:
      TopicName: "MyNotificationTopic"
      Subscription:
        - Endpoint: !Ref LambdaArn
          Protocol: "lambda"

  LambdaInvokePermission:
    Type: "AWS::Lambda::Permission"
    Properties:
      FunctionName: !Ref LambdaArn
      Action: "lambda:InvokeFunction"
      Principal: "sns.amazonaws.com"
      SourceArn:  !Ref NotificationTopic

Answer 5

This issue is not specific to Cloudwatch, but is a SNS-topic/Lambda permissions thing.

See also:

AWS SNS ought to trigger my lambda, but does not