Adding a Lambda function into Kinesis Firehose via Cloudformation Template

3
votes

I want to add a Lambda function to my Kinesis Firehose to transform the source data as described here.

Looking at the Cloudformation Template for Kinesis firehose, I don't see an option for that,

Type: "AWS::KinesisFirehose::DeliveryStream"
Properties: 
  DeliveryStreamName: String
  DeliveryStreamType: String
  ElasticsearchDestinationConfiguration:
    ElasticsearchDestinationConfiguration
  ExtendedS3DestinationConfiguration:
    ExtendedS3DestinationConfiguration
  KinesisStreamSourceConfiguration:
    KinesisStreamSourceConfiguration
  RedshiftDestinationConfiguration:
    RedshiftDestinationConfiguration
  S3DestinationConfiguration:
    S3DestinationConfiguration

How do I setup the CF template to have a Lambda function process the input for Kinesis Firehose?

1
amazon-web-servicesamazon-cloudformationamazon-kinesis-firehose

1 Answers

2
votes

It's possible to achieve this via ProcessingConfiguration which is available for ES, S3 and Redshift destination configs. Should look something like the following:

"ElasticsearchDestinationConfiguration": {
...
   "ProcessingConfiguration": {
         "Enabled": "true",
         "Processors": [
           {
             "Parameters": [
               {
                 "ParameterName": "LambdaArn",
                 "ParameterValue": "arn:aws:lambda:eu-west-1:123456789:function:cw-transformer:$LATEST"
               }
             ],
             "Type": "Lambda"
           }
         ]
       }
...
}