AWS Lambda: can one Lambda function have Event Sources of both Kinesis and DynamoDB Streams?

Question

Can one AWS Lambda function have two Event Sources both of one Kinesis stream and one DynamoDB Stream?

I've looked but I haven't found any documentation which says that I can or cannot have different types of event sources for the same AWS Lambda function.

I can't think of a reason why this wouldn't work, assuming the function itself understands any differences in the incoming messages from different sources. Do you get an error? — Michael - sqlbot

Alex Q Alex Q · Accepted Answer · 2018-02-12T01:37:27

Yes a lambda functions can have event sources of different types but you must use the com.amazonaws.services.lambda.runtime.RequestStreamHandler to be able to correctly deserialize the input data. This is because the internal lambda code that invokes com.amazonaws.services.lambda.runtime.RequestHandler won't deserialize the data dynamically based on types then invoke an overloaded method with correct type, but instead seems to reflectively pick a method and invoke it.

Sample inputs:

Kinesis Event input:

{
    "Records": [
        {
            "kinesis": {
                "kinesisSchemaVersion": "1.0",
                "partitionKey": "1",
                "sequenceNumber": "11111111111111111111111111111111111111111111111111111111",
                "data": "e30=",
                "approximateArrivalTimestamp": 1518397399.55
            },
            "eventSource": "aws:kinesis",
            "eventVersion": "1.0",
            "eventID": "shardId-000000000000:11111111111111111111111111111111111111111111111111111111",
            "eventName": "aws:kinesis:record",
            "invokeIdentityArn": "arn:aws:iam::111111111111:role/lambda_test-lambda-multipe-sources",
            "awsRegion": "us-east-1",
            "eventSourceARN": "arn:aws:kinesis:us-east-1:111111111111:stream/test-lambda-multipe-sources"
        }
    ]
}

DynamoDb Stream Record:

{
    "Records": [
        {
            "eventID": "11111111111111111111111111111111",
            "eventName": "INSERT",
            "eventVersion": "1.1",
            "eventSource": "aws:dynamodb",
            "awsRegion": "us-east-1",
            "dynamodb": {
                "ApproximateCreationDateTime": 1518397440,
                "Keys": {
                    "key": {
                        "S": "asdf"
                    }
                },
                "NewImage": {
                    "key": {
                        "S": "asdf"
                    }
                },
                "SequenceNumber": "111111111111111111111111",
                "SizeBytes": 14,
                "StreamViewType": "NEW_AND_OLD_IMAGES"
            },
            "eventSourceARN": "arn:aws:dynamodb:us-east-1:111111111111:table/test-lambda-multipe-sources/stream/2018-02-11T18:57:44.017"
        }
    ]
}

Sample code:

public final class MultipleEventSourcesRequestHandler
        implements RequestHandler<KinesisEvent, Void>
//        implements RequestStreamHandler
{
    private static final Logger LOG = LoggerFactory.getLogger(MultipleEventSourcesRequestHandler.class);

    public Void handleRequest(final DynamodbEvent input, final Context context)
    {
        LOG.info("In DynamodbEvent handler with event of source: " + input.getRecords().get(0).getEventSource());
        return null;
    }

//    public final void handleRequest(final InputStream input, final OutputStream output, final Context context)
//            throws IOException
//    {
//        final byte[] serializedSpeechletRequest = IOUtils.toByteArray(input);
//        LOG.info("In Stream handler. Request bytes: " + new String(serializedSpeechletRequest, StandardCharsets.UTF_8));
//        output.close();
//    }

    public Void handleRequest(final KinesisEvent input, final Context context)
    {
        LOG.info("In KinesisEvent handler with event of source: " + input.getRecords().get(0).getEventSource());
        return null;
    }
}

Sample logs:

2018-02-12 01:32:57 INFO (main) com.example.lambda.eventsourcetest.MultipleEventSourcesRequestHandler - In KinesisEvent handler with event of source: aws:dynamodb

AWS Lambda: can one Lambda function have Event Sources of both Kinesis and DynamoDB Streams?

1 Answers