Amazon Step Function with a Lambda that takes trigger from Kinesis

Question

So I am trying to create a simple pipeline in Amazon AWS. I want to execute a step function using data generated by a stream which triggers the first lambda of the state machine

What I want to do is following.

Input data is streamed by AWS Kinesis

This Kinesis stream is used as a trigger for a lambda1 that executes and writes to S3 Bucket.

This would trigger (using step function) a lambda2 that would read the content from the given bucket and write it to another bucket

Now I want to implement a state machine using Amazon Step Function. I have created the state machine which is quite straightforward

  {
  "Comment": "Linear step function test",
  "StartAt": "lambda1",
  "States": {
    "lambda1": {
      "Type": "Task",
      "Resource": "arn:....",
      "Next": "lambda2"
    },
    "lambda2": {
      "Type": "Task",
      "Resource": "arn:...",
       "End": true
        }  
    }
}

What I want is, that Kinesis should trigger the first Lambda and once its executed the step function would execute lambda 2. Which does not seem to happen. Step function does nothing even though my Lambda 1 is triggered from the stream and writing to S3 bucket. I have an option to manually start a new execution and pass a JSON as input, but that is not the work flow I am looking for

Cong Wang Cong Wang · Accepted Answer · 2017-09-01T20:28:16

you did wrong to kick off State machine.

you need to add another Starter Lambda function to use SDK to invoke State Machine. The process is like this:

kinesis -> starter(lambda) -> StateMachine (start Lambda 1 and Lambda 2)

The problem of using Step Function is lack of triggers. There are only 3 triggers which are CloudWatch Events, SDK or API Gateway.

Amazon Step Function with a Lambda that takes trigger from Kinesis

1 Answers