Is it Possible to Call One Step Function from another Step function?

Question

I am new to this Step Functions and AWS. Is it Possible to call one Step Function from another Step Function in AWS (EMR)? I am developing one Step function and i have to include Step Function in EMR before Stopping EMR.

Here I am having Two scenarios.

1) I have to call SFN3 from SFN2 and Stop the EMR in SFN2. 2) I have to Stop EMR after Execution of All Step funcitons.

I have included Attachment for your reference.

Kindly help me.

Many thanks.

Answer 1

You can call StartExecution from Task State (Lambda or Activity) and then wait for it to complete (or not, if you do not require it).

If you do not need to wait for it to complete (I may have misunderstood the question, but I believe this is not the case), you may just fire&forget with StartExecution.

In the opposite case, when you want to wait for nested State Machine to finish, you may be interested in Job Status Poller pattern (https://docs.aws.amazon.com/step-functions/latest/dg/job-status-poller-sample.html) or implement waiting as an Activity running on a EC2 / ECS / etc. (losing serverless approach).

There is also another approach to the problem of waiting for nested State Machine without active loop (status poller) or servers. It is described here: https://medium.com/semantive/part-1-asynchronous-actions-within-aws-step-functions-without-servers-f58e030a0e8b

Answer 2

Fire and Forget the Inner Workflow from the Main Workflow

Step Function Code for the Main Workflow

 "<StepName>":{
  "Type": "Task",
  "Next":"<NextStepName>",
  "Resource": "arn:aws:states:::states:startExecution",
  "Parameters":{  
    "Input":{
      "Comment": "Hello world!"
     },
    "StateMachineArn":"<InnerStepFunctionArn>"
   }
 },

IAM Policy for the Main Workflow Role

Start Execution Policy

StepFunctions/Write/Start Execution
Resource:Inner Step Function


{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "states:StartExecution",
            "Resource": "<InnerStepFunctionARN>"
        }
    ]
}

Wait for the Inner Workflow Execution to Finish

Step Function Code for the Main Workflow

 "<StepName>":{
  "Type": "Task",
  "Next":"<NextStepName>",
  "Resource": "arn:aws:states:::states:startExecution.sync",
  "Parameters":{  
    "Input":{
      "Comment": "Hello world!"
     },
    "StateMachineArn":"<InnerStepFunctionArn>"
   }
 },

IAM Policy for the Main Workflow Role

Start Execution Policy

StepFunctions/Write/Start Execution
Resource:Inner Step Function


{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "states:StartExecution",
            "Resource": "<InnerStepFunctionARN>"
        }
    ]
}

Event Policy

CloudWatchEvent Full Access
    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Sid": "CloudWatchEventsFullAccess",
                "Effect": "Allow",
                "Action": "events:*",
                "Resource": "*"
            },
            {
                "Sid": "IAMPassRoleForCloudWatchEvents",
                "Effect": "Allow",
                "Action": "iam:PassRole",
                "Resource": "arn:aws:iam::*:role/AWS_Events_Invoke_Targets"
            }
        ]
    }

Answer 3

Recently, AWS Step function added a functionality to call one step function from another step function.

It works like calling any other service ( AWS Glue, AWS Lambda, AWS Sagemaker , AWS Batch ). It does support both Sync and aysnc. Will need to check for callback support though.

You might also want to check permissions which is required to invoke one state function from another.

Official documentation links: https://docs.aws.amazon.com/step-functions/latest/dg/sample-start-workflow.html https://docs.aws.amazon.com/step-functions/latest/dg/connect-stepfunctions.html