AWS Cloudformation to create an AWS Cloudwatch Event to trigger AWS Batch

Question

I have been reading the documentation for AWS Cloudwatch events to trigger AWS Batch and I cannot figure out how to trigger a aws batch from cloudwatch events:

In the aws cli I can successfully execute the aws batch job with this bash code:

BATCH_JOB_QUEUE_NAME="test-batch-job-queue"
BATCH_JOB_DEFINITION_NAME="test-batch-job-def"

BATCH_JOB_DEFINITION_ARN=$( aws batch describe-job-definitions \
  --job-definition-name ${BATCH_JOB_DEFINITION_NAME} \
  --status ACTIVE \
  | jq -r '.jobDefinitions | max_by(.revision).jobDefinitionArn' \
) && echo ${BATCH_JOB_DEFINITION_ARN}

echo "Submitting job with additional ${SIZE_OF_RAM}GB RAM\n"
aws batch submit-job \
    --job-name ${BATCH_JOB_NAME} \
    --job-queue `aws batch describe-job-queues --job-queues $BATCH_JOB_QUEUE_NAME | jq ".jobQueues[].jobQueueArn" -r` \
    --job-definition $BATCH_JOB_DEFINITION_ARN \
    --parameters configFile="s3://${BUCKET_NAME}/${PROJECT_NAME}/config.json" \   <=== *****important configuration
    --container-overrides vcpus=16,memory=16000 \ .               <=== *****important configuration
    --profile ${PROJECT_NAME}-${environment}

But with AWS Cloudwatch Events I get an error in my cloudformation yaml

Resources:
  TestBatchSchedule:
    Type: AWS::Events::Rule
    Properties:
      Description: Test bi-monthly schedule
      Name: TestBiMonthlySchedule
      ScheduleExpression: cron(0 7 * * 2) # every two weeks at 7 am
      State: 'ENABLED'
      Targets: 
        - Arn: test-arn
          BatchParameters: 
            JobDefinition: 
              Fn::ImportValue: !Sub "${Environment}-test-batch-def"
            JobName: "test-batch-scheduled-job"
            RetryStrategy: 
              Attempts: 1
          Id: test-id
          InputTransformer: 
            InputPathsMap:
              parameters:
                configFile: s3://test-batch-bucket/test-project/config.json
              containerProperties:
                vcpus: 16
                memory: 32000
            InputTemplate: "s3://test-batch-bucket/test-project/config.json"

Returns this error:

Value of property InputPathsMap must be an object with String (or simple type) properties

My specific question is how do I add a parameters and container overrides section to the cloudformation AWS::Events::Rule? I would like to keep it in yaml also.

Beney Kim Beney Kim · Accepted Answer · 2020-02-12T08:06:58

According to https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-events-rule-inputtransformer.html#cfn-events-rule-inputtransformer-inputpathsmap

InputPathsMap is an array key-value pairs, where each value is a valid JSON path. You can have as many as 10 key-value pairs. You must use JSON dot notation, not bracket notation.

I've not checked, but could you try it as following?

InputPathsMap:
   parameters.configFile: s3://test-batch-bucket/test-project/config.json
   parameters.containerProperties.vcpus: 16
   parameters.containerProperties.memory: 32000

AWS Cloudformation to create an AWS Cloudwatch Event to trigger AWS Batch

1 Answers