How to write a policy in .yaml for a python lambda to read from S3 using the aws sam cli

Question

I am trying to deploy a python lambda to aws. This lambda just reads files from s3 buckets when given a bucket name and file path. It works correctly on the local machine if I run the following command:

sam build && sam local invoke --event testfile.json GetFileFromBucketFunction

The data from the file is printed to the console. Next, if I run the following command the lambda is packaged and send to my-bucket.

sam build && sam package --s3-bucket my-bucket --template-file .aws-sam\build\template.yaml --output-template-file packaged.yaml

The next step is to deploy in prod so I try the following command:

sam deploy --template-file packaged.yaml --stack-name getfilefrombucket --capabilities CAPABILITY_IAM --region my-region

The lambda can now be seen in the lambda console, I can run it but no contents are returned, if I change the service role manually to one which allows s3 get/put then the lambda works. However this undermines the whole point of using the aws sam cli.

I think I need to add a policy to the template.yaml file. This link here seems to say that I should add a policy such as one shown here. So, I added:

Policies: S3CrudPolicy

Under 'Resources:GetFileFromBucketFunction:Properties:', I then rebuild the app and re-deploy and the deployment fails with the following errors in cloudformation:

1 validation error detected: Value 'S3CrudPolicy' at 'policyArn' failed to satisfy constraint: Member must have length greater than or equal to 20 (Service: AmazonIdentityManagement; Status Code: 400; Error Code: ValidationError; Request ID: unique number

and

The following resource(s) failed to create: [GetFileFromBucketFunctionRole]. . Rollback requested by user.

I delete the stack to start again. My thoughts were that 'S3CrudPolicy' is not an off the shelf policy that I can just use but something I would have to define myself in the template.yaml file?

I'm not sure how to do this and the docs don't seem to show any very simple use case examples (from what I can see), if anyone knows how to do this could you post a solution?

I tried the following:

S3CrudPolicy:
  PolicyDocument:
    -
      Action: "s3:GetObject"
      Effect:  Allow
      Resource: !Sub arn:aws:s3:::${cloudtrailBucket}
      Principal:  "*"

But it failed with the following error:

Failed to create the changeset: Waiter ChangeSetCreateComplete failed: Waiter encountered a terminal failure state Status: FAILED. Reason: Invalid template property or properties [S3CrudPolicy]

If anyone can help write a simple policy to read/write from s3 than that would be amazing? I'll need to write another one so get lambdas to invoke others lambdas as well so a solution here (I imagine something similar?) would be great? - Or a decent, easy to use guide of how to write these policy statements?

Many thanks for your help!

user3062260 user3062260 · Accepted Answer · 2019-07-09T15:41:40

Found it!! In case anyone else struggles with this you need to add the following few lines to Resources:YourFunction:Properties in the template.yaml file:

  Policies: 
    - S3CrudPolicy:
        BucketName: "*"

The "*" will allow your lambda to talk to any bucket, you could switch for something specific if required. If you leave out 'BucketName' then it doesn't work and returns an error in CloudFormation syaing that S3CrudPolicy is invalid.

How to write a policy in .yaml for a python lambda to read from S3 using the aws sam cli

1 Answers