With the boto library, can I avoid granting list permissions on a base bucket in S3?

Question

I currently have an IAM role that has a policy like so:

{
  "Version":"2008-10-17",
  "Statement": [
  {
    "Effect":"Allow",
    "Action":["s3:ListBucket"],
    "Resource":[
      "arn:aws:s3:::blah.example.com"
    ]
  },
  {
    "Effect":"Allow",
    "Action":["s3:GetObject", "s3:GetObjectAcl", "s3:ListBucket", "s3:PutObject", "s3:PutObjectAcl", "s3:DeleteObject"],
    "Resource":[
      "arn:aws:s3:::blah.example.com/prefix/"
    ]
  }
  ]
}

Boto seems to require the ListBucket permission be present on the root of the bucket to do the get_bucket call. If I remove the first hash in the Statement array, get_bucket('blah.example.com') will fail. Here's the error text:

*** S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>xxxx</RequestId><HostId>yyyy</HostId></Error>

Is there any way to restrict listing of the bucket to a certain prefix (e.g. "prefix/") while still using boto?

UPDATE

In order to get everything working, I used the following policy:

{
  "Version":"2008-10-17",
  "Statement": [
  {
    "Effect":"Allow",
    "Action":["s3:ListBucket"],
    "Resource":[
      "arn:aws:s3:::blah.example.com"
    ],
    "Condition":{
      "StringLike":{
         "s3:prefix":"prefix/*"
      }
    }
  },
  {
    "Effect":"Allow",
    "Action":["s3:GetObject", "s3:GetObjectAcl", "s3:PutObject", "s3:PutObjectAcl", "s3:DeleteObject"],
    "Resource":[
      "arn:aws:s3:::blah.example.com/prefix/*"
    ]
  }
  ]
}

You still have to use the validate=False parameter to the get_bucket method, but it allows listing within the prefix.

garnaat garnaat · Accepted Answer · 2012-07-13T21:47:01

By default boto tries to validate the existence of a bucket by doing a LIST operation on the bucket, asking for zero results. If you would prefer that it skip this validation step, just call it like this:

>>> import boto
>>> s3 = boto.connect_s3()
>>> bucket = s3.get_bucket('mybucket', validate=False)

This should skip the LIST operation.

With the boto library, can I avoid granting list permissions on a base bucket in S3?

1 Answers