IAM Group Policy for S3 bucket: deny folders but not files

Question

this question references the same aws blog I've been trying to leverage in order to meet my needs, but without any success. I've been hacking on the solution in that answer AND the example policies it references all day without any success. I'm basically looking to allow different IAM groups full access to a common subfolder object within a bucket while selectively allowing/denying access to OTHER subfolders object based on group membership.

Bucket structure:
my-finance-bucket\
my-finance-bucket\files
my-finance-bucket\Our-Finance-Team\
my-finance-bucket\Our-Finance-Team\files
my-finance-bucket\Our-Finance-Team\Reports\
my-finance-bucket\Our-Finance-Team\Reports\files
my-finance-bucket\Our-Finance-Team\Reports\report1-name\
my-finance-bucket\Our-Finance-Team\Reports\report1-name\files
my-finance-bucket\Our-Finance-Team\Reports\report2-name\
my-finance-bucket\Our-Finance-Team\Reports\report2-name\files
my-finance-bucket\Our-Finance-Team\Data\
my-finance-bucket\Our-Finance-Team\Data\files
my-finance-bucket\Our-Finance-Team\Stuff\
my-finance-bucket\Our-Finance-Team\Stuff\files

I'm not even sure that what I'm looking to do is possible, but I want to create 3 groups named: finance-all, finance-data, and finance-reports.

I want all 3 groups to have access to:
my-finance-bucket\
my-finance-bucket\files
my-finance-bucket\Our-Finance-Team\
my-finance-bucket\Our-Finance-Team\files
my-finance-bucket\Our-Finance-Team\Stuff\
my-finance-bucket\Our-Finance-Team\Stuff\files

additionally, I have these requirements:
finance-all group has s3:* allow access to EVERYTHING under my-finance-bucket
(This is easy I think, and can probably stand alone as a separate group policy allowing s3:* to arn:aws:s3:::/my-finance-bucket and arn:aws:s3:::/my-finance-bucket/*)

finance-data group has the shared access listed above plus s3:*Object* allow to:
my-finance-bucket\Our-Finance-Team\Data\
my-finance-bucket\Our-Finance-Team\Data\files
no access to anything else that isn't specified

finance-reports has the shared access listed above, plus s3:*Object* allow to:
my-finance-bucket\Our-Finance-Team\Reports\
my-finance-bucket\Our-Finance-Team\Reports\files
my-finance-bucket\Our-Finance-Team\Reports\report1-name\
my-finance-bucket\Our-Finance-Team\Reports\report1-name\files
my-finance-bucket\Our-Finance-Team\Reports\report2-name\
my-finance-bucket\Our-Finance-Team\Reports\report2-name\files
no access to anything else that isn't specified

I'm aware that I will have to start with allowing "ListAllMyBuckets" and that as a consequence the objects representing the folders and files will be visible and listable for all folders, recursively. Though sub-optimal, it's not a show stopper since viewing, getting or putting files is to be denied for users not in their appropriate groups.

I realize most folks will ask "what have you done" in attempt to help correct whatever work I've done already - however I've gone through literally dozens of iterations and nothing has even come close to what I'm looking to do. the code listed in the linked answer and the linked aws blog post have been my start points for hacking on this and can serve as a baseline

    {
 "Version":"2012-10-17",
 "Statement": [
   {
     "Sid": "AllowUserToSeeBucketListInTheConsole",
     "Action": ["s3:ListAllMyBuckets", "s3:GetBucketLocation"],
     "Effect": "Allow",
     "Resource": ["arn:aws:s3:::*"]
   },
  {
     "Sid": "AllowRootAndHomeListingOfCompanyBucket",
     "Action": ["s3:ListBucket"],
     "Effect": "Allow",
     "Resource": ["arn:aws:s3:::yourbucketname"],
     "Condition":{"StringEquals":{"s3:prefix":["","yourfoldername/"],"s3:delimiter":["/"]}}
    },
   {
     "Sid": "AllowListingOfUserFolder",
     "Action": ["s3:ListBucket"],
     "Effect": "Allow",
     "Resource": ["arn:aws:s3:::yourbucketname"],
     "Condition":{"StringLike":{"s3:prefix":["yourfoldername/*"]}}
   },
   {
     "Sid": "AllowAllS3ActionsInUserFolder",
     "Effect": "Allow",
     "Action": ["s3:GetObject"],
     "Resource": ["arn:aws:s3:::yourbucketname/yourfoldername/*"]
   },
{
      "Action": [
        "s3:*"
      ],
      "Sid": "Stmt1375581921000",
      "Resource": [
"arn:aws:s3:::yourbucketname/anotherfolder1/*",
"arn:aws:s3:::yourbucketname/anotherfolder2/*",
"arn:aws:s3:::yourbucketname/anotherfolder3/*",
"arn:aws:s3:::yourbucketname/anotherfolder4/*"
      ],
      "Effect": "Deny"
    }
 ]
}

What you're trying to do is a little hard to follow on the specifics, but it's simple in principle: anything not explicitly allowed is denied, and anything explicitly denied is always denied even if it is also explicitly allowed. Remember that deny StringNotLike is also available, and is not the precise opposite of allow StringLike, since deny has precedence. A broader allow can thus have "deny everything but" exceptions carved out of it. — Michael - sqlbot
thanks for the feedback. From my reading on the subject, it seems that the concept of s3 objects as key value pairs and the usage of the prefix and delimiter operators may also be useful. I intend to post an edit or answer my own question when I get something useful working. I'm currently playing with the policy simulator and some test buckets to get a feel for the operators that I don't really have a good handle on (StringLike, StringNotLike, Prefix, Delimiter). — Nicholas Abbruzzese

Nicholas Abbruzzese Nicholas Abbruzzese · Accepted Answer · 2016-02-08T19:23:04

I ended up spending about 3 days on this before I gave up and convinced the person with the business need for this that I could get the whole thing done in a half hour if they would let me re-organize the bucket. They agreed, so I ended up just creating "subfolders" in the bucket and then just created IAM groups granting access to each individual "subfolder".

So I reorganized the bucket like so:

my-finance-bucket/
my-finance-bucket/files
my-finance-bucket/shared/
my-finance-bucket/shared/files
my-finance-bucket/
my-finance-bucket/data/
my-finance-bucket/data/files
my-finance-bucket/reports/
my-finance-bucket/reports/files

Following this pattern, I condensed existing "subfolders" such as
my-finance-bucket/some-report/*
my-finance-bucket/some-other-report/*
down to
my-finance-bucket/reports/some-report/*
my-finance-bucket/reports/some-other-report/*

and the same for condensing things under /shared/ and /data/

This being done, I was able to rely on the inherent deny functionality and simply carve out allow access for the individual top level folders using IAM group policies. By simply adding users to those groups, I was able to selectively grant access to some subfolders and not others.

All of the policies I created for access to each individual bucket followed this format:

{
"Version": "2012-10-17",
"Statement": [
    {
        "Sid": "AllowListAllMyBuckets",
        "Effect": "Allow",
        "Action": [
            "s3:ListAllMyBuckets",
            "s3:GetBucketLocation"
        ],
        "Resource": [
            "arn:aws:s3:::*"
        ]
    },
    {
        "Sid": "AllowedListAccess",
        "Effect": "Allow",
        "Action": [
            "s3:ListBucket",
            "s3:GetBucketAcl",
            "s3:GetBucketLocation"
        ],
        "Resource": [
            "arn:aws:s3:::my-finance-bucket",
            "arn:aws:s3:::my-finance-bucket/shared"
        ]
    },
    {
        "Sid": "AllowedObjectAccess",
        "Effect": "Allow",
        "Action": [
            "s3:*Object*"
        ],
        "Resource": [
            "arn:aws:s3:::my-finance-bucket/shared/*"
        ]
    }
]

}

Note that by virtue of the /* allows for list and object access, that the subfolders are also accessible to folks with access to the top level subfolders. Should nested subfolder access have been required, I believe it would have been the same issue as the original question. BUT, I think that with the simple and straightforward re-organization of this bucket I could have made it happen with explicit denies to the subfolders, and additional groups with explicit allows to those same subfolders. I'm fairly certain this also would have worked for the original question, but the way the bucket was organized would have made it an onerous task to create and maintain the policies.

Last, it's worth noting that this method makes it impossible to deny list access to "folders" and "files". This means that while users in the shared group can see the names of the files and folders in the data and reports folders, they cannot perform any other operations (no get or put aka list access only, no read, no write)

IAM Group Policy for S3 bucket: deny folders but not files

1 Answers