AWS S3 Object Expiry Issue - Prefix

Question

I want to use the AWS S3 Object Expiry and have a bucket set up & a lifecycle rule.

The rule is shown below:

The issue I'm having I believe is to do with the Prefix.

I created this structure under the bucket "online-dating/dating-cropping/" and started to put files into this location that I wanted removed - tmp files. The idea was these files would be removed automatically.

When I checked I found the structure "online-dating/dating-cropping/" had been removed. Eg: it didn't clean out the files under here but removed it entirely.

Therefore am I right in assuming you can't use the Expiry Rule like this (pointing to a location under a bucket to cleanup? Does the Prefix have to reflect the actual file(s) you want removed - eg: remove apples.jpg - you could then set the rule to app* for example?

Therefore I assume it doesn't matter where in the bucket the files actually reside. Just the prefix must match the files?

Answer 1

While many S3 browsers (including the AWS management console) will show paths with "/" as directories, this is not what they are. Removing all files with the specified prefix will mean that the browsers no longer have the ability to detect the presence of the "folder" of that name.

Whatever process you are using that creates the temporary files should not have any issue writing more files and as soon as it does write a new temp file the "folder" will reappear in an S3 browser.