S3 Lifecycle rule does not remove expired object delete markers

3
votes

I have an S3 bucket which will not remove expired object delete markers.

The bucket is:

  • versioning enabled
  • has a cross-region replication rule, so everything is backed up to another bucket in another region

The Lifecycle Rule I have set (attached picture below):

  • Applies to entire bucket
  • No transitions
  • Applies only to previous versions
  • Permanently delete previous versions set to 1 day
  • Clean up expired object delete markers set to true

I went through and deleted (using the UI EDIT: and the lifecycle rule) a bunch of objects, and I was able to verify that the Life Cycle rule did permanently remove the original objects after the interval of 1 day, but their delete markers (which are now technically expired object delete markers) have remained. It has been 5 days and the delete markers still remain. Is there anything further I need to do in order for the delete markers to be permanently removed as well?

Lifecycle rule expiration

1
amazon-s3
When you "deleted (using the UI) a bunch of objects", did you use Hidden=Show and delete the specific versions of the objects, or did you just delete the objects? If you simply deleted the objects (not the versions), then it would have kept the object and added the delete marker. The versioned object is still there. - John Rotenstein
@JohnRotenstein sorry, I should have clarified, the deletions were made both through the UI and via this lifecycle rule. Mostly the lifecycle rule, in reality. I've combed through the data, using the hidden=show tab, and it's definitely only the delete markers left. - NateH06

1 Answers

6
votes

It turns out that the answer to this is: patience. The lifecycle rule DID eventually clean up the expired object delete markers.

I had deleted ~100TB worth of data, so it generated quite a bit of expired object delete markers - so it took AWS > 1 week to kick in and remove them. After receiving customer support - they mentioned some things that were helpful in understanding this, emphasis mine:

it should be noted that Lifecycle can take some time to actually expire the object or delete marker even though it marks them as eligible and puts them in queue. So, although S3 stops charging once the objects are marked for expiration, they may continue to exist till Lifecycle gets to actually expire them. Additionally, this duration is also affected by the overall load on Lifecycle service for that particular AWS region.

That being said, please note that S3 Lifecycle will eventually expire/transition the eligible objects, however it doesn't have a fixed SLA to be relied upon. In most of the use cases, this is normal and acceptable. Whereas for use case that needs timely clean up of delete markers, it's better to have some other process that synchronously deletes these delete markers rather than waiting for Lifecycle to eventually clean them.