S3 - if file was deleted more than X days ago, delete all previous versions?

2
votes

Using Wasabi S3 compatible storage with a versioned bucket with unlimited retention. Unfortunately although Wasabi is S3 compatible, you can't set lifecycle rules on versioned buckets like you can with Amazon.

So what I want to do is to write a script to cycle through my bucket (recursively) and if the delete marker is more than X days old, I want to delete the delete marker plus all previous versions of that file. If there is no delete marker I don't want to touch the file at all, and if the delete marker is newer than X days old I don't want to touch it.

Any suggestions or help would be appreciated - I could not find much on this other than using the bucket lifecycle rules (unfortunately not an option on Wasabi)

1
amazon-s3wasabi-hot-cloud-storage
docs.aws.amazon.com/AmazonS3/latest/user-guide/… lifecycle policies might be what you are looking for my dude – nrmad
Wasabi doesn't support lifecycle rules yet (even though it's S3 compatible) so I'm trying to figure out how to do this without them. – Justin Balletta

1 Answers

0
votes

This resource provides information on a retention period after creation with an auto delete option.It seems you are incorrect about it being unsupported by Wasabi.

You can set the compliance policy on any bucket controlling all the objects that are stored in that bucket. Specifying the bucket compliance policy with the following XML tags.

RetentionDays: An integer for the minimum number of days that objects are always retained after their creation date or release from conditional hold. You can extend this retention date for any individual object, but may not shorten the date. This parameter is always required.

DeletAfterRetention: A Boolean value indicating if the object should be deleted automatically at the end of the retention period. The default is to not delete objects after the retention period. Note that this setting may be changed even after the settings are locked.

The compliance settings for a bucket are specified using the “?compliance” query string along with the compliance settings as the XML body in the request. For example:

PUT http://s3.wasabisys.com/my-bucket?complianceHTTP./1.1
<BucketComplianceConfiguration>
<Status>enabled</Status>
<LockTime>off</LockTime>
<RetentionDays>365</RetentionDays>
<DeleteAfterRetention>true</DeleteAfterRetention>
</BucketComplianceConfiguration>

After compliance is enabled for a bucket, the policy is immediately applied to all objects in the bucket.

These are the docs which I found these excerpts from: https://wasabi.com/help/docs/

A step by step guide: https://wasabi.com/wp-content/themes/wasabi/docs/User_Guide/topics/Enabling_Compliance.htm

Hope this helps friend.