Sync AWS glacier storage changes with RDS

Question

I am using S3 Lifecycle Rule to move objects to Glacier. Since objects will be moved to glacier storage I need to make sure my application RDS is also updated with similar details.

As per my discussion over this thread AWS Lambda for objects moved to glacier, there is no way currently to generate SQS notification to get notified about object being moved to glacier.

Also, as per my understanding currently Lifecycle rule will be evaluated once in a day, but there is not specific time when this will happen in a day. If there was i was planning to have a scheduler which will run after that and update status of archived objects in RDS.

Is there a way that you can suggest which will be close enough to sync this status changes between AWS & RDS?

Let me know your feedback or if you need more information on this to understand use case.

=== My Current approach is as per below.

Below is exact flow that I have implemented, please review and let me know if there is anything that could have been done in better way.

When object is uploaded to system I am marking it with status Tagged and also capturing creation date. My Lifecycle rule is configured with 30 days from creation. So, I have a scheduler which calculates difference between today's date and object creation date for all objects with status Tagged, and check if diff is greater than equal to 30. If so, it updates status to Archived.

If user performs any operation on object with status Archived, we explicitly check in s3 whether object is actually moved to glacier or not. If not we perform operation requested. If moved to glacier we initiate restore process and wait for restore to finish to initiate operation requred.

I appreciate your thoughts and would like to hear your inputs on above approach that i have taken.

Regards.

Answer 1

If I wanted to implement this, I would set the storage class of the object inside my database as "Glacier/Archived" at the beginning of the day it is supposed to transition.

You already know your lifecycle policies, and, as part of object metadata, you also know the creation time of each object. Then it becomes a simple query, which can be scheduled to run every night at 12:00 AM.

You could further enhance your application by defining an algorithm that checks if an object has transitioned to Glacier today, at the moment when object access is requested, it would go and explicitly check if it is actually transitioned or not. If it is marked as Glacier/Archive for more than a day, then checking is no longer required.

Of course, if for any reason, the above solution doesn't work for you, it is possible to write a scanner application to continuously check the status of those objects that are supposed to transition at "DateTime.Today" and are not marked as Glacier/Archive yet.