Automatically Delete/Expire Azure Blobs after a time period

37
votes

With Azure Blob storage is it possible to either have an individual blob or all blobs within a container delete themselves after a certain period of time similar to Amazon AWS S3's Object Expiration Feature? Or does Azure storage not provide such functionality?

5
azureazure-storage-blobs
See also: stackoverflow.com/questions/tagged/azure-logic-apps – dreftymac

5 Answers

27
votes

It is possible with the Azure Blob storage lifecycle. Please take a look here

https://docs.microsoft.com/en-us/azure/storage/blobs/storage-lifecycle-management-concepts?tabs=azure-portal

15
votes

Because I've missed the feature for years I wrote a small project with a nice 'Deploy to Azure button'. Not yet perfect but works https://github.com/nulllogicone/ExpireBlobFunction

And now I see that Microsoft has released this as a feature on March 27, 2019.

Excerpt from that article:

Azure Blob storage lifecycle management offers a rich, rule-based policy for GPv2 and Blob storage accounts. Use the policy to transition your data to the appropriate access tiers or expire at the end of the data's lifecycle.

The lifecycle management policy lets you:

  • Transition blobs to a cooler storage tier (hot to cool, hot to archive, or cool to archive) to optimize for performance and cost
  • Delete blobs at the end of their lifecycles
  • Define rules to be run once per day at the storage account level Apply rules to containers or a subset of blobs (using prefixes as filters)
3
votes

The Azure Storage Team recently posted (Oct 5, 2017) an update on expiring blobs. It seems that this is now possible using an Azure Logic App template and they will have a native blob storage solution later this year.

Link: Provide Time to live feature for Blobs

We are pleased to announce that we have made an Azure Logic Apps template available to expire old blobs. To set up this automated solution in your environment: Create a new Logic Apps instance, select the “Delete old Azure blobs” template, customize and run. We will release a blog post detailing instructions and providing more templates in the coming weeks.

Allowing users to define expiration policies on blobs natively from storage is still planned for the coming year. As soon as we have progress to share, we will do so. We will continue to provide updates at least once per quarter.

For any further questions, or to discuss your specific scenario, send us an email at [email protected].

2
votes

Azure Storage does not have an expiration feature; you must delete blobs via your app. How you do this is up to you; you'll need to store your expiration date target somewhere (whether in a database or in blob properties).

You can effectively create TTL on blob access, via Shared Access Signatures (by setting an end-date on the SAS). This would let you have an effective way of removing access when it's time to remove access, and then have a follow-on process remove the now-expired blobs.