TTL By Document with Cosmos Db Mongo API

2
votes

In Cosmos Db, I am using a document level Time to Live (TTL) and Cosmos does not appear to be expiring documents. Does this feature work in Cosmos Db using MongoDB API? If it does, what am I missing?

  • I am using Cosmos Db with the MongoDB API.
  • A "ttl" field is set in each document for my collection.
  • In Azure, Time to Live is set to "On (no default)" for my collection.
  • I am doing this without the emulator because the emulator defaults to the SQL API. In the emulator, I see "_ts" set and I do not see this field in Azure.
  • I can switch to collection level expiration by setting Time to Live to "On" and documents expire as expected. When I do this, my "ttl" field is ignored and the value I set for "second(s)" in Azure is followed. I still see my "ttl" field in the document.

Although I don't see a "_ts" field in my documents, an article about indexing mentions that it is a reserved property. This makes be think that it is set behind the scenes and it is not returned in queries.

https://docs.microsoft.com/en-us/azure/cosmos-db/mongodb-indexing "_ts is a Cosmos DB-specific field and is not accessible from MongoDB clients. It is a reserved (system) property that contains the timestamp of the document's last modification."

Update: I checked the MongoDB support page (https://docs.microsoft.com/en-us/azure/cosmos-db/mongodb-feature-support) and it indicates that collection level TTL is available and says nothing about document level.

Azure Cosmos DB supports a relative time-to-live (TTL) based on the timestamp of the document. TTL can be enabled for MongoDB API collections through the Azure portal.

Update: My Azure Portal Preview Features now show this: Cosmos DB Preview Features

1
azure-cosmosdb

1 Answers

1
votes

I got document level Time to Live working in Cosmos Db using MongoDb API. I had to ask help from Microsoft support to get this working. Response from Microsoft Big data team was following.

Before enabling Document level TTL feature , I would like to clarify following about Document TTL feature details here.

The TTL feature is controlled by TTL properties at two levels - Collection level and the Document level.

Right now per Document level TTL for MongoDB accounts are not available by default. However, we can enable this feature for specific customers and this feature is set at an account level.

TTL is at a document level but the feature is enabled at an account level which means for all collections under the account, if there is a document with a TTL set, it will take effect. For other collections, if the TTL value is not set for each document, it would not be affected.

You needs to have an index on the _ts field for this to work.

To summarize this : - This feature works at Cosmos DB account level. We need to enable Document TTL feature in Cosmos DB backend on our side.