Delete a very large number of entries in Azure Table Storage

1
votes

Here's my setup:

I'm running a Node.js Web App in Azure, which is using Azure Table Storage (Non-SQL). To work with table storage I'm using the azure-storage npm module.

What I'm trying to do:

So I have a system that's tracking events for devices. In storage I'm setting my PartitionKey to be the deviceId and I'm setting the RowKey to be the eventId.

Adding events is straight forward; add them one at a time as they occur.

Retrieving them is easy using the query structure.

However, deleting large quantities of entries seems to be a pain. It appears you can only delete one entity at a time. There doesn't seem to be a query based implementation.

There is the option to use batches to create a large batch of delete operations; but I've just found that there is a cap of 100 operations per batch.

So I'm trying to delete all events for a single device; in my current case I have about 5000 events. So to achieve this I first have to query all my events with a GET request (and concatenate them using continuation tokens), then separate them into batches of 100, and then send 50 large requests in order to delete all the entries...

The same thing in SQL would be DELETE * WHERE deviceId='xxxxxxxx'

Surely there must be a better way than this!

1
javascriptnode.jsazureazure-table-storage

1 Answers

3
votes

The same thing in SQL would be DELETE * WHERE deviceId='xxxxxxxx'

Surely there must be a better way than this!

Sadly, there isn't :). You must fetch the entities based on your requirement and then delete them (either in batches or individually).

You can however optimize the fetching process by only fetching PartitionKey and RowKey from your table instead of all attributes as you only need these two attributes for deleting an entity.