Azure CLI storage delete-batch with exclude pattern

Question

Consider this list of blobs (or any storage data):

backup-2018-08-29-0000.archive
backup-2018-08-29-0100.archive
backup-2018-08-29-0200.archive
backup-2018-08-29-0300.archive
backup-2018-08-29-0400.archive
backup-2018-08-29-0500.archive
backup-2018-08-29-0600.archive
backup-2018-08-29-0700.archive
backup-2018-08-29-0800.archive
backup-2018-08-29-0900.archive
backup-2018-08-29-1000.archive
backup-2018-08-29-1100.archive
backup-2018-08-29-1200.archive
backup-2018-08-29-1300.archive
backup-2018-08-29-1400.archive
backup-2018-08-29-1500.archive
backup-2018-08-29-1600.archive
backup-2018-08-29-1700.archive
backup-2018-08-29-1800.archive
backup-2018-08-29-1900.archive
backup-2018-08-29-2000.archive
backup-2018-08-29-2100.archive
backup-2018-08-29-2200.archive
backup-2018-08-29-2300.archive

I wish to delete all files except one. So my initial idea is to use --pattern flag.

--pattern

The pattern used for globbing files or blobs in the source. The supported patterns are '*', '?', '[seq]', and '[!seq]'.

source

But I can not find info about how '*', '?', '[seq]', and '[!seq]' works.

In the command below, what pattern will seize all files excluding backup-2018-08-29-0000.archive?

$ az storage blob delete-batch --source mycontainer --pattern <pattern>

Update

Additional issue is that I have about 10000 backups collected in more than one year. Using non-batch operations seems non practical.

Answer 1

I doubt there is an easy way to do that with wildcards (it would be easy with regex). [seq] and [!seq] works like that:

--pattern backup-2018-08-29-[01]???.archive

would delete all with files where the first digit after 29- is either 0 or 1:

backup-2018-08-29-0000.archive
backup-2018-08-29-0100.archive
backup-2018-08-29-0200.archive
backup-2018-08-29-0300.archive
backup-2018-08-29-0400.archive
backup-2018-08-29-0500.archive
backup-2018-08-29-0600.archive
backup-2018-08-29-0700.archive
backup-2018-08-29-0800.archive
backup-2018-08-29-0900.archive
backup-2018-08-29-1000.archive
backup-2018-08-29-1100.archive
backup-2018-08-29-1200.archive
backup-2018-08-29-1300.archive
backup-2018-08-29-1400.archive
backup-2018-08-29-1500.archive
backup-2018-08-29-1600.archive
backup-2018-08-29-1700.archive
backup-2018-08-29-1800.archive
backup-2018-08-29-1900.archive

[!seq] just negates that:

--pattern backup-2018-08-29-[!01]???.archive

This would delete:

backup-2018-08-29-2000.archive
backup-2018-08-29-2100.archive
backup-2018-08-29-2200.archive
backup-2018-08-29-2300.archive

---

To answer your question. I would rename (copy) the blob to e.g. backup-keep.archive and then delete the remaining backups using the pattern backup-2018-08-29-????.archive

Answer 2

You could use Acquire lease of the blob(in the portal or use az storage blob lease acquire ), then use the command az storage blob delete-batch to delete other blobs. If you lease the blob, the blob could not be deleted, if you want to delete it, just break the lease in the portal or use az storage blob lease break

My test command(I specifie the duration of the lease with 15 seconds):

az storage blob lease acquire --blob-name "azureProfile.txt"--container-name "testdel" --account-key "accountkey" --account-name "storagename" --lease-duration "15"
az storage blob delete-batch --source "testdel" --account-key "accountkey" --account-name "storagename"

It gives a warning, but it works fine on my side.

Check in the portal:

Answer 3

I solved the problem by doing two batch-delete commands:

#!/bin/bash

set -e

# AZURE_CONNECTION_STRING has taken from env
CONTAINER=backups
DATES="201[78]-??-??"

# delete blobs with a range of 1000-2300 timestamps    
az storage blob delete-batch \
  --connection-string $AZURE_CONNECTION_STRING \
  --source $CONTAINER \
  --pattern "$DATES-[1-2][0-9]00--mongo.archive"

# delete blobs with a range of 0100-0900 timestamps
az storage blob delete-batch \
  --connection-string $AZURE_CONNECTION_STRING \
  --source $CONTAINER \
  --pattern "$DATES-0[1-9]00--mongo.archive"

With this script, I'm deleting all backups excluding backups made at midnight (with 0000 timestamp).