Delete files in batch from azure blob storage using service account

Question

I am using azure blob storage to store my project files.

I have a service account of azure blob storage(client_id and client_secret).I have created CloudBlobClient using StorageCredentialsToken as below:

StorageCredentialsToken credentialsToken = new StorageCredentialsToken("account name", "access token generated uing client_id and client_secret");
CloudBlobClient blobClient = new CloudBlobClient(new URI("https://accountname.blob.core.windows.net/"), credentialsToken);
CloudBlobContainer cloudBlobContainer = blobClient.getContainerReference("conteiner name");

Now using CloudBlobContainer I can delete one file at a time:

CloudBlockBlob blockBlobReference = cloudBlobContainer.getBlockBlobReference(key);
if (blockBlobReference.exists()) {
    blockBlobReference.delete();
}

How can I delete multiple files using a single call?

I find this doc which says we can delete multiple files with BlobBatchClient. In the document, I can not find any ways to create BlobBatchClient using a service account(using access token obtained by client_id and client_secret).

Can we delete files in async call as I need to delete 100s of files? Any alternative solutions to delete files in batch?

SDK version compile group: 'com.microsoft.azure', name: 'azure-storage', version: '8.6.5'

The doc shows that you could create a BlobBatchClient from a BlobServiceClient. Have you tried to use BlobServiceClient? — Pamela Peng
There is no way to create BlobServiceClient using the service account (StorageCredentialsToken) ref: github.com/Azure/azure-sdk-for-java/tree/master/sdk/storage/… — Nitin
Sorry, the link in my previous comment is about Python. I can just find the method using StorageSharedKeyCredential with account_name and account_key to create BlobServiceClient. — Pamela Peng
@NitinVavdiya In the new java storage SDK v12, StorageCredentialsToken has been removed. If you want to use Azure AD auth to access blob, we need to provide a TokenCredential object. For more details, please refer to github.com/Azure/azure-sdk-for-java/issues/6509 — Jim Xu
Hey, to create StorageSharedKeyCredential we need accountKey. what is accountKey here? I tried with passing access token generated using client_id and client_secret but it is giving error java.lang.IllegalArgumentException: Illegal base64 character 2e It can not decode the access token — Nitin

Nitin Nitin · Accepted Answer · 2020-10-19T13:38:11

As per comment by Jim, I have created BlobServiceAsyncClient using access token sample method:

public void delete(List<String> files) {
        String endpoint = "https://azureaccount.blob.core.windows.net/";
        AccessToken accessToken = new AccessToken("access token created with client id and client secret", OffsetDateTime.now().plusHours(1)); 
        BlobServiceAsyncClient storageClient = new BlobServiceClientBuilder().credential(request -> Mono.just(accessToken))
                .endpoint(endpoint)
                .buildAsyncClient();
        BlobBatchClient blobBatchClient = new BlobBatchClientBuilder(storageClient).buildClient();
        List<String> blobUrls = new ArrayList<>();
        files.forEach(name -> {
            try {
                String blobUrl = endpoint + "conteinerName/" + URLEncoder.encode(name, "UTF-8");
                blobUrls.add(blobUrl);
            } catch (UnsupportedEncodingException e) {
                LOGGER.debug("Can not encode blob name={}", name);
            }
        });
        blobBatchClient.deleteBlobs(blobUrls, DeleteSnapshotsOptionType.INCLUDE).forEach(response -> {
                    LOGGER.debug("File with name={} deleted, status code={}", response.getRequest().getUrl(), response.getStatusCode());
                }
        );
}

Gradle dependencies:

compile group: 'com.azure', name: 'azure-storage-blob', version: '12.0.0'
compile group: 'com.azure', name: 'azure-storage-blob-batch', version: '12.6.0'

Delete files in batch from azure blob storage using service account

1 Answers