how to create azure batch pool based on custom VM image using java sdk

Question

I want to use custom ubuntu VM image that I had created for by batch job. I can create a new pool by selecting the custom image from the azure portal itself but I wanted to write build script to do the same using the azure batch java sdk. This is what I was able to come up with:

List<NodeAgentSku> skus = client.accountOperations().listNodeAgentSkus().findAll({ it.osType() == OSType.LINUX })
String skuId = null
ImageReference imageRef = new ImageReference().withVirtualMachineImageId('/subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.Compute/images/$CUSTOM_VM_IMAGE_NAME')

for (NodeAgentSku sku : skus) {
    for (ImageReference imgRef : sku.verifiedImageReferences()) {
        if (imgRef.publisher().equalsIgnoreCase(osPublisher) && imgRef.offer().equalsIgnoreCase(osOffer) && imgRef.sku() == '18.04-LTS') {
            skuId = sku.id()
            break
        }
    }
}

VirtualMachineConfiguration configuration = new VirtualMachineConfiguration()
configuration.withNodeAgentSKUId(skuId).withImageReference(imageRef)
client.poolOperations().createPool(poolId, poolVMSize, configuration, poolVMCount)

But I am getting exception:

Caused by: com.microsoft.azure.batch.protocol.models.BatchErrorException: Status code 403, {
  "odata.metadata":"https://analyticsbatch.eastus.batch.azure.com/$metadata#Microsoft.Azure.Batch.Protocol.Entities.Container.errors/@Element","code":"AuthenticationFailed","message":{
    "lang":"en-US","value":"Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:bf9bf7fd-2ef5-497b-867c-858d081137e6\nTime:2019-04-17T23:08:17.7144177Z"
  },"values":[
    {
      "key":"AuthenticationErrorDetail","value":"The specified type of authentication SharedKey is not allowed when external resources of type Compute are linked."
    }
  ]
}

I definitely think the way I am getting the skuId is wrong. Since client.accountOperations().listNodeAgentSkus() does not list the custom image, I just thought of giving skuId based of the ubuntu version that I had used to create the custom image.

So what is the correct way to create pool using custom VM image for azure batch account using java sdk?

Answer 1

You must use Azure Active Directory credentials in order to create a pool with a custom image. It is in the prerequisites section of the Batch Custom Image doc.

This is a frequently asked question:

1. Custom Image under AzureBatch ImageReference class not working
2. Azure Batch Pool: How do I use a custom VM Image via Python?

Answer 2

Just shows as the error, you need to authenticate to Azure first and then you could create the pool with a custom image as you want.

First, you need an Azure Batch Account, you can create it in the Azure portal or using Azure CLI. Or you also can create the batch account through Java. See Manage the Azure Batch Account through Java.

Then I think you also need to authenticate to your batch account. There are two ways below:

• Use the account name, key, and URL to create a BatchSharedKeyCredentials instance for authentication with the Azure Batch service. The BatchClient class is the simplest entry point for creating and interacting with Azure Batch objects.

  BatchSharedKeyCredentials cred = new BatchSharedKeyCredentials(batchUri, batchAccount, batchKey); BatchClient client = BatchClient.open(cred);

• The other way is using AAD (Azure Active Directory) authentication to create the client. See this document for detail.

  BatchApplicationTokenCredentials cred = new BatchApplicationTokenCredentials(batchEndpoint, clientId, applicationSecret, applicationDomain, null, null); BatchClient client = BatchClient.open(cred);

Then you can create the pool with the custom as you want. Just like this:

System.out.println("Created a pool using an Azure Marketplace image.");

VirtualMachineConfiguration configuration = new VirtualMachineConfiguration();
configuration.withNodeAgentSKUId(skuId).withImageReference(imageRef);
client.poolOperations().createPool(poolId, poolVMSize, configuration, poolVMCount);

System.out.println("Created a Pool: " + poolId);

For more details, see Azure Batch Libraries for Java.