Specifying HDInsight 3.2 Instance Size via powershell

Question

According to new HDInsight pricing schema and release notes, you can now specify the size of the instance (A3-A9 and D3-D14). I, however, couldn't find a way of how to specify the instance size when creating a new cluster via powershell New-AzureHDInsightCluster command. I did find that see that now it accepts -DataNodeVMSize, however it doesn't understand keywords like "A7", and when specifying "Extra Large" it creates a standard A3 cluster. Any suggestions whether it's actually possible to specify instance size at the moment?

The "available" VM sizes https://msdn.microsoft.com/en-us/library/azure/dn197896.aspx

Here is the script I am running:

$VmSize = "Small";

New-AzureHDInsightCluster -Name $clusterName `
              -ClusterType Hadoop `
              -Version $MyClusterVersion `
              -Location $MyClusterLocation `
              -ClusterSizeInNodes $NumClusterNodes `
              -Credential $HdInsightCreds `
              -DefaultStorageAccountName $DefaultStorageAccountFqdn `
              -DefaultStorageAccountKey $storageAccountKey `
              -DefaultStorageContainerName $hadoopContainer' `
              -DataNodeVMSize $VmSize `
              -HeadNodeVMSize $HeadNodeVmSize

Mikel Urkia Mikel Urkia · Accepted Answer · 2015-02-24T08:12:25

As you already mention, as for now, HDInsight clusters only support nodes within A3-A9 and D3-D14.

The link you provided shows pretty clear the codes asociated to each node type and size. The code Small belongs to the instance type A1\Small, which is not supported by HDInsight. Thus, it might consider the use of the smallest available instance, which is, in this case, A3\Large.

You should try specifying other instances that fall under the ranges supported by the HDInsight cluster. I can see you have already tried with extra large. You might want to try typing the code according to the code index available here (under Sizes for Web and Worker Role Instances), for example.

A3: Large
A4: Extralarge

Specifying HDInsight 3.2 Instance Size via powershell

3 Answers