HDInsight - Spark cluster validation fails due to subscription limit of -8 cores

Question

When setting up an Azure HDInsight Spark cluster via the Azure Portal, I am getting the following error message:

You have reached your subscription's limit of -8 cores in West Central US. Please choose a data source in a different region to increase your limit for West Central US. The value must be between 0 and 1.

A value between 0 and 1 is obviously not possible.

The validation failure occurs irrespective of the selected region, number / type of worker nodes, etc.

Under Subscription --> Usage + Quotas, it shows that there are enough cores available to run multiple worker nodes of any considered type in any considered region.

Anyone with the same issue and a solution?

Answer 1

After a week-long discussion with Azure support engineers (involving Product Group and ), it turns out that this is a known (but untracked) problem with their backend.

Due to an unresolved issue, they have simply set the max core limit property to 0 in the backend for new users causing the described error.

They have now raised the limit for me to 60, which resolved the issue.

However, there is no way to check the max core limit property on the frontend, which was admitted by the support engineer.

They say it's a temporary problem, which should anyway be fixed soon.

Hopefully this answer will help others to identify the issue more quickly and specifically mention this to the support.