Azure Machine Learning - slow to start

0
votes

I am working with Azure AI and initially had been using Machine Learning Studio Classic.

It was working well but was slow training my models. From looking around, it seems that if I use Azure Machine Learning Studio, I can control the hardware used to run the experiments, so this is what I am trying.

My issue is that Azure Machine Learning Studio is extremely slow in starting the experiments-it can take 10 minutes to even start.

Is this as expected or am I missing something?

Incidentally, NC24 was actually slower than NC6 - is this because of the configuration of my experiment?

GPU Training Whole run NC6 2m 36s 10m 48s NC24 2m 52s 16m 48s

1
azure-machine-learning-studioazure-machine-learning-service

1 Answers

1
votes

I'm assuming that you're:

  • using Azure ML Studio (i.e. ml.azure.com/)'s Pipeline Designer, and
  • creating a new compute target before running?

If so, then 10 minutes is normal for the first run, given that a cluster of VMs has to be created and provisioned with a Docker container and Conda environment. After the run first completes the compute target is configured to stay on and available for two hours so future runs should execute without the 10 minute delay (provided you don't change the Conda dependencies or choose a new compute target).

more information about AMLCompute