Access the leaderboard in h2o automl in between and extract metrics of models that are completed?

Question

In h2o automl can we find which models are completed and see their metrics while training is going on ?

eg: maxmodels=5 , so can we extract the information of model 1 while others(models 2 ,3) are getting trained.

Seb Seb · Accepted Answer · 2020-07-24T15:58:52

If you're using Python or R client to run H2OAutoML, you can see the progression — e.g. which model is being built or completed — using the verbosity='info' parameter. For example:

aml = H2OAutoML(project_name="my_aml",
                ...,
                verbosity='info')
aml.train(...)

If you're using the Flow web UI, this progress is shown automatically when you click on the running job.

You can then obtain the model ids from those console logs, and retrieve the models as usual in a separate client:

m = h2o.get_model('the_model_id')

or you can even access the current state of the leaderboard with the model metrics:

h2o.automl.get_automl('my_aml').leaderboard

The same logic applies to R client.

An alternative, maybe simpler if the goal is just to monitor progress, is to use Flow ( usually availble at http://localhost:54321 ) > 'Admin' > 'Jobs' > search for the AutoML job..

Access the leaderboard in h2o automl in between and extract metrics of models that are completed?

1 Answers