Using Keras-tuner to create a hyperparameter tuning object and calling the search method, it is easy to retrieve the best hyperparameter configurations once the search is complete, however there does not appear to be any in-built way to also return the corresponding validation loss values on which they are ranked. How can I return the validation losses of each trial alongside the tuner.get_best_hyperparameters(3) method? I expect it may be possible using callbacks but I am not sure how. Depending on the verbose argument I can print results for each trial as it is considered but I would rather be able to call them as I can with the hyperparameters themselves.
1 Answers
0
votes
I think you're right, using callbacks is the easiest way in my opinion to keep track of those metrics. My favorite to use when using keras-tuner is Tensorboard. It is an easy way to keep track of each trial and look at the data in a great interface.
To do so:
Define the callback with a statement like this:tensorboard = TensorBoard(log_dir=path_to_logs, histogram_freq=1, embeddings_freq=1, write_graph=True, update_freq='batch')
Then, in your tuner.search() call, specify your callback using the following argument:callbacks=[tensorboard],
Note that you save the events (callbacks) in a log directory and you can reference it from there. You can either navigate to it in your command line, or load the tensorboard extension in your notebook using %load_ext tensorboard. Then, run this command:tensorboard --logdir=path_to_logs --host=localhost
This will then display an interface to look at each trial. It will have information on the train/validation accuracy and loss, it will give information on model weights over time, and even include information on hyperparameter selection. Worth taking a look at.
