3
votes

On Linux, H2O 3.16.0.2 - Exception occured while running GBM with hyperparameter search.

The below information is about H2O and its giving exception.

  • H2O cluster uptime: 3 hours 17 mins
  • H2O cluster version: 3.16.0.2
  • H2O cluster version age: 1 month and 4 days
  • H2O cluster name: beast
  • H2O cluster total nodes: 1
  • H2O cluster free memory: 25.46 Gb
  • H2O cluster total cores: 32
  • H2O cluster allowed cores: 32
  • H2O cluster status: locked, healthy
  • H2O connection url: http://localhost:54321
  • H2O connection proxy:
  • H2O internal security: False
  • H2O API Extensions: XGBoost, Algos, AutoML, Core V3, Core V4
  • Python version: 2.7.12 final

    gbm Grid Build progress: [################################################] 100% Errors/Warnings building gridsearch model

    Hyper-parameter: col_sample_rate, 0.44 Hyper-parameter: col_sample_rate_change_per_level, 1.03 Hyper-parameter: col_sample_rate_per_tree, 0.38 Hyper-parameter: histogram_type, QuantilesGlobal Hyper-parameter: max_depth, 4 Hyper-parameter: min_rows, 128.0 Hyper-parameter: min_split_improvement, 1e-06 Hyper-parameter: nbins, 256 Hyper-parameter: nbins_cats, 512 Hyper-parameter: sample_rate, 0.55 failure_details: None failure_stack_traces: java.lang.NullPointerException at hex.Model$Parameters.read_unlock_frames(Model.java:322) at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:209) at hex.ModelBuilder.trainModelNested(ModelBuilder.java:262) at hex.grid.GridSearch.startBuildModel(GridSearch.java:332) at hex.grid.GridSearch.buildModel(GridSearch.java:314) at hex.grid.GridSearch.gridSearch(GridSearch.java:213) at hex.grid.GridSearch.access$000(GridSearch.java:68) at hex.grid.GridSearch$1.compute2(GridSearch.java:135) at water.H2O$H2OCountedCompleter.compute(H2O.java:1263) at jsr166y.CountedCompleter.exec(CountedCompleter.java:468) at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263) at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974) at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477) at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)

    Traceback (most recent call last): File "malware_detection.py", line 167, in validation_frame = valid_split) File "/home/beast/local/lib/python2.7/site-packages /h2o/grid/grid_search.py", line 189, in train self.build_model(parms) File "/home/beast/local/lib/python2.7/site-packages /h2o/grid/grid_search.py", line 204, in build_model self._model_build(x, y, training_frame, validation_frame, algo_params) File "/home/beast/local/lib/python2.7/site-packages /h2o/grid/grid_search.py", line 250, in _model_build failure_messages_stacks += error_message+'\n' TypeError: unsupported operand type(s) for +: 'NoneType' and 'unicode' H2O session _sid_af98 closed.

1
I am experiencing the same problem and can't find out what the problem is. did you solve the issue?dnks23

1 Answers

2
votes

This is particularly frustrating because it waits until after training all models to generate this error.

It was happening to me after running one grid search, changing hyperparameters, then doing a second grid search. Somewhere online saw a way to append new grid search results to old ones, but short of that, restarting h2o got it at least working again for me:

h2o.cluster().shutdown()
h2o.init()