neuralnet: requires numeric/complex matrix/vector arguments

Question

I used the package neuralnet to build a classification model in R. However, I encountered the famous error:

Error in cbind(1, pred) %*% weights [[num_hidden_layers + 1]]:    
requires numeric/complex matrix/vector arguments

There are many other similar questions here, but none of them solved my problem. These are the steps I am taking:

Using model.matrix to create dummy columns (ensuring no column is factor or string)
Creating a manual formula by paste function
Training neuralnet using the one-hot-encoded dataset of step 1 and the formula created in step 2

Up to here, everything is fine. No error. The model converges after 5000 iterations. However, when I use either compute or predict functions to have a prediction on the test data, it gives me the above error.

I am pretty sure that the columns are the same and have the same name. Also, the class is numeric for each and every attribute. I told myself that maybe the test set is not transformed well using model.matrix, so I used the same training set in the predict/compute function! Surprisingly, it gives a similar error for the same training set! If the data is not a numeric/complex matrix, how is it trained at first and cannot be predicted now?

PS: I cannot share the data due to a privacy issue. This is the simplified code:

trainset = model.matrix(~., data=train_roig)
NN_model = neuralnet(f, trainset[,-c(1:2)], hidden = c(4,2))
# NO ERROR
compute(NN_model, trainset[,-c(1:2)])
# GIVES ME THE ERROR
predict(NN_model, trainset[,-c(1:2)])
# GIVES ME THE SAME ERROR

Double-checking column names:

NN_model$model.list$variables == colnames(trainset[,-c(1:2)])
# TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE TRUE

Checking structure of trainset after applying model.matrix:

str(trainset)
# num [1:134260, 1:19] 0 0 0 0 0 0 0 0 0 ... 
# -attr(*, "dimnames") = list of 2
# ..$ : chr [1:134260] "1" "2" "3" "4" ...
# ..$ : chr [1:19] "Y" "n_trips" "age" "sexM" ...

You seem to have some syntax errors here. Things like missing ] and ). Are you sure this is the code you are running? The error message sure make it sound like a data problem but if you are unable to share a reproducible example it's going to be very difficult to help you. — MrFlick
@MrFlick As I was unable to copy-paste the code from the remote desktop, I typed them. That's the reason for the missing ) or ]. I fixed them here. They are correct there. This is a very specific issue and is not easy to create any other reproducible example for it. What other debugging steps you suggest? You mention a data problem. What exactly you mean by it? — Elnaz YZH
What are the column data types involved? What does str(trainset) return? It seems to work with the sample in the help page nn <- neuralnet(Species == "setosa" ~ Petal.Length + Petal.Width, iris, hidden=c(4,2)); predict(nn, iris) so how is your data different? Do you have any NA or infinite values? Try using debugonce(neuralnet:::predict.nn) to step through the code to see what's going on in the function — MrFlick
There is no NA value or infinite number. iris dataset is a data frame; however, I used model.matrix to convert the factor columns to numeric. Therefore, the result of str function is a little bit weird. I added the result of str to my original question if it helps. — Elnaz YZH

Hadij Hadij · Accepted Answer · 2020-12-21T15:58:12

One possible issue is convergence. If the number of stepmax reaches but the default threshold does not meet, then the model will not generate any weight.

Use plot(trainset) to make sure your weights are generated. If it was unable to plot your network due to weight absence, then you need to increase the threshold or stepmax to let the model fit.

This issue happens only with neuralnet. Other functions I worked with, even if they do not converge, will give you the last weight before the final step. However, this package only releases weights if it converges.

neuralnet: requires numeric/complex matrix/vector arguments

1 Answers