the way to do the cross validation

Question

Let say I have fold1, fold2 , fold3.

I trained fold1,fold2,fold3 with modelA.

A) modelA(fold1) -> modelA(fold2) -> modelA(fold3)

B) modelA(fold1) -> saved weight modelA(fold1) -> modelA(fold2)-> saved weight modelA(fold2) -> modelA(fold3)-> saved weight modelA(fold3) -> ensemble 3 weight

which way is the right way to do the k-fold cross validation and why?

The idea behind cross-validation is that you use for example 90% of the data to train the model, and then test with the 10% remaining data. You do that 10 times (each time with a completely different test set) and take the average. Here it seems like you used all your data to train a model? — Willem Van Onsem

Szymon Maszke Szymon Maszke · Accepted Answer · 2019-06-30T14:28:51

It depends what is your end goal. K-Fold CV is used for finding model hyperparameters.

After this phase you may change your validation dataset and train your model with it.

If you want to harness as much data as you can (performing predictions) it might be a good idea to train N models on N different folds and ensemble their predictions. This one is similar to boostrap, all in all your ensemble saw all the data yet it didn't overfit. This approach is N times more computationally intensive though, so it still comes down to your goals.

Finally, you should get better results with fitting different models to your folds instead of a single one, but this would require separate hyperparameter space for each algorithm.

the way to do the cross validation

1 Answers