Good day! (or night)
How Low can be misclassification error on a training set for fully connected neural network on CIFAR-10 dataset (for < 30 Hidden layers)?
Is it possible to achive 0.001 or 0?
Can you tell me from your experience?
P.S. I want to train NN starting from overfitting it, and then tweaking parameters to have better generalization...
Since you did not specify any restriction on the width of a hidden layer, the answer is simple - 0. A shallow, single hidden layer neural network (given enough hidden units) can achieve 0 training error (in terms of missclassification) on any finite dataset (as long as there are no two samples with different labels). This comes directly from UAT (Universal Approximation Theorem, from Cybenko). What you will encounter in practise can differ of course, but in general you can get 100% training accuracy on Cifar10.
