How to overcome overfitting in CNN - standard methods don't work

5
votes

I've been recently playing around with car data set from Stanford (http://ai.stanford.edu/~jkrause/cars/car_dataset.html). From the very beginning I had an overfitting problem so decided to:

  1. Add regularization (L2, dropout, batch norm, ...)
  2. Tried different architectures (VGG16, VGG19, InceptionV3, DenseNet121, ...)
  3. Tried trasnfer learning using models trained on ImageNet
  4. Used data augmentation

Every step moved me a little bit forward. However I finished with 50% validation accuracy (started below 20%) compared to 99% train accuracy.

Do you have an idea what more can I do to get to around 80-90% accuracy?

Hope this can help some people!:)

2
machine-learningneural-networkdeep-learningartificial-intelligenceconv-neural-network
50% validation accuracy means, its no better than random. Does your training set resemble validation set? If all architecture failed, it must be something to do with this. Training batches are shuffled? – Littleone
@Littleone He has 196 classes, so random would be around 0.5% accuracy. – Imran

2 Answers

5
votes

Things you should try include:

  • Early stopping, i.e. use a portion of your data to monitor validation loss and stop training if performance does not improve for some epochs.
  • Check whether you have unbalanced classes, use class weighting to equally represent each class in the data.
  • Regularization parameter tuning: different l2 coefficients, different dropout values, different regularization constraints (e.g. l1).

Other general suggestions may be to try and replicate the state of the art models on this particular dataset, see if those perform as they should.
Also make sure to have all implementation details ironed out (e.g. convolution is being performed along width and height, and not along the channels dimension - this is a classic rookie mistake when starting out with Keras, for instance).

It would also help to have some more details on the code that you are using, but for now these suggestions will do.
50% accuracy on a 200-class problem doesn't sound so bad anyway.

Cheers

1
votes

For those who encounter the same problem I managed to get 66,11% accuracy by playing with drop out, learning rate and learning decay mainly.

The best results were achieved on VGG16 architecture.

The model is on https://github.com/michalgdak/car-recognition