Validation accuracy dropping in CNN Chainer

Question

I am classifying cell images using CNN in Chainer. The validation accuracy of the model is dropping along with the rise of main/training accuracy. I want to know why the validation accuracy is decreasing and what are the ways to increase the validation accuracy.

The data shape is (32, 3, 60, 80) where 32 is batchsize, 3 is channel, 60 and 80 are height and width respectively.

I have applied relu and dropout in all the layers except the last. In that model the training and validation accuracy were unchanged even after 50 epochs. The model that is given below shows a gradually increasing main/training accuracy from 0.78 to 0.98 and validation accuracy changes from 0.79 to 0.66 after 100 epochs.

model = Sequential(
    L.Convolution2D(None, 128, 3, 2),
    F.relu,
    L.Convolution2D(128, 64, 3, 2),
    F.relu,
    L.Convolution2D(64, 32, 3, 2),
    F.relu,
    L.Linear(None, 16),
    F.dropout,
    L.Linear(16, 4)
)

corochann corochann · Accepted Answer · 2019-01-08T13:32:47

This is not unexpected behavior, it is called "overfit". During training, machine learning model will fit to the training data (increasing training accuracy) along with the validation data, but from some time model only fits to the training data so that it won't fit to the validation data (decreasing validation accuracy).

See Overfitting for other reference.

Validation accuracy dropping in CNN Chainer

1 Answers