Keras CNN module stops improving accuracy after about 7 epochs

0
votes

I have a data set of 12,311 and I am using 80%, 20% split for validation data. I am applying 4 different random augmentations using a batch generator which works fine as I have tested it. When I train my module, the accuracy seems to stop improving every time after about 7 epochs.

My model:

def nvidiaModel():
        model = Sequential()

        model.add(Conv2D(24, (5, 5), padding="same", subsample=(2, 2), input_shape=(112, 256, 3), activation="elu"))
        model.add(Convolution2D(36, 5, 5, subsample=(2, 2), activation="elu")) # Second CNN
        model.add(Convolution2D(48, 5, 5, subsample=(2, 2), activation="elu")) # Third CNN
        model.add(Convolution2D(64, 3, 3, activation="elu")) # Fourth CNN   # No need for more stride skipping.
        model.add(Convolution2D(64, 3, 3, activation="elu")) # Fifth CNN

        model.add(Flatten())

        model.add(Dense(100, activation="elu"))
        model.add(Dense(50, activation="elu"))
        model.add(Dense(10, activation="elu"))

        model.add(Dense(3, activation="softmax"))   # Which will hold the steering angel.

        optimizer = Adam(lr=1e-5)

        model.compile(loss="mse", optimizer=optimizer, metrics=["accuracy"])

        return model

Summary:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_1 (Conv2D)            (None, 56, 128, 24)       1824      
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 26, 62, 36)        21636     
_________________________________________________________________
conv2d_3 (Conv2D)            (None, 11, 29, 48)        43248     
_________________________________________________________________
conv2d_4 (Conv2D)            (None, 9, 27, 64)         27712     
_________________________________________________________________
conv2d_5 (Conv2D)            (None, 7, 25, 64)         36928     
_________________________________________________________________
flatten_1 (Flatten)          (None, 11200)             0         
_________________________________________________________________
dense_1 (Dense)              (None, 100)               1120100   
_________________________________________________________________
dense_2 (Dense)              (None, 50)                5050      
_________________________________________________________________
dense_3 (Dense)              (None, 10)                510       
_________________________________________________________________
dense_4 (Dense)              (None, 3)                 33        
=================================================================
Total params: 1,257,041
Trainable params: 1,257,041
Non-trainable params: 0

Training arguments:

history = model.fit_generator(batchGenerator(X_train, y_train, 1000, 1),
                              steps_per_epoch = 25,
                              epochs = 30,
                              validation_data = batchGenerator(X_valid, y_valid, 300, 0),
                              validation_steps = 20,
                              verbose = 1,
                              shuffle = 1)

Epochs:

Epoch 1/30
25/25 [==============================] - 52s 2s/step - loss: 0.1709 - acc: 0.6624 - val_loss: 0.1618 - val_acc: 0.6718
Epoch 2/30
25/25 [==============================] - 48s 2s/step - loss: 0.1579 - acc: 0.6764 - val_loss: 0.1524 - val_acc: 0.6767
Epoch 3/30
25/25 [==============================] - 48s 2s/step - loss: 0.1535 - acc: 0.6686 - val_loss: 0.1444 - val_acc: 0.6737
Epoch 4/30
25/25 [==============================] - 48s 2s/step - loss: 0.1460 - acc: 0.6748 - val_loss: 0.1311 - val_acc: 0.7063
Epoch 5/30
25/25 [==============================] - 48s 2s/step - loss: 0.1366 - acc: 0.7076 - val_loss: 0.1262 - val_acc: 0.7370
Epoch 6/30
25/25 [==============================] - 48s 2s/step - loss: 0.1322 - acc: 0.7249 - val_loss: 0.1238 - val_acc: 0.7485
Epoch 7/30
25/25 [==============================] - 48s 2s/step - loss: 0.1313 - acc: 0.7294 - val_loss: 0.1238 - val_acc: 0.7508
Epoch 8/30
25/25 [==============================] - 48s 2s/step - loss: 0.1276 - acc: 0.7370 - val_loss: 0.1173 - val_acc: 0.7538
Epoch 9/30
25/25 [==============================] - 48s 2s/step - loss: 0.1275 - acc: 0.7380 - val_loss: 0.1181 - val_acc: 0.7513
Epoch 10/30
25/25 [==============================] - 50s 2s/step - loss: 0.1260 - acc: 0.7414 - val_loss: 0.1177 - val_acc: 0.7537
Epoch 11/30
25/25 [==============================] - 48s 2s/step - loss: 0.1256 - acc: 0.7430 - val_loss: 0.1159 - val_acc: 0.7553
Epoch 12/30
25/25 [==============================] - 49s 2s/step - loss: 0.1245 - acc: 0.7453 - val_loss: 0.1185 - val_acc: 0.7578
Epoch 13/30
25/25 [==============================] - 49s 2s/step - loss: 0.1232 - acc: 0.7491 - val_loss: 0.1183 - val_acc: 0.7582
Epoch 14/30
25/25 [==============================] - 48s 2s/step - loss: 0.1224 - acc: 0.7501 - val_loss: 0.1219 - val_acc: 0.7423
Epoch 15/30
25/25 [==============================] - 48s 2s/step - loss: 0.1222 - acc: 0.7510 - val_loss: 0.1162 - val_acc: 0.7582
Epoch 16/30
25/25 [==============================] - 49s 2s/step - loss: 0.1218 - acc: 0.7487 - val_loss: 0.1165 - val_acc: 0.7587
Epoch 17/30
25/25 [==============================] - 48s 2s/step - loss: 0.1234 - acc: 0.7454 - val_loss: 0.1185 - val_acc: 0.7442
Epoch 18/30
25/25 [==============================] - 49s 2s/step - loss: 0.1208 - acc: 0.7539 - val_loss: 0.1159 - val_acc: 0.7572
Epoch 19/30
25/25 [==============================] - 49s 2s/step - loss: 0.1215 - acc: 0.7509 - val_loss: 0.1165 - val_acc: 0.7543
Epoch 20/30
25/25 [==============================] - 49s 2s/step - loss: 0.1216 - acc: 0.7507 - val_loss: 0.1171 - val_acc: 0.7590
Epoch 21/30
25/25 [==============================] - 48s 2s/step - loss: 0.1217 - acc: 0.7515 - val_loss: 0.1140 - val_acc: 0.7618
Epoch 22/30
25/25 [==============================] - 49s 2s/step - loss: 0.1208 - acc: 0.7496 - val_loss: 0.1170 - val_acc: 0.7565
Epoch 23/30
25/25 [==============================] - 48s 2s/step - loss: 0.1200 - acc: 0.7526 - val_loss: 0.1169 - val_acc: 0.7575
Epoch 24/30
25/25 [==============================] - 49s 2s/step - loss: 0.1209 - acc: 0.7518 - val_loss: 0.1105 - val_acc: 0.7705
Epoch 25/30
25/25 [==============================] - 48s 2s/step - loss: 0.1198 - acc: 0.7540 - val_loss: 0.1176 - val_acc: 0.7543
Epoch 26/30
25/25 [==============================] - 48s 2s/step - loss: 0.1206 - acc: 0.7528 - val_loss: 0.1127 - val_acc: 0.7608
Epoch 27/30
25/25 [==============================] - 48s 2s/step - loss: 0.1204 - acc: 0.7526 - val_loss: 0.1185 - val_acc: 0.7532

I have tried increasing the batch size, but same result, it will stop improving after certain number of epochs. I tried adding dropout layers, same result.

Anyone have a suggestion on what could be going wrong here?

2
pythontensorflowkerasconv-neural-network

2 Answers

1
votes

Without knowing more about the type of problem you're solving and the dataset involved it's impossible to say for sure. For example, you may have a dirty dataset or hard problem for which 75% is as good as you can get (though I'd say that's unlikely). Another possibility is 75% of your examples come from a single class and your model is just learning the mode (i.e. always guessing that class).

At first glance, I'd try a different loss - mse on the output of a softmax will likely lead to vanishing gradients. If you're doing classification I'd start with (sparse_categorical_)crossentropy.

0
votes

I agree with @DomJack, without knowing more about the input data, it is difficult to give correct directions. However, you could try adding dropout layer and maxpool layer after first, second and third conv layers. You could check couple of learning rates and couple of other optimizers such as adagrad, momentum optimizers. You could also increase number of filters 16,32, 64,128 and 256.

It would be helpful if you can create and share Google colab gist or any other way to reproduce the issue. If your data is private, you could use public datasets and demonstrate the issue. Check couple of optimizers here and here. Hope this helps you. Thanks!