Training acc decreasing, validation - increasing. Training loss, validation loss decreasing

0
votes

I am trying to implement LRCN but I face obstacles with the training. Currently, I am trying to train only the CNN module, alone, and then connect it to the RNN. The result you see below is somewhat the best possible one I have achieved so far. The problem with it is that everything seems to be going well except the training accuracy. It is gradually dropping.

My model has aggressive dropouts between the FC layers, so this may be one reason but still, do you think something is wrong with these results and what should I aim for changing if they continue the trend?

The number classes to predict is 3.The code is written in Keras.

Epoch 1/20 16602/16602 [==============================] - 2430s 146ms/step - loss: 1.2583 - acc: 0.3391 - val_loss: 1.1373 - val_acc: 0.3306

Epoch 00001: val_acc improved from -inf to 0.33058, saving model to weights.01-1.14.hdf5 Epoch 2/20 16602/16602 [==============================] - 2441s 147ms/step - loss: 1.1998 - acc: 0.3356 - val_loss: 1.1342 - val_acc: 0.3719

Epoch 00002: val_acc improved from 0.33058 to 0.37190, saving model to weights.02-1.13.hdf5 Epoch 3/20 8123/16602 [=============>................] - ETA: 20:30 - loss: 1.1889 - acc: 0.3325

I have 2 more short questions which I cannot answer in a while.

  1. Why the tensor I output from my custom video data generator is of dimensions: (4, 288, 224, 1) but the layer of my input shape is generated as (None, 288, 224, 1)? To clarify the shape, I am classifying batches of 4 containing single images in a non-time distributed CNN. I use functional API.
  2. Later, when I train the RNN, I will have to make predictions per time-step, then average them out and choose the best one as a prediction of my overall model's prediction. Does metrics['accuracy'] do that or I need a custom metric function? If the latter, how do I write one as according to: Keras doc. the results from evaluating a metric are not used when training the model. Do I need a custom objective (loss) function for that purpose?

Any help, expertise will be highly appreciated, I really need it. Thank you in advance!

1
pythontensorflowmachine-learningkerasdeep-learning

1 Answers

1
votes

As long as the loss keeps dropping the accuracy should eventually start to grow. Since you only yet trained for 2-3 Epochs, I would say it's normal that the accuracy may fluctuate.

As for your other questions:

  1. The notion for the input shape of a layer is (batchSize, dim1, dim2, nChannels). Since your model doesn't know the batch size before training None is used as a sort of placeholder. The image dimensions seem to be right and a number of Channels of one means that you don't use colored images, so there's only one entry per pixel.
  2. I think that the accuracy metric should do fine, however I have no experience with RNN, so maybe someone else can answer this.