improve learning of neural network

1
votes

Can someone help me what to do with a classification, if I get a training and validation error shown in the picture to improve my neural network? I tried to stop the training earlier, so that the validation error is smaller, but it's still too high. I get a validation accuracy of 62.45%, but thats too low. The dataset are images that show objects somewhere in the image (not centered). If I use the same network with the same number of images, but where the shown objects are always centered to the principal point, it works much better with a validation accuracy of 95%,

enter image description here

1
neural-networkclassificationconv-neural-networkvalidationerror
You must precise size of your sets and of your classes. Seems like you have not enough data to cover all positions. – pirela
Thanks! So I have to change the batch size, right? Do I have to make it smaller, is that right? And with size of my classes you mean to put more images per class in it? – csi
Seems you You overfit. Try to put more images with several positions per class. One way is to flip you images left right so you obtain twice data. – pirela

1 Answers

1
votes

One can look for following things while implementing the Neural net:

  1. Dataset Issues:

    i) Check if the input data you are feeding the network makes sense and is there too much noise in the data.

    ii) Try passing random input and see if the error performance persist. If it does, then it's time to make changes in your net.

    iii) Check if the input data has appropriate labels.

    iv) If the input data is not shuffled and is passed in a specific order of label, leads to negative impact on the learning. So, shuffling of data and label together is necessary.

    v) Reduce the batch size and make sure batch don't contain the same label.

    vi) Too much data augmentation is not good as it has a regularizing effect and when combined with other forms of regularization (weight L2, dropout, etc.) can cause the net to underfit.

    vii) Data must be pre-processed as per the requirement of the data. For example, if you are training the network for face classification then the image face without or very any background should be passed to the network for learning.

  2. Implementation Issues:

    i) Check your loss function, weight initialization, and gradient checking to make sure the backpropagation works in an appropriate manner.

    ii) Visualize the biases, activation, and weights for each layer with help of visualization library like Tensorboard.

    iii) Try using dynamic learning rate concept, where the learning rate changes with a designed set of epochs.

    iv) Increase the network size by adding more layer or more neurons, as it might not be enough to capture the features of its mark.