Any suggestions to improve my CNN model (always the same low test accuracy)?

Question

I am working on a project to detect the presence of a person in a painting. I have 4000 training images and 1000 test images resized to (256,256,3)

I tried a CNN model with 3 (Conv layers, MaxPool, BatchNormalization) and 2 fully connected layers.

model = Sequential()
model.add(Conv2D(32, kernel_size = (7, 7), activation='relu', input_shape=shape))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(BatchNormalization())
model.add(Conv2D(64, kernel_size=(7,7), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(BatchNormalization())
model.add(Conv2D(96, kernel_size=(5,5), activation='relu'))
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(BatchNormalization())

model.add(Flatten())
model.add(Dense(128, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1, activation = 'sigmoid'))

The train accuracy always converges to 1 (with just 20-50 epochs) and the test accuracy always remains constant around 0.67.

I tried the following:

I tried changing the size of the layers and adding more layers.
I tried data augmentation
I tried smaller images 128x128x3.

But I always have the same results.

I don't know if this is due to the few images I have, or if the architecture isn't big enough to learn from complex paintings.

I thought of trying Transfer Learning (But I don't know if this will help because it is my first time trying it). Also, do you have any idea where can I find trained models?

So, I am asking from some suggestions to improve my model.

is that a test set or validation set? you should observe the validation set loss against train set loss, not the train against the test. Try to split the train data into train and validation set. In addition, if your test set is the validation set, the graph signifies that your model overfits — Damzaky
Yes, actually it is the validation set. I don't understand how it can overfit so quickly while the images are hard to learn from. — Wassim Bouatay
CNN tends to overfit, there are of course many techniques to prevent this and many reasons why it happens, but I don't think this is the right place to discuss this kind of things, you might better have your luck asking on data science stack exchange instead. — Damzaky

Alpay Durukan Alpay Durukan · Accepted Answer · 2020-03-22T15:05:35

It might be that you are overfitting on your training data, in that case you can use dropout. The other thing is If you have not already normalized your data, you can do that. I am not sure whether that would be much help but give it a try with sth like:

X_training = X_training / X_training.max()

Any suggestions to improve my CNN model (always the same low test accuracy)?

2 Answers