Accuracy remains constant after every epoch

0
votes

I Have created a model to classify plane and cars images bu after very epoch the acc and val_acc remains same

import numpy as np
import matplotlib as plt
from keras.models import Sequential
from keras.layers import Convolution2D
from keras.layers import MaxPooling2D
from keras.layers import Flatten
from keras.layers import Dense
from keras.preprocessing.image import ImageDataGenerator
from keras.preprocessing.image import image
import os 

model=Sequential()
model.add(Convolution2D(32,(3,3),input_shape=(64,64,3),activation="relu"))
model.add(MaxPooling2D(2,2))
model.add(Convolution2D(64,(3,3),activation="relu"))
model.add(MaxPooling2D(2,2))
model.add(Convolution2D(64,(3,3),activation="sigmoid"))
model.add(MaxPooling2D(2,2))
model.add(Flatten())

model.add(Dense(32,activation="sigmoid"))
model.add(Dense(32,activation="sigmoid"))
model.add(Dense(32,activation="sigmoid"))
model.add(Dense(1,activation="softmax"))
model.compile(optimizer='adam', loss='binary_crossentropy',
               metrics=['accuracy'])
 train_datagen = ImageDataGenerator(
    rescale=1./255,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True)

test_datagen = ImageDataGenerator(rescale=1./255)

train_set = train_datagen.flow_from_directory( 'train_images', target_size=(64,64), batch_size=32, class_mode='binary')

test_set = train_datagen.flow_from_directory( 'val_set', target_size=(64,64), batch_size=32, class_mode='binary')

model.fit_generator( train_set, steps_per_epoch=160, epochs=25, validation_data=test_set, validation_steps=40)

Epoch 1/25
30/30 [==============================] - 18s 593ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 2/25
30/30 [==============================] - 15s 491ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 3/25
30/30 [==============================] - 19s 640ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 4/25
30/30 [==============================] - 14s 474ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 5/25
30/30 [==============================] - 16s 532ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 6/25
30/30 [==============================] - 14s 473ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 7/25
30/30 [==============================] - 14s 469ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 8/25
30/30 [==============================] - 14s 469ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 9/25
30/30 [==============================] - 14s 472ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 10/25
30/30 [==============================] - 16s 537ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 11/25
30/30 [==============================] - 18s 590ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 12/25
30/30 [==============================] - 13s 441ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 13/25
30/30 [==============================] - 11s 374ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 14/25
30/30 [==============================] - 11s 370ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 15/25
30/30 [==============================] - 13s 441ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 16/25
30/30 [==============================] - 13s 419ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 17/25
30/30 [==============================] - 12s 401ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 18/25
30/30 [==============================] - 16s 536ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 19/25
30/30 [==============================] - 16s 523ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 20/25
30/30 [==============================] - 16s 530ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 21/25
30/30 [==============================] - 16s 546ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 22/25
30/30 [==============================] - 15s 500ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 23/25
30/30 [==============================] - 16s 546ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 24/25
30/30 [==============================] - 16s 545ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
Epoch 25/25
30/30 [==============================] - 15s 515ms/step - loss: 7.9712 - acc: 0.5000 - val_loss: 
7.9712 - val_acc: 0.5000
2
keras
This happens because the model is not learning. The gradients in the intermediate layers are same. Any reason why you are using sigmoid in the intermediate layers instead of relu ? – YOLO
Actually it's because of the softmax. While using binary_crossentropy we should use sigmoid. – venkatesh_akhouri
And while using sigmoid instead of relu,the model will learn but the accuracy will be around 50% – venkatesh_akhouri

2 Answers

1
votes

You have several issues in your model structure.

  • First of all, for the output of your model

model.add(Dense(1,activation="softmax"))

You are using a softmax, which means you try to solve a multi-class classification, not a binary classification. If it is really the case, you need to change your loss to categorical_crossentropy. In this way the compile line will turn into:

model.compile(optimizer='adam', loss='categorical_crossentropy',
               metrics=['accuracy'])

If it is not the case and you want only solve a binary classification, you might be good, but I do suggest to change the last layer activation to sigmoid

  • Second: That is a bad idea to use sigmoid as the activation in the middle layer since it can easily cause the gradient to vanish (read more here ). Try to change all the sigmoid activation in the middle layer with wether relu or even better with leakyrelu
0
votes

The problem is exactly here:

model.add(Dense(1,activation="softmax"))

You cannot use softmax with one neuron, as it normalizes over neurons, meaning that with one neuron it will always produce a constant 1.0 value. For binary classification you have to use sigmoid activation at the output:

model.add(Dense(1,activation="sigmoid"))

Also it is not wise to use sigmoid activations in hidden layers, as they will produce vanishing gradient problems. Please prefer ReLU or similar activations.