Image classification using Keras flow_from_dataframe

Question

I am working on a multi-class image classification problem using Keras. Train and test images are in 2 separate folders i.e. training_set & test_set. I also have 2 csv file train.csv and test.csv which contains details of the images. I am using Keras flow_from_dataframe method.

I have given the code which I am using for this task below. This give cross-validation score of about 75% but on the test set results are very poor (only 20% accuracy). I know overfitting is one possibility but I am also thinking that in the below code I am missing some important piece of details also (may be wrt imagedatagenerator). Because when I am trying to solve different image classification using flow_from_dataframe there also I am getting very poor results on test set.

Can someone please guide me what I am missing or provide pointers.

model = Sequential()
model.add(Conv2D(32, (3, 3), input_shape=(64, 64, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(32, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Conv2D(64, (3, 3)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())  # this converts our 3D feature maps to 1D feature vectors

model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(6))
model.add(Activation('softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

# this is the augmentation configuration we will use for training
train_datagen = ImageDataGenerator(rescale=1./255,
                                   shear_range=0.20,
                                   zoom_range=0.20,
                                   horizontal_flip=True,
                                   vertical_flip=True)

# this is the augmentation configuration we will use for testing:
test_datagen = ImageDataGenerator(rescale=1./255)

train_generator = train_datagen.flow_from_dataframe(dataframe=train_df,
                                                    directory='train/',
                                                    x_col='image_name',
                                                    y_col='label',
                                                    has_ext=True,
                                                    seed=42,
                                                    target_size=(64, 64),
                                                    batch_size=16,
                                                    shuffle=False,
                                                    class_mode='categorical')

test_generator = test_datagen.flow_from_dataframe(dataframe=test_df,
                                                  directory='test/Test set/',
                                                  x_col='image_name',
                                                  y_col=None,
                                                  has_ext=True,
                                                  target_size=(64, 64),
                                                  class_mode=None,
                                                  batch_size=1,
                                                  shuffle=False, 
                                                  seed=42)

STEP_SIZE_TRAIN=train_generator.n//train_generator.batch_size

model.fit_generator(generator=train_generator,
                    steps_per_epoch=STEP_SIZE_TRAIN,
                    epochs=10)

test_generator.reset()
pred = model.predict_generator(test_generator, verbose=1)
predicted_class_indices = np.argmax(pred, axis=1)
labels = train_generator.class_indices
labels = dict((v,k) for k,v in labels.items())
predictions = [labels[k] for k in predicted_class_indices]

test_df['label'] = pd.DataFrame(data=predictions)
submission_df.to_csv('submission.csv', index=False)

It looks like you don't have labels for the test set, is this correct? — Dr. Snoopy
@MatiasValdenegro That's right.. I have referred this article on flow_from_dataframe **y_col ** — (str or list of str) If class_mode is not “other” or not “input” you should pass the name of the column which contains the class names. None, if used for test_generator. — Chetan Ambi
Well the problem with that is that if you don't have labels, you can't really evaluate anything, since the labels are required to compute metrics such as accuracy. You need labels for any meaningful evaluation on the test set. — Dr. Snoopy
The code I have put above is only for final code. I am getting around 75% accuracy on validation. But accuracy on test test is poor. I don't have access to test set. When I upload predictions on test set I get only around 20% accuracy. Please let me know if you need any further details — Chetan Ambi

Chetan Ambi Chetan Ambi · Accepted Answer · 2019-01-02T02:50:59

After some research I am able to locate the issue. The order in which flow_from_dataframe or flow_from_directory reads file is different than order in which files are stored in folder. Because of this predictions were incorrectly mapped in the submission. To resolve this issue I have appended zeros to the file names so that order in which flow_from_dataframe reads the files matches with files stored in local system.

Print the generator's filenames test_generator.filenames and compare it with os.listdir. If you see difference in the order of files then we need to correct it to resolve this issue.

Used below code to rename the files:

for i in os.listdir(folder):
    filename, extension = (os.path.splitext(i))
    filenumber = filename.zfill(5)
    new_filename = (f'{filenumber}{extension}')
    os.rename(i, new_filename)

Image classification using Keras flow_from_dataframe

1 Answers