Not able to load weights after fine tuning the model with VGG16

Question

I have loaded the weights from VGG16 and added to my Sequential Model. I want to train the lower weights of VGG16 by freezing the top layers (Fine Tuning).

Everything was good: I was able to build the model and predict new images. But now I want to load the model, which I was unable to do.

This is what I have tried shown as following code:

model1 = applications.VGG16(weights='imagenet', 
include_top=False,input_shape=(img_width,img_height,3))

train_datagen = ImageDataGenerator(rescale=1./255,
                                   rotation_range=40,
                                   width_shift_range=0.2,
                                   height_shift_range=0.2,
                                   shear_range=0.2,
                                   zoom_range=0.2,
                                   horizontal_flip=True,
                                   fill_mode='nearest')

test_datagen = ImageDataGenerator(rescale=1./255)


 train_generator = train_datagen.flow_from_directory(train_data_dir,
                                        target_size=(img_width, img_height),
                                        batch_size=size_batch, 
                                        class_mode='binary', 
                                        shuffle=False) 


# repeat with the validation data

 test_generator = test_datagen.flow_from_directory(validation_data_dir,
                                       target_size=(img_width, img_height),
                                       batch_size=size_batch,
                                       class_mode='binary',
                                       shuffle=False)


 model = Sequential()
 model.add(Flatten(input_shape=model1.output_shape[1:]))
 model.add(Dense(256, activation='relu'))
 model.add(Dropout(0.2))
 model.add(Dense(1, activation='sigmoid'))

 new_model=Sequential()

 for l in model1.layers:
      new_model.add(l)

  new_model.add(model)

 for layer in new_model.layers[:25]:
       layer.trainable=False

 new_model.compile(optimizer=optimizers.SGD(lr=1e-3, 
 momentum=0.9),loss='binary_crossentropy',
              metrics=['accuracy'])
 checkpoint = ModelCheckpoint(fine_tuned_model_path, monitor='val_acc', 
                             verbose=1, save_best_only=True, 
                             save_weights_only=False, mode='auto')
# fine-tune the model
fit=new_model.fit_generator(train_generator,
                          steps_per_epoch=33,
                          nb_epoch=1,

                          validation_data=test_generator,

                          verbose=1,callbacks=[checkpoint]) 

I then was trying to load the model:

load_model("C:/Users/hi/POC/Fine_Tune/model.h5")

This is the error I am receiving:

ValueError: You are trying to load a weight file containing 14 layers into a model with 1 layers.

Answer 1

According to Keras issue 8898, this error can be avoided by editing the Keras code keras/applications/vgg16.py

so that the line(s) that used to read model.load_weights(weights_path) now read model.load_weights(weights_path, by_name=True)

I have found this to work for imagenet weights with other Applications models as well, e.g. nasnet.

Answer 2

I don't see why you had to define a new model and load the previous layers of VGG16 into your new model.

The best work around that l would advice is freezing the layer of the VGG16 architecture you want and have the ones you want as trainable layers, as you did in the last for loop

This will ultimately result in you removing the two for loops you have embedded inside.

# the way you loaded your images and did not include_top layer 
model1 = applications.VGG16(weights='imagenet', include_top=False, input_shape = (img_width, img_height, 3))

#to see the structure of your architecture
model.summary() 

#freezing the layers you do not want for training in your architecture
for layer in model1.layers[:25]:
    layer.trainable = False

#the rest is the same from here on forth with the exclusion of the two for loops
#which you need to remove as they are no longer required.
train_datagen = ImageDataGenerator(rescale=1./255,
                               rotation_range=40,
                               width_shift_range=0.2,
                               height_shift_range=0.2,
                               shear_range=0.2,
                               zoom_range=0.2,
                               horizontal_flip=True,
                               fill_mode='nearest')

#etc...