CNN Visualization of output layers with pre-trained model

Question

I trained my model and saved it:

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Model
from tensorflow.keras.preprocessing.image import load_img
from tensorflow.keras.preprocessing.image import img_to_array

new_model=tf.keras.models.load_model('the_model.h5')
new_model.summary()

img = load_img('e.jpg',target_size=(227,227))
img=img_to_array(img)

img = np.expand_dims(img,axis=0)
img=img/255.
print(img.shape)
#prints out (1,227,227,3) the expected shapes

so the architecture of my model is the following one, i'm using pre-trained resnet50

backbone = ResNet50(input_shape=(227,227,3),weights='imagenet', include_top=False)
    model = Sequential()
    model.add(backbone)
    model.add(GlobalAveragePooling2D())
    model.add(Dropout(0.5))
    model.add(Dense(64,activation='relu'))
    model.add(Dropout(0.5))
    model.add(Dense(1,activation='sigmoid'))

i tried visualize outputs of hidden layers, however with keras or keract i can't get the outputs

with keras :

layer_outputs=[]
for layer in new_model.layers:
    if layer.name=='resnet50':
        temp = [l.output for l in layer.layers]
        layer_outputs=temp
    else:
        layer_outputs.append(layer.output)
    

activation_model = Model(inputs=new_model.input,  outputs=layer_outputs)

the error caused by the last line :

ValueError: Graph disconnected: cannot obtain value for tensor Tensor("input_1:0", shape=(None, 227, 227, 3), dtype=float32) at layer "input_1". The following previous layers were accessed without issue: []

i feel like my model inputs are matching with layer_outputs so i don't really understand the error, indeed when i'm checking :

print(new_model.layers[0].input)
#prints out    :Tensor("input_1:0", shape=(None, 227, 227, 3), dtype=float32)

print(layer_outputs[0])
#prints out :  Tensor("input_1:0", shape=(None, 227, 227, 3), dtype=float32)

when using keract :

a = keract.get_activations(new_model, img)  # with just one sample.
keract.display_activations(a, directory='f', save=True)

tensorflow.python.framework.errors_impl.InvalidArgumentError:  You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,227,227,3]

Any idea of how i can fix it, or another viable solution to get outputs from hidden layers using pre-trained model?

Thanks,

Skaddd Skaddd · Accepted Answer · 2020-08-05T07:20:38

Okey, i found a convenient solution to my problem.

Indeed i think this problem occurs because my sequential model is itself made up of another model (resnet).

Since i didn't add many layers on top of the pre-trained resnet model, i just decided to visualize the feature maps from the resnet model

img = load_img('e.jpg',target_size=(227,227))
img=img_to_array(img)

img = np.expand_dims(img,axis=0)
img=img/255.
print(img.shape)



loaded=tf.keras.models.load_model('age_gender_train.h5')


layer_outputs=[ layer.output for layer in loaded.layers[0].layers]
res = loaded.layers[0]

activation_model = Model(inputs=res.input, outputs=layer_outputs)
activations=activation_model.predict(img)
img = np.squeeze(img,axis=0)

Then you can easily display features maps using activations variable.

Note that since you have the outputs of the resnet model, it might be possible to get the feature maps from the layers on top by repeating the process. Using the output of resnet as input and removing the resnet model from the layer_outputs.(i did not try this, could not work)

Hope it could help someone

CNN Visualization of output layers with pre-trained model

1 Answers