Keras CNN model parameters calculation

Question

My cnn model, which is created using Keras 1.1.1, has two convolution-pooling layers followed by two dense layers, and dropout is added following the second convolution-pooling layer and the first dense layer. The codes are as follows:

model = Sequential()
#convolution-pooling layers
model.add(Convolution2D(32, 5, 5, input_shape=input_shape))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Convolution2D(64, 5, 5))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
#dense layers
model.add(Flatten()) 
model.add(Dense(100))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add((Dense(2)))
model.add(Activation('softmax'))
#optimizer
sgd = SGD(lr=1e-3, decay=1e-6, momentum=0.9, nesterov=True) 
model.compile(loss='categorical_crossentropy',
              optimizer = sgd,
              metrics=['accuracy'])
print model.summary()

The model summary gives the table as follows:

I am not clear of how the number of parameters of the second convolution layer (i.e., 51264 indicated by the red rectangle) is computed. I thought the number would be (5*5 + 1)*64 = 1664, since the convolution kernel is 5*5 in size and 64 feature maps are to be extracted.

Besides, I have already implemented dropout. Why does the parameter table not reflect this point. It seems the parameter number without dropout is given, although the dropout (layer) is listed in the table. Anyone can help me to interpret the parameter summary?

Regarding dropout: This randomly disables neurons during training. They still are present in your model and therefore aren´t discounted from the number of params in your model summary. — petezurich

blitu12345 blitu12345 · Accepted Answer · 2017-06-18T09:08:09

Its a rather simple calculation with basic concept.And by looking at your code and model summary this were my steps.

Step 1: Formula to calculate parameters

total_params =
(filter_height * filter_width * input_image_channels + 1) * number_of_filters

Step 2: Calculate parameters for first layer

filter_height = 5,
filter_weight = 5,
input_image_channels = 1
number_of_filters = 32
Though you havent provided us with imput image channels, but i figured it out from by your parameters value.

Now we will calculate the number of parameters for first conv layer.

total_param = (5*5*1 + 1)*32 = 832

Step 3: Similarly we can calculate for 2nd conv layer. Note that number of filters from previous layer become the number of channels for current layer's input image.

filter_height = 5,
filter_weight = 5,
input_image_channels = 32
number_of_filters = 64

Now we will calculate the number of parameters for 2nd conv layer.

total_param = (5*5*32 + 1)*64 = 51264

Keras CNN model parameters calculation

3 Answers