Why are my Keras Conv2D kernels 3-dimensional?

Question

In a typical CNN, a conv layer will have Y filters of size NxM, and thus it has N x M x Y trainable parameters (not including bias).

Accordingly, in the following simple keras model, I expect the second conv layer to have 16 kernels of size (7x7), and thus kernel weights of size (7x7x16). Why then are its weights actually size (7x7x8x16)?

I understand the mechanics of what is happening: the Conv2D layers are actually doing a 3D convolution, treating the output maps of the previous layer as channels. It has 16 3D kernels of size(7x7x8). What I don't understand is:

• why this is Keras's default behavior?
• how do I get a "traditional" convolutional layer without dropping down into the low-level API (avoiding that is my reason for using Keras in the first place)?

_

from keras.models import Sequential
from keras.layers import InputLayer, Conv2D

model = Sequential([
    InputLayer((101, 101, 1)),
    Conv2D(8, (11, 11)),
    Conv2D(16, (7, 7))
])
model.weights

Answer 1

Q1:and thus kernel weights of size (7x7x16). Why then are its weights actually size (7x7x8x16)?

No, the kernel weights is not the size(7x7x16).

from cs231n:

Example 2. Suppose an input volume had size [16x16x20]. Then using an example receptive field size of 3x3, every neuron in the Conv Layer would now have a total of 3*3*20 = 180 connections to the input volume. Notice that, again, the connectivity is local in space (e.g. 3x3), but full along the input depth (20).

Be careful the 'every'.

In your model, 7x7 is your single filter size, and it will connect to previous conv layer, so the parameters on a single filter is 7x7x8, and you have 16, so the total parameters is 7x7x8x16

Q2:why this is Keras's default behavior?

See Q1.

Answer 2

In the typical jargon, when someone refers to a conv layer with N kernels of size (x, y), it is implied that the kernels actually have size (x, y, z), where z is the depth of the input volume to that layer.

Imagine what happens when the input image to the network has R, G, and B channels: each of the initial kernels itself has 3 channels. Subsequent layers are the same, treating the input volume as a multi-channel image, where the channels are now maps of some other feature.

The motion of that 3D kernel as it "sweeps" across the input is only 2D, so it is still referred to as a 2D convolution, and the output of that convolution is a 2D feature map.

Edit:

I found a good quote about this in a recent paper, https://arxiv.org/pdf/1809.02601v1.pdf

"In a convolutional layer, the input feature map X is a W₁ × H₁ × D₁ cube, with W₁, H₁ and D₁ indicating its width, height and depth (also referred to as the number of channels), respectively. The output feature map, similarly, is a cube Z with W₂ × H₂ × D₂ entries. The convolution Z = f(X) is parameterized by D₂ convolutional kernels, each of which is a S × S × D₁ cube."