Updating Filter weights in CNN

0
votes

I have designed a CNN with 2 convolution layers followed by 2 fully connected layers.

The architecture of my CNN is as follows:

Input -> [Conv1->ReLU->Pool] -> [Conv2->ReLU->Pool] -> FC1 -> FC2 -> output

28x28   28x28         14x14      14x14           7x7     49x1   49x1    10x1

While training the CNN, I have applied backpropagation till the Fully Connected layers. Without updating the filters I am getting the accuracy of around 94% but I need to update the filters for full functioning.

I am not able to figure out how do I propagate the error from FC layer to the filters in the conv layer.

I have referred to various sources but none of them helped me really. How do I propagate the error from a 49x49 FC layer to a 3x3 filter in conv layer?

Can anyone help me with this ?

Thank you.

1
filterconv-neural-networkmaskbackpropagation

1 Answers

0
votes

The same way as you did for the fully connected layers but with convolution instead of matrix multiplications. Suppose you're at layer 5 (L5). During backprop you have a delta matrix that you have computed at L6, let's call it D6. In L5 you have 2tasks... You need to calculate dF that you will use to update filters, and you need to calculate D5 that you will use as the delta in L4.

How did you perform these in the dense layers? Well for dW you multiplied the layer input with the delta, while for the next delta you multiplied your delta with the weights and the derivative of the activation function.

Now comes the magic... If you have a conv layer make a wild guess what you do. Let me help a bit. To calculate dF you convolve D6 with the original input to L5. Looks familiar I hope. And to get the next delta you perform a full convolution of the original filters with D6, and pass that to L4. The pooling layers have a gazillion resources on them and they're pretty simple so I rather skip them.

If you're using python it's a good practice but don't expect it to be usable. It's awfully slow process, so for anything more than playing around use some framework like tensorflow/keras/etc.