Does convolution kernel need to be designed in CNN (Convolutional Neural Networks)?

Question

I am new to Convolutional Neural Networks. I am reading some tutorial and testing some sample codes using Keras. To add a convolution layer, basically I just need to specify the number of kernels and the size of the kernel.

My question is what each kernel looks like? Are they generic to all computer vision applications?

A kernel is just a matrix that is initialized with random numbers. The (n,m) size you specify for the kernel is the size of the n x m matrix. Note, the word "kernel" is interchangeably used with the word "filter." This video explains the concept in further detail: youtu.be/YRhxdVk_sIs — blackHoleDetector

DarkCygnus DarkCygnus · Accepted Answer · 2018-01-05T16:21:21

My question is what each kernel looks like?

This depends on the parameters you chose for your Convolutional Layer:

It will indeed depend on the kernel_size parameter you mentioned, as it will determine the shape and size of your kernel. Say you pass this parameter as (3,3) (on a Conv2D layer naturally), you will then obtain a 3x3 Kernel Matrix.
It will depend on your kernel_initializer parameter, which determines the way that MxN Kernel Matrix is going to be filled. It's default value is "glorot_uniform", which is explained on its doc page:

Glorot uniform initializer, also called Xavier uniform initializer. It draws samples from a uniform distribution within [-limit, limit] where limit is sqrt(6 / (fan_in + fan_out)) where fan_in is the number of input units in the weight tensor and fan_out is the number of output units in the weight tensor.

This is telling us the specific way it fills that kernel matrix. You may well select any other kernel initializer you desire to fit your needs. You may even build Custom Initializers, also exemplified in that doc page:
```
from keras import backend as K

def my_init(shape, dtype=None):
    #or whatever you want to customize
    return K.random_normal(shape, dtype=dtype)

model.add(Dense(64, kernel_initializer=my_init))
```
Furthermore, it will depend on your kernel_regularizer parameter, which defines regularization functions applied to the weights of your kernel. It's default value is None but you can select others from the ones available. You can again define your own custom initializers in a similar fashion:
```
def l1_reg(weight_matrix):
    #same here, fit your own needs
    return 0.01 * K.sum(K.abs(weight_matrix))

model.add(Dense(64, input_dim=64,
        kernel_regularizer=l1_reg)
```

Are they generic to all computer vision applications?

This I think may be a bit broad, however I would venture and say yes. Keras has available many kernels that were designed to specifically adapt to Deep Learning applications; it includes those ones that are most commonly used throughout the literature and well-known applications.

The good thing is that, as illustrated before, if any of those kernels does not fit your needs you could well define your own custom initializer, or well enhance it by using regularizes. This enables you to tackle those really specific CV problems you may have.

Does convolution kernel need to be designed in CNN (Convolutional Neural Networks)?

2 Answers