Keras conv1d layer parameters: filters and kernel_size

Question

I am very confused by these two parameters in the conv1d layer from keras: https://keras.io/layers/convolutional/#conv1d

the documentation says:

filters: Integer, the dimensionality of the output space (i.e. the number output of filters in the convolution).
kernel_size: An integer or tuple/list of a single integer, specifying the length of the 1D convolution window.

But that does not seem to relate to the standard terminologies I see on many tutorials such as https://adeshpande3.github.io/adeshpande3.github.io/A-Beginner's-Guide-To-Understanding-Convolutional-Neural-Networks/ and https://machinelearningmastery.com/sequence-classification-lstm-recurrent-neural-networks-python-keras/

Using the second tutorial link which uses Keras, I'd imagine that in fact 'kernel_size' is relevant to the conventional 'filter' concept which defines the sliding window on the input feature space. But what about the 'filter' parameter in conv1d? What does it do?

For example, in the following code snippet:

model.add(embedding_layer)
model.add(Dropout(0.2))
model.add(Conv1D(filters=100, kernel_size=4, padding='same', activation='relu'))

suppose the embedding layer outputs a matrix of dimension 50 (rows, each row is a word in a sentence) x 300 (columns, the word vector dimension), how does the conv1d layer transforms that matrix?

Many thanks

Answer 1

You're right to say that kernel_size defines the size of the sliding window.

The filters parameters is just how many different windows you will have. (All of them with the same length, which is kernel_size). How many different results or channels you want to produce.

When you use filters=100 and kernel_size=4, you are creating 100 different filters, each of them with length 4. The result will bring 100 different convolutions.

Also, each filter has enough parameters to consider all input channels.

---

The Conv1D layer expects these dimensions:

(batchSize, length, channels)

I suppose the best way to use it is to have the number of words in the length dimension (as if the words in order formed a sentence), and the channels be the output dimension of the embedding (numbers that define one word).

So:

batchSize = number of sentences    
length = number of words in each sentence   
channels = dimension of the embedding's output.    

The convolutional layer will pass 100 different filters, each filter will slide along the length dimension (word by word, in groups of 4), considering all the channels that define the word.

The outputs are shaped as:

(number of sentences, 50 words, 100 output dimension or filters)   

The filters are shaped as:

(4 = length, 300 = word vector dimension, 100 output dimension of the convolution)  

Answer 2

Below code from the explanation can help do this. I went similar question and answered it myself.

from tensorflow.keras.layers import MaxPool1D
import tensorflow.keras.backend as K
import numpy as np
import tensorflow as tf
from tensorflow.keras.layers import Conv1D 
tf.set_random_seed(1)

batch,rows,cols = 3,8,3
input_shape = (batch,rows,cols)

np.set_random_seed = 132
data = np.random.randint(low=1,high=6,size=input_shape,dtype='int32')
data = np.float32(data)
data = tf.constant(data)

print("Data:")
print(K.eval(data))
print()
print(f'm,n,k:{input_shape}')
from tensorflow.keras.layers import Conv1D

#############################
# Understandin filters and kernel_size
##############################
num_filters=5
kernel_size= 3

'''
Few Notes about Kernel_size:
1. max_kernel_size == max_rows
2. since Conv1D, we are creating 1D Matrix of 1's with kernel_size 
if kernel_size = 1, [[1,1,1..]]
if kernel_size = 2, [[1,1,1..][1,1,1,..]]
if kernel_size = 3, [[1,1,1..][1,1,1,..]]

I have chosen tf.keras.initializers.constant(1) to create a matrix of Ones.
Size of matrix is Kernel_Size

'''
y= Conv1D(filters=num_filters,kernel_size=kernel_size,
          kernel_initializer=tf.keras.initializers.constant(1), 
 #glorot_uniform(seed=12)
          input_shape=(k,n)
         )(data)
#########################
# Checking the out outcome
#########################
print(K.eval(y))
print(f' Resulting output_shape == (batch_size, num_rows-kernel_size+1,num_filters): {y.shape}')

# # Verification
K.eval(tf.math.reduce_sum(data,axis=(2,1), # Sum along axis=2, and then along 
 axis=1,keep_dims=True)



###########################################
# Understanding MaxPool and Strides in 
##########################################
pool = MaxPool1D(pool_size=3,strides=3)(y)
print(K.eval(pool))
print(f'Shape of Pool: {pool.shape}')