Keras CNN too few parameters

1
votes

I am trying to recreate the following tutorial CNN with 3 inputs and sigmoid activation functions in keras:

enter image description here

So the number of parameters should be 7 (assuming 1 filter of size 2 convolved over 2 locations (either top 2 inputs or 2 lower inputs), 2 shared weights (shown as 1.0's on the synapses) and no padding in the conv1d layer). When I write the following in Keras:

enter image description here

I only get 5 parameters when I check it in model.summary():

enter image description here

What do I need to do to get the correct number of parameters? There are probably several things that are wrong in my code since I'm new to Keras.

1
machine-learningneural-networkkeraskeras-layerconvolutional-neural-network
Please don't post code as pictures. – Maxim

1 Answers

2
votes

All convolutional parameters are shared spatially (in case of 1D this means across the input sequence). Precisely, the convolutional filter of length 2 is applied twice to inputs (x[0], x[1]) and (x[1], x[2]), but it's the same filter in both cases and correspondingly the trainable parameters are the same too.

This explains the size of the model you are getting right now: Conv1D has 3 parameters (weight (2) and bias (1)), the dense layer has 2 parameters because the output of Conv1D is (?, 2, 1).

Finally, I can't comment on the network you're trying to implement. Probably they mean 2 filters (but then the layer will have 6 parameters)... But I'm not aware of any implementation, in which the convolutional layer has separate parameters for each patch.