Dense layer weights shape

2
votes

In a model with the following partial architecture: Model architecture in Keras

where we have let's say Dense1 and Dense2 layers (Fully connected).

I need to get the weights of my already trained model for Dense2, but the shape I get for this layer is (128,2048) which confuses me as the theoretical explanation of this layer is that it should provide a a single value for my array (here: 2048). I print the weights of Dense2 as:

    weights  =  model.layers[layer_idx].get_weights()

Shouldn't I see the shape of the architecture? (None, 2048)

2
pythonneural-networkdeep-learningkeras
The layer will produce as many outputs as there are units in it. If your dense layer has units=2048 it will output 2048 numbers. - Aechlys
@Aechlys that's what confuses me. I created a filter with 2048 units. Why it get the compination of the units of the previous FC layer and the latter? - Lara Larsen
Maybe I misunderstood the problem. Let's start again. You expected the weights to be just 2048 values? And instead you got 128x2048 values. Am I understanding the problem correctly, now? - Aechlys

2 Answers

3
votes

Here in the 2nd dense layer has 2048 neuron and each and every neuron are connected with previous layer output. So for single neuron there will be 128 previous layer neurons contributing.

y = w1*x1 + w2*x2 + .......... + w128*x128

so total (128*2048) = 262144 parameters with 2048 bias vectors totally 264192 (262144 + 2048) parameters.

Thanks

1
votes

Not each neuron has a weight, but each connection of neurons. And if you fully connect 128 neurons with 2048 you get 128 x 2048 weights.

That's why you have a shape of (128, 2048).