Keras - Reuse weights from a previous layer - converting to keras tensor

Question

I am trying to reuse the weight matrix from a previous layer. As a toy example I want to do something like this:

import numpy as np
from keras.layers import Dense, Input
from keras.layers import merge
from keras import backend as K
from keras.models import Model

inputs = Input(shape=(4,))
inputs2 = Input(shape=(4,))
dense_layer = Dense(10, input_shape=(4,))
dense1 = dense_layer(inputs)

def my_fun(my_inputs):
    w = my_inputs[0]
    x = my_inputs[1]

    return K.dot(w, x)

merge1 = merge([dense_layer.W, inputs2], mode=my_fun)

The problem is that dense_layer.W is not a keras tensor. So I get the following error:

Exception: Output tensors to a Model must be Keras tensors. Found: dot.0

Any idea on how to convert dense_layer.W to a Keras tensor?

Thanks

Answer 1

It seems that you want to share weights between layers. I think You can use denselayer as shared layer for inputs and inputs2.

merge1=dense_layer(inputs2)

Do check out shared layers @ https://keras.io/getting-started/functional-api-guide/#shared-layers

Answer 2

I don't think that you can use the merge layer like this.

But to answer your question, you will probably have to create a custom layer which has tied weights. Look at this example.

Otherwise, the way to access the weights of a layer is to use get_weights() method on that layer, this will retrun a list of numpy arrays containing the weights. For the case of the Dense layer, it will contain weights and bias.

Answer 3

There are two cases for the solution, depending on what you are trying to do:

1. You would like to share the W matrix between your two operations, and the W matrix for these two operations are kept the same even if its value changed during training or for some other reason. Then you should use dense.weights[0] which is the W matrix as a tensor from your dense layer.

2. If you are only going to use the value of W matrix at the time of your code is written and this value is never going to change, then use K.constant(dense.get_weights[0]) which extracts the weights as numpy array and is converted into tensor.