keras Concatenate mulitple layers cause AttributeError: 'NoneType' object has no attribute '_inbound_nodes'

Question

I am trying to add some fixed kernels in my CNN, please see my codes below.

This is how I create my kernels:

# Kernels
def create_kernel(x):
    t = pipe(
        x, 
        lambda x: tf.constant(x, dtype=tf.float32), 
        lambda x: tf.reshape(x, [3, 3, 1, 1]))
    return t

k_edge1 = create_kernel([1, 0, -1, 0, 0, 0, -1, 0, 1])
k_edge2 = create_kernel([0, 1, 0, 1, -4, 1, 0, 1, 0])
k_edge3 = create_kernel([-1, -1, -1, -1, 8, -1, -1, -1, -1])

and my convolution network is like:

# Convolution network
# Input layer
l_input = Input(shape=(28**2, ))
# Reshape layer
l_reshape = Reshape(target_shape=(28, 28, 1))(l_input)
# Convolution layers
l_conv1 = Conv2D(filters=20, kernel_size=(3, 3), padding='valid')(l_reshape)
l_edge1 = tf.nn.conv2d(l_reshape, k_edge1, strides=[1, 1, 1, 1], padding='VALID')
l_edge2 = tf.nn.conv2d(l_reshape, k_edge2, strides=[1, 1, 1, 1], padding='VALID')
l_edge3 = tf.nn.conv2d(l_reshape, k_edge3, strides=[1, 1, 1, 1], padding='VALID')
l_conv1a = Concatenate(axis=3)([l_conv1, l_edge1, l_edge2, l_edge3])  # <- The error should be caused by this line.
l_conv2 = Conv2D(filters=20, kernel_size=(3, 3), padding='valid')(l_conv1a)
l_pool1 = MaxPooling2D(pool_size=(2, 2), border_mode='valid')(l_conv2)
# Flatten layer
l_flat = Flatten()(l_pool1)
# Fully connected layers
l_fc1 = Dense(50, kernel_initializer='he_normal')(l_flat)
l_act1 = PReLU()(l_fc1)
l_fc3 = Dense(10, kernel_initializer='he_normal')(l_act1)
l_output = Activation('softmax')(l_fc1)

# Model
cnn_model = Model(l_input, l_output)

However, I got the following error:

Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "C:\Users\Perry Cheng\AppData\Local\conda\conda\envs\ml_py_3_6\lib\site-packages\keras\legacy\interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
File "C:\Users\Perry Cheng\AppData\Local\conda\conda\envs\ml_py_3_6\lib\site-packages\keras\engine\network.py", line 93, in __init__
    self._init_graph_network(*args, **kwargs)
File "C:\Users\Perry Cheng\AppData\Local\conda\conda\envs\ml_py_3_6\lib\site-packages\keras\engine\network.py", line 237, in _init_graph_network
    self.inputs, self.outputs)
File "C:\Users\Perry Cheng\AppData\Local\conda\conda\envs\ml_py_3_6\lib\site-packages\keras\engine\network.py", line 1353, in _map_graph_network
    tensor_index=tensor_index)
File "C:\Users\Perry Cheng\AppData\Local\conda\conda\envs\ml_py_3_6\lib\site-packages\keras\engine\network.py", line 1340, in build_map
    node_index, tensor_index)
File "C:\Users\Perry Cheng\AppData\Local\conda\conda\envs\ml_py_3_6\lib\site-packages\keras\engine\network.py", line 1340, in build_map
    node_index, tensor_index)
File "C:\Users\Perry Cheng\AppData\Local\conda\conda\envs\ml_py_3_6\lib\site-packages\keras\engine\network.py", line 1340, in build_map
    node_index, tensor_index)
[Previous line repeated 2 more times]
File "C:\Users\Perry Cheng\AppData\Local\conda\conda\envs\ml_py_3_6\lib\site-packages\keras\engine\network.py", line 1312, in build_map
    node = layer._inbound_nodes[node_index]
AttributeError: 'NoneType' object has no attribute '_inbound_nodes'

After some testing, I think the error comes from:

l_conv1a = Concatenate(axis=3)([l_conv1, l_edge1, l_edge2, l_edge3])

Is there any way to solve it?

Answer 1

Keras layers accepts Keras Tensors and not Tensors as their input. So if you would like to use tf.nn.conv2d instead of Conv2D layers in Keras, you need to wrap them inside a Lambda layer:

l_edge1 = Lambda(lambda x: tf.nn.conv2d(x, k_edge1, strides=[1, 1, 1, 1], padding='VALID'))(l_reshape)
l_edge2 = Lambda(lambda x: tf.nn.conv2d(x, k_edge2, strides=[1, 1, 1, 1], padding='VALID'))(l_reshape)
l_edge3 = Lambda(lambda x: tf.nn.conv2d(x, k_edge3, strides=[1, 1, 1, 1], padding='VALID'))(l_reshape)

Answer 2

You cannot use TF functions directly on Keras tensors as you are doing here:

l_edge1 = tf.nn.conv2d(l_reshape, k_edge1, strides=[1, 1, 1, 1], padding='VALID')
l_edge2 = tf.nn.conv2d(l_reshape, k_edge2, strides=[1, 1, 1, 1], padding='VALID')
l_edge3 = tf.nn.conv2d(l_reshape, k_edge3, strides=[1, 1, 1, 1], padding='VALID')

What you should do is to just use the Conv2D layer and then set the weights manually using layer.set_weights(array). To keep weights non-trainable, just set layer.trainable = False, like:

conv = Conv2D(filters=1, kernel_size(3, 3), padding='valid')
conv.set_weights(your_weight_array)
conv.trainable = False

l_edge1 = conv(l_reshape)

And similarly for the other two Conv2D layers.