Adding new units to a Keras model layer and changing their weights

Question

I am working on a project that requires me to add new units to the output layer of a neural network to implement a form of transfer learning. I was wondering if I could do this and set the units' weights using either Keras or TensorFlow.

Specifically I would like to append an output neuron to the output layer of the Keras model and set that neuron's initial weights and bias.

Are you wanting to append to the output vector and then use this an input to a new NN? — djd
Yes, I agree with Dom. Could you please be more specific. Thank you. — ReInvent_IO

rsyed0 rsyed0 · Accepted Answer · 2018-07-20T18:01:37

Stumbled upon the answer to my own question. Thanks everyone for the answers/comments.

https://keras.io/layers/about-keras-layers/

The first few lines of this source detail how to load and set weights. Essentially, appending an output neuron to a Keras model can be accomplished by loading the old output layer, appending the new weights, and setting weights for a new layer. Code is below.

# Load weights of previous output layer, set weights for new layer
old_layer_weights = model.layers.pop().get_weights()
new_neuron_weights = np.ndarray(shape=[1,bottleneck_size])

# Set new weights

# Append new weights, add new layer
new_layer = Dense(num_classes).set_weights(np.append(old_layer_weights,new_neuron_weights))
model.add(new_layer)

Adding new units to a Keras model layer and changing their weights

2 Answers