Keras: Store sequence values in RNN without using as inputs in the next layer

Question

I want to interpret an RNN by looking at the sequence-by-sequence values. It is possible to output these values with return_sequences. However, those values are then used as inputs into the next layer (e.g., a dense activation layer). I would like to output only the last value but record all values over the full sequence for interpretation. What's the easiest way to do this?

Answer 1

Create two models with the same layer, but in one of them you feed the Dense layer only with the last step of the RNN:

inputs = Input(inputShape)
outs = RNN(..., return_sequences=True)(inputs)

modelSequence = Model(inputs,outs)

#take only the last step
outs = Lambda(lambda x: x[:,-1])(outs)
outs = Dense(...)(outs)

modelSingle = Model(inputs,outs)

Use modelSingle,fit(x_data,y_data) to train as you've been doing normally.
Use modelSequence.predict(x_data) to see the results of the RNN without training.