Extract Cell State LSTM Keras

Question

I was wondering if it would be possible to extract the last cell state of an LSTM in Keras after training the model. For example, in this simple LSTM model:

number_of_dimensions = 128
number_of_examples = 123456

input_ = Input(shape = (10,100,))
lstm, hidden, cell = CuDNNLSTM(units = number_of_dimensions, return_state=True)(input_)

dense = Dense(num_of_classes, activation='softmax')(lstm)

model = Model(inputs = input_, outputs = dense)
parallel_model = multi_gpu_model(model, gpus=2)
parallel_model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['acc'])

# fit the model
parallel_model.fit(X1, onehot_encoded, epochs=100, verbose=1, batch_size = 128, validation_split = 0.2)

I tried printing 'cell' but the result was

tf.Tensor 'cu_dnnlstm_2/strided_slice_17:0' shape=(?, 128) dtype=float32

I would like to get the cell state as a numpy array of shape (number_of_examples, number_of_dimensions) or (123456, 128). Is it possible to do this keras?

Thank you!

rvinas rvinas · Accepted Answer · 2018-07-06T09:54:12

Assuming that you are using TensorFlow as a backend, you could specifically run cell within the TensorFlow session. For example:

from keras.layers import LSTM, Input, Dense
from keras.models import Model
import keras.backend as K
import numpy as np

number_of_dimensions = 128
number_of_examples = 123456

input_ = Input(shape=(10, 100,))
lstm, hidden, cell = LSTM(units=number_of_dimensions, return_state=True)(input_)
dense = Dense(10, activation='softmax')(lstm)
model = Model(inputs=input_, outputs=dense)

with K.get_session() as sess:
    x = np.zeros((number_of_examples, 10, 100))
    cell_state = sess.run(cell, feed_dict={input_: x})
    print(cell_state.shape)

Extract Cell State LSTM Keras

2 Answers