How can I build an LSTM network for two dimensional data?

0
votes

I see that LSTM in Keras accepts (batch_size, timesteps, data_dim) as the input shape. But I am wondering if there is an easy way to expand these to 4 dimensions? For example, how can we apply lstm to the moving-MNIST dataset that we have images of size 64 by 64?

1
tensorflowkeraslstmrecurrent-neural-network

1 Answers

1
votes

You can use a CNN model to extract features of your images before feeding them into your LSTM. 

# define CNN model
cnn = Sequential()
cnn.add(Conv2D(...))
cnn.add(MaxPooling2D(...))
cnn.add(Flatten())

# define LSTM model
model = Sequential()
model.add(TimeDistributed(cnn, ...))
model.add(LSTM(..))
model.add(Dense(...))