LSTM Keras neural network representation

Question

I've came across the following code that generates an Sequential LSTM neural network with Keras and I wounder how would it be represented in therms of drawing.

model = Sequential()
model.add(LSTM(60, input_shape=(1, 6),
               activation='softmax'))
model.add(Dense(2))

If someone could explain it to me, that would be great!

Edit: I'm looking for something like in this image: https://adventuresinmachinelearning.com/wp-content/uploads/2018/02/Keras-LSTM-tutorial-architecture.png

More specific: How would the input/output of the LSTM layer would look like and how would they be linked to the dense layer?

@Edit 2: I've made the following image: https://pasteboard.co/IEgqiVp.png Would the neural network look like what I draw in the image, if I add the extra synaptic connections made by each input with each of the 60 neurons?

ad2004 ad2004 · Accepted Answer · 2019-10-29T18:59:13

Here is some code that runs in Google Colab, that can give you some information about the input and outputs:

from keras import Sequential
from keras.layers import LSTM, Dense

model = Sequential()
model.add(LSTM(60, input_shape=(1, 6),
               activation='softmax'))
model.add(Dense(2))

# write out model architecture
model.summary()

# Graphical plot
from IPython.display import SVG
from keras.utils import model_to_dot

iv1_dot = model_to_dot(model, show_shapes=True, show_layer_names=True, dpi=60)
SVG(iv1_dot.create(prog=['dot'], format='svg'))

In regard to the specific design, the LSTM layer has 60 nodes which are densely connected to the 2 final nodes. That means that each of the 60 LSTM's are directly connected (with appropriate weights) to both of the final Dense nodes. On the input side, you have defined a single time step with 6 features to the LSTM layer via the input_shape=(1,6). So, the full input to the LSTM layer would look like:

(num_samples, num_timesteps, num_features)

where in this case, num_timesteps=1, num_features=6. I believe, each feature then connects to each LSTM node. Usually in keras, the num_samples is not directly specified in the network and appears as "None" in the architecture diagrams and summaries.

I hope this helps.

LSTM Keras neural network representation

1 Answers