How does tensorflow determine which LSTM units will be selected as outputs?

0
votes

I created a LSTM model using the below code:

model = tensorflow1.keras.Sequential()

model.add(tensorflow1.keras.layers.LSTM(128, input_shape=(720, 4), return_sequences=True)) model.add(tensorflow1.keras.layers.LeakyReLU(alpha=0.5)) model.add(tensorflow1.keras.layers.LSTM(128, return_sequences=True)) model.add(tensorflow1.keras.layers.LeakyReLU(alpha=0.5)) model.add(tensorflow1.keras.layers.Dropout(0.3)) model.add(tensorflow1.keras.layers.LSTM(64, return_sequences=False)) model.add(tensorflow1.keras.layers.Dropout(0.3)) model.add(tensorflow1.keras.layers.Dense(1))

For the code model.add(tf.keras.layers.LSTM(128, input_shape=(720,4), return_sequences=True)), in my understanding, the code will have 128 LSTM output units, with an input shape of 720 time steps and 4 features. According to tensorflow keras documentation, units refers to "dimensionality of the output space". (https://www.tensorflow.org/api_docs/python/tf/keras/layers/LSTM)

My question is: Which of the 128 cells as output units out of the 720 cells will be selected as output cells? How does tensorflow choose which cells to select as an output?

1
pythontensorflowkeraslstm
Can you give the full model code? Because to understand your question need some more details. But I think LSTM can return you the whole sequence of hidden states. Check this blog from machine learning mastery. I think it solves your doubt. – Dark debo
This is a common misconception, there are only 128 cells, not 720. Each cell is fully connected with the input. So there are only 128 outputs, not 720. TensorFlow does not select anything here. – Dr. Snoopy

1 Answers

1
votes

The first layer is composed by 128 LSTM cells.

Each cell will give an output that will be provided as an input for the subsequent layer.

Since you selected (correctly) "return_sequences=True", each LSTM cell will provide an output value per time step due to sequence unrolling.

So, for the first LSTM layer, you will have 128 output values (one per cell) for each of the 720 time steps.

Maybe this can be useful to understand: https://www.tensorflow.org/guide/keras/rnn