How to set up a LSTM network with only one input an time 0 for Sequence Generation?

Question

I set up a training set with X of shape (batch_size, 50) and Y of shape (batch_size, 10(sequence length), 10(output vector)).

The Keras Documentation of the LSTM cell says, that a 3D input is required, but Sequence 2 Sequence models work and do exactly this. Is there another common way to address this task?

model = Sequential()
model.add(LSTM(50, input_shape=(50,)))
model.add(TimeDistributed(Dense(10, activation='softmax')))

I get the error: "ValueError: Input 0 is incompatible with layer lstm_1: expected ndim=3, found ndim=2"

Answer 1

This return only the last step of the Sequence, you should set the argument: return_sequences=True

For the following model:

 model = Sequential()
 model.add(LSTM(50, input_shape=(1, 50), return_sequences=True))
 model.add(TimeDistributed(Dense(10, activation='softmax')))

I Have:

Layer (type)                 Output Shape              Param #   
=================================================================
lstm_3 (LSTM)                (None, 1, 50)             20200     
_________________________________________________________________
time_distributed_2 (TimeDist (None, 1, 10)             510       
=================================================================
Total params: 20,710
Trainable params: 20,710
Non-trainable params: 0
_________________________________________________________________

Answer 2

Changing the input_shape to (1, 50) and adding a

model.add(RepeatVector(10, input_shape=(50, )))

as first layer, fixes the problem. The additional RepeatVector makes the input visible to the network at all times step.

A good overview over Keras LSTM nets is: How to use return_sequences option and TimeDistributed layer in Keras?