LSTM Nerual Network Input/Output dimensions error

Question

I am fairly new to TensorFlow and LSTM architecture. I have an issue with figuring out input and output (x_train,x_test,y_train,y_test) for my dataset.

Shape of my inputs originally:

X_train: (366,4)
X_test: (104,4)
Y_train: (366,)
Y_test: (104,)

The Ytrain and Ytest are a series of stock prices. The Xtrain and Xtest are four features that I want to learn to predict the stock prices.

# Splitting the training and testing data

train_start_date = '2010-01-08'
train_end_date = '2017-01-06'
test_start_date = '2017-01-13'
test_end_date = '2019-01-04'

train = df.ix[train_start_date : train_end_date]
test = df.ix[test_start_date:test_end_date]


X_test = sentimentScorer(test)
X_train = sentimentScorer(train)

Y_test = test['prices'] 
Y_train = train['prices']

#Conversion in 3D array for LSTM INPUT

X_test = X_test.reshape(1, 104, 4)
X_train = X_train.reshape(1, 366, 4)





model = Sequential()

model.add(LSTM(128, input_shape=(366,4), activation='relu', 
return_sequences=True))
model.add(Dropout(0.2))

model.add(LSTM(128, activation='relu'))
model.add(Dropout(0.1))

model.add(Dense(32, activation='relu'))
model.add(Dropout(0.2))

model.add(Dense(10, activation='softmax'))

opt = tf.keras.optimizers.Adam(lr=0.001, decay=1e-6)

# Compile model
model.compile(
    loss='sparse_categorical_crossentropy',
    optimizer=opt,
    metrics=['accuracy'],
)

model.fit(X_train,
          Y_train,
          epochs=3,
          validation_data=(X_test, Y_test))

This is the error generated:

--------------------------------------------------------------------------- ValueError Traceback (most recent call last) in 65 Y_train, 66 epochs=3, ---> 67 validation_data=(X_test, Y_test))

c:\users\talal\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, **kwargs) 1507 steps_name='steps_per_epoch', 1508 steps=steps_per_epoch, -> 1509 validation_split=validation_split) 1510 1511 # Prepare validation data.

c:\users\talal\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\training.py in _standardize_user_data(self, x, y, sample_weight, class_weight, batch_size, check_steps, steps_name, steps, validation_split) 991 x, y = next_element 992 x, y, sample_weights = self._standardize_weights(x, y, sample_weight, --> 993 class_weight, batch_size) 994 return x, y, sample_weights 995

c:\users\talal\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\training.py in _standardize_weights(self, x, y, sample_weight, class_weight, batch_size) 1110 feed_input_shapes, 1111
check_batch_axis=False, # Don't enforce the batch size. -> 1112 exception_prefix='input') 1113 1114 if y is not None:

c:\users\talal\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\keras\engine\training_utils.py in standardize_input_data(data, names, shapes, check_batch_axis, exception_prefix) 314 ': expected ' + names[i] + ' to have ' + 315 str(len(shape)) + ' dimensions, but got array ' --> 316 'with shape ' + str(data_shape)) 317 if not check_batch_axis: 318 data_shape = data_shape[1:]

ValueError: Error when checking input: expected lstm_18_input to have 3 dimensions, but got array with shape (366, 4)

Szymon Maszke Szymon Maszke · Accepted Answer · 2019-01-11T01:51:58

Your code is almost fine.

Your y_test and y_train should be an array with one element or array of shape (1,1), it doesn't matter.

Your input shape is wrong though, first LSTM should be:

model.add(LSTM(128, input_shape=(None,4), activation='relu', return_sequences=True))

Notice None, as your test and train sequences length are different, you cannot specify it (and Keras accepts first dimension unspecified). Error was due to lengths of 366 and 104 respectively. If you want to use batches with RNNs you should perform zero-padding with keras.preprocessing.sequence.pad_sequences.

No need to specify input_shape with batch, rest of the network should be fine.

And if you are performing regression, not classification, as is the case probably, you should perform two last steps written by @Ankish Bansal, e.g. changing loss to mean squared error and making last layer output 1 value instead of 10.

LSTM Nerual Network Input/Output dimensions error

3 Answers