Keras LSTM --> CNN Dimension Error

Question

Here is the model:

model.add(LSTM(24, input_shape=(120,26), return_sequences = True, activation = 'relu'))
model.add(convolutional.Conv1D(filters = 1, kernel_size=120,activation ='relu'))
model.add(Dense(1))
model.add(Activation('sigmoid'))
opt1 = keras.optimizers.RMSprop(lr=0.0001, rho=0.9, epsilon=1e-08, decay=0.0)
model.compile(loss='binary_crossentropy', optimizer=opt1, metrics=['accuracy'])

filepath="model_check-{epoch:02d}-{loss:.4f}.hdf5"
checkpoint = ModelCheckpoint(filepath, monitor='loss', verbose=1, 
                             save_best_only=True, mode='min')
callbacks_list = [checkpoint]

model.fit(df_matrix3, y_matrix3, epochs=2000, batch_size=10000, verbose=2)

Here are the dimensions of the input:

df_matrix3 = [0]*10000
df_matrix3t= [0]*10000
for x in range(10000):
    df_matrix3[x]=df_matrix[:,0+x:120+x,:]
    df_matrix3t[x] = df_matrix[:,1740000+x:1740120+x,:]
df_matrix3 = np.reshape(df_matrix3,(10000,120,26))


y_matrix3 = y[0:10000]
y_matrix3t = y[1740000:1750000]
y_matrix3t = np.reshape(y_matrix3t,(10000,1))
y_matrix3 = np.reshape(y_matrix3,(-1,10000,1))
y_matrix3c = y[1740000:1750000]

Here is the error I'm getting:

ValueError: Error when checking target: expected activation_15 to have shape (None, 1, 1) but got array with shape (1, 10000, 1)

This error doesn't make a lot of sense to me because when I run the LSTM as a batch of 10,000 by itself it runs properly ( although return Sequences is set to False). All I've done since then is add one CNN layer set return sequences = True in the first layer, and the output should still be 1 item per input which I would expect to be a 1,10000,1 array. However it's looking for something else, and I'm not sure why.

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
lstm_15 (LSTM)               (None, 120, 24)           4896      
_________________________________________________________________
conv1d_14 (Conv1D)           (None, 1, 1)              2881      
_________________________________________________________________
dense_14 (Dense)             (None, 1, 1)              2         
_________________________________________________________________
activation_14 (Activation)   (None, 1, 1)              0         
=================================================================
Total params: 7,779
Trainable params: 7,779

Those are my layer sizes which coincide perfectly with what I would expect.

Daniel Möller Daniel Möller · Accepted Answer · 2017-09-09T02:23:34

Batch sizes are always the first dimension.

You should really expect (10000,1,1) instead.

10000 samples of (1,1).

That's (None,1,1) - the only free dimension is the first, which is the batch size.

Keras LSTM --> CNN Dimension Error

1 Answers