"Invalid shape for y" for Keras LSTM w/ return_sequences=True (and sklearn API)

Question

I have a sequence I am trying to classify, using a Keras LSTM with return_sequences=True. I have 'data' and 'labels' datasets both of which are the same shape - 2D matrices with rows by location and columns by time interval (cell values are my 'signal' feature). So an RNN w/ return_sequences=True seems like an intuitive approach.

After reshaping my data (X) and labels (Y) to 3D tensors of shape (rows, cols, 1), I call model.fit(X, Y) but get the following error:

ValueError('Invalid shape for y')

It points me to the code for class KerasClassifier()'s fit method which checks that len(y.shape)==2.

Ok so maybe I was supposed to reshape my 2D 'X' to a 3D Tensor of shape (rows, cols, 1) but leave my labels as 2D for sklearn interface? But then when I try that I get another Keras error:

ValueError: Error when checking model target: expected lstm_17 to have 3 dimensions, but got array with shape (500, 2880)

...So how does one fit a Sklearn-style Keras RNN to return sequences? Different parts of Keras seem to demand that my target be both 2D and 3D. Or (more likely) I'm misunderstanding something.

... Here's a reproduceable code example:

from keras.layers import LSTM
from keras.wrappers.scikit_learn import KerasClassifier

# Raw Data/Targets    
X = np.array([1,2,3,4,5,6,7,8,9,10,11,12]).reshape(3,4)
Y = np.array([1,0,1,1,0,1,0,1,0,1,0,1]).reshape(3,4)

# Convert X to 3D tensor per Keras doc for recurrent layers
X = X.reshape(X.shape[0], X.shape[1], 1)

# .fit() at bottom will throw an error whether or not this line is used to reshape Y
to reshape Y
Y = Y.reshape(Y.shape[0], Y.shape[1], 1)


# Define function to return compiled Keras Model (to pass to Sklearn API)
def keras_rnn(timesteps, num_features):
    '''Function to return compiled Keras Classifier to pass to sklearn wrapper'''

    model = Sequential()
    model.add(LSTM(8, return_sequences=True, input_shape=(timesteps, num_features)))
    model.add(LSTM(1, return_sequences=True, activation = 'sigmoid'))

    model.compile(optimizer = 'RMSprop', loss = 'categorical_crossentropy')
    return model

# Convert compiled Keras model to Scikit-learn-style classifier (compatible w/ sklearn model-tuning methods)
rnn_sklearn = KerasClassifier(build_fn=keras_rnn, 
                        timesteps=4,
                        num_features=1) 

# Fit RNN Model to Data, Target                            
rnn_sklearn.fit(X, Y)

ValueError: Invalid shape for y

When I switch to binary_crossentropy in your code example, I have no error (Keras 1.2.2) — Merwann Selmani
Thanks for the response Merwan. Unfortunately, I'm still getting the same error "invalid shape for y" after switching model.compile's loss to binary_crossentropy. I have the latest Keras (2.0.2) — Max Power

Javier C. Javier C. · Accepted Answer · 2017-03-31T21:13:47

This code work with Keras 2.0.2:

import numpy as np
from keras.models import Sequential
from keras.layers import LSTM, Flatten
from keras.wrappers.scikit_learn import KerasClassifier

# Raw Data/Targets    
X = np.array([1,2,3,4,5,6,7,8,9,10,11,12]).reshape(3,4)
Y = np.array([1,0,1,1,0,1,0,1,0,1,0,1]).reshape(3,4)

# Convert X to 3D tensor per Keras doc for recurrent layers
X = X.reshape(X.shape[0], X.shape[1], 1)

# .fit() at bottom will throw an error whether or not this line is used to reshape Y to reshape Y
Y = Y.reshape(Y.shape[0], Y.shape[1], 1)


# Define function to return compiled Keras Model (to pass to Sklearn API)
def keras_rnn(timesteps, num_features):
    '''Function to return compiled Keras Classifier to pass to sklearn wrapper'''

    model = Sequential()
    model.add(LSTM(8, return_sequences=True, input_shape=(timesteps, num_features)))
    model.add(LSTM(1, return_sequences=True, activation = 'sigmoid'))

    model.compile(optimizer = 'RMSprop', loss = 'binary_crossentropy')
    return model

# Convert compiled Keras model to Scikit-learn-style classifier (compatible w/ sklearn model-tuning methods)
rnn_sklearn = KerasClassifier(build_fn=keras_rnn, 
                        timesteps=4,
                        num_features=1) 

# Fit RNN Model to Data, Target                            
rnn_sklearn.fit(X, Y)

"Invalid shape for y" for Keras LSTM w/ return_sequences=True (and sklearn API)

2 Answers