Neural network dimension mis-match

Question

I have a neural network setup for the MNIST digits dataset in Keras that looks like this:

input_size = features_train.shape[1]
hidden_size = 200
output_size = 9
lambda_reg = 0.2
learning_rate = 0.01
num_epochs = 50
batch_size = 30

model = Sequential()
model.add(Dense(input_size, hidden_size, W_regularizer=l2(lambda_reg), init='normal'))
model.add(Activation('tanh'))
model.add(Dropout(0.5))

model.add(Dense(hidden_size, output_size, W_regularizer=l2(lambda_reg), init='normal'))
model.add(Activation('softmax'))

sgd = SGD(lr=learning_rate, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='mean_squared_error', optimizer=sgd)

history = History()

model.fit(features_train, labels_train, batch_size=batch_size, nb_epoch=num_epochs, show_accuracy=True, verbose=2, validation_split=0.2, callbacks=[history])
score = model.evaluate(features_train, labels_train, show_accuracy=True, verbose=1)
predictions = model.predict(features_train)
print('Test score:', score[0])
print('Test accuracy:', score[1])

features_train is of shape (1000,784), labels_train is (1000,1), and both are numpy arrays. I want 784 input nodes, 200 hidden, and 9 output to classify the digits

I keep getting an input dimension mismatch error:

Input dimension mis-match. (input[0].shape[1] = 9, input[1].shape[1] = 1)
Apply node that caused the error: Elemwise{Sub}[(0, 0)](AdvancedSubtensor1.0, AdvancedSubtensor1.0)
Inputs types: [TensorType(float32, matrix), TensorType(float32, matrix)]
Inputs shapes: [(30L, 9L), (30L, 1L)]
Inputs strides: [(36L, 4L), (4L, 4L)]
Inputs values: ['not shown', 'not shown']

I'm trying to identify where my dimensions may be incorrect but I'm not seeing it. Can anyone see the problem?

Simon Simon · Accepted Answer · 2015-09-04T02:50:28

I've been training 2 class classification models for so long that I'm used to dealing with labels that are just single values. For this problem (classifying more than 1 outcome) I just had to change the labels to be vectors themselves.

This solved my problem:

from keras.utils.np_utils import to_categorical

labels_train = to_categorical(labels_train)

Neural network dimension mis-match

1 Answers