Feed tensorflow or keras neural nets input with custom dimensions

Question

I would like to feed a neural net inputs of following shape: Each training entry is a 2D array with dimensions 700x10. There are in total 204 training entries. Labels is just 1-dimensional array of size 204 (binary output)

I tried to just use Dense layers:

model = Sequential()
model.add(Dense(300, activation='relu', input_shape=(700, 10)))
model.add(Dense(1, activation='sigmoid'))

But then I am getting following error (not related to input_shape on the first layer, but during validation of output):

ValueError: Error when checking target: expected dense_2 to have 3 dimensions, but got array with shape (204, 1)

204 - amount of training data.

Stacktrace:

    model.fit(xTrain, yTrain, epochs=4, batch_size=6)
  File "keras\models.py", line 867, in fit
    initial_epoch=initial_epoch)
  File "keras\engine\training.py", line 1522, in fit
    batch_size=batch_size)
  File "keras\engine\training.py", line 1382, in _standardize_user_data
    exception_prefix='target')
  File "keras\engine\training.py", line 132, in _standardize_input_data

What I found out while debugging Keras code:

It fails during validation before training. It validates output array.

According to the neural network structure, first Dense layer produces somehow 700, 1 dimensional output and it fails afterwards, since my output is just 1-d array with 204 in it.

How do I overcome this issue? I tried to add Flatten() after Dense() layer, but it probably affects accuracy in a bad way: I would like to keep information specific to one point from 700 array grouped.

Daniel Möller Daniel Möller · Accepted Answer · 2017-11-14T18:54:59

The Dense layers works on only one dimension, the last.

If you're inputting (700,10) to it, it will output (700,units). Check your model.summary() to see this.

A simple solution is to flatten your data before applying dense:

model.add(Flatten(input_shape=(700,10)))
model.add(Dense(300,...))
model.add(Dense(1,...))

This way, the Dense layer will see a simple (7000,) input.

Now if you do want your model to understand those 2 dimensions separately, you should perhaps try more elaborated structures. What to do will depend a lot on what your data is and what you want to do, how you want your model to understand it, etc.

Feed tensorflow or keras neural nets input with custom dimensions

1 Answers