Simple Conv1D as first layer in keras

Question

Here is my input

x_train.shape # (12, 7) 12 observations each of length 7
x_train # dtype('int32')

Here's the architecture I'd like to achieve:

I'd like a kernel of size 3 convolved over the sequence. From keras documentation at https://keras.io/layers/convolutional/

"When using this layer as the first layer in a model, provide an input_shape argument (tuple of integers or None, e.g. (10, 128) for sequences of 10 vectors of 128-dimensional vectors, or (None, 128) for variable-length sequences of 128-dimensional vectors."

Honestly I'm having a hard time understanding their logic. Here's my attempt

docs_sequence = Input(shape=(7,), dtype='float32') # Longest document is 7 words
convolution = Conv1D(filters = 1,  # only 1 convolution
                     kernel_size = 3, # tri grams
                     strides = 1,
                     input_shape = (1, 7),
                     padding = 'valid',
                     activation = 'relu')(docs_sequence)
output = Dense(1, activation='sigmoid')(convolution)
cnn_model = Model(inputs = docs_sequence, outputs = [output])
cnn_model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

and I'm consistently getting

ValueError: Input 0 is incompatible with layer conv1d_30: expected ndim=3, found ndim=2

I've tried playing with the input shape, but again I'm having a hard time following their logic. — Andrew Cassidy
From the link you posted - Input shape 3D tensor with shape: (batch_size, steps, input_dim). This could help you orbifold.net/default/2017/01/10/… to tokenize your sentences — sladomic

Andre Holzner Andre Holzner · Accepted Answer · 2018-01-22T20:49:57

As the error message says, your input is two dimensional while the convolutional layer expects a three dimensional input.

With the following

docs_sequence = Input(shape=(7,1), ...

instead of

docs_sequence = Input(shape=(7,), ...

Keras accepts the model. Basically this adds a dimension of size one to the input (the three dimensions from the error message include the minibatch dimension which one can think of being prepended to the shape argument above).

cnn_model.summary() then gives:

_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
input_1 (InputLayer)         (None, 7, 1)              0
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 5, 1)              4
_________________________________________________________________
dense_1 (Dense)              (None, 5, 1)              2
=================================================================

When preparing the actual input data, you may have to add this dimension of size one to your input data. You may want to use numpy.atleast_2d() or numpy.atleast_3d() for this, possibly combined with taking the transpose or use numpy.expand_dims().

In your case, np.atleast_3d(x_train) gives a shape of (12, 7, 1).

Simple Conv1D as first layer in keras

1 Answers