Combining 2D CNN with GRU in Keras

Question

I would like to build this type of neural network architecture: 2DCNN+GRU. Consider that the input is a 4D-tensor (batch_size, 1, 1500, 40), then I've got 3 2D-CNN layers (with batch norm, relu, max pooling and dropout). In output from the third cnn layer I obtain a 4D-tensor (None, 120, 1500, 1). Now it comes my issue, how do I connect the GRU layer with this input shape? I tried to do a reshape in keras (so it becomes (None, 1500, 120)) and feed the output through a gru layer but there's something wrong... Consider, also, that my labels for training is a 3D-tensor (batch_size, 1500, 2). I copy here the keras model and the output from the summary() command:

    input_data = Input(shape=[1,1500,40])
    x = input_data
    for i in range(len([32,96,120])):
        x = Conv2D(filters=[32,96,120],
                   kernel_size=[5,5],
                   activation='relu',
                   padding='same'
                   )(x)
        x = BatchNormalization(axis=3)(x)
        x = Dropout(0.3)(x)
        x = MaxPooling2D(pool_size=[(1,5),(1,4),(1,2)],
                         data_format="channels_first")(x)

    x = Reshape((1500, 120))(x)

    x = GRU(units=120,
            activation='tanh',
            recurrent_activation='hard_sigmoid',
            dropout=0.3,
            recurrent_dropout=0.3,
            )(x)

    predictions = Dense(2, activation='softmax')(x)
    network = Model(input_data, predictions)
    network.summary()

Network Summary

Can you help me? Thank you

What do you mean by "something is wrong"? Do you get an error message? Does it not work as expected? — sietschie
I'm sorry I didn't explain that...I mean that in the output of gru layer I get a 2D-tensor (batch_size, 120) and then after the dense (batch_size, 2) , whereas as input labels Y I've got (batch_size, 1500, 2) tensor, so I can't train the network because of this error: Error when checking target: expected dense_1 to have 2 dimensions, but got array with shape (491, 1500, 2) — SilverMatt

sietschie sietschie · Accepted Answer · 2018-02-14T13:45:49

It seems like you expect a prediction for every timestep of your input. To get this you need to add the parameter return_sequences set to True when you create the GRU layer.

Combining 2D CNN with GRU in Keras

1 Answers