keras: ValueError: Error when checking model target: expected activation_1 to have shape (None, 60) but got array with shape (10, 100)

Question

I'm trying to transplant RocAlphaGo to play Game of Amazons, and there are problems when trying to implement supervised policy trainer.

from keras.models import Sequential, Model
from keras.layers.core import Activation, Flatten
from keras.layers import convolutional

defaults = {
            "board": 10,
            "filters_per_layer": 128,
            "layers": 12,
            "filter_width_1": 5
}
# copy defaults, but override with anything in kwargs
params = defaults
network = Sequential()
# create first layer
network.add(convolutional.Convolution2D(
      input_shape=(6, 10, 10),
      nb_filter=128,
      nb_row=5,
      nb_col=5,
      init='uniform',
      activation='relu',
      border_mode='same'))

# create all other layers
for i in range(2, 13):
     # use filter_width_K if it is there, otherwise use 3
     filter_key = "filter_width_%d" % i
     filter_width = params.get(filter_key, 3)

     # use filters_per_layer_K if it is there, otherwise use default value
     filter_count_key = "filters_per_layer_%d" % i
     filter_nb = params.get(filter_count_key, 128)

     network.add(convolutional.Convolution2D(
         nb_filter=filter_nb,
         nb_row=filter_width,
         nb_col=filter_width,
         init='uniform',
         activation='relu',
         border_mode='same'))

# the last layer maps each <filters_per_layer> feature to a number
network.add(convolutional.Convolution2D(
    nb_filter=1,
    nb_row=1,
    nb_col=1,
    init='uniform',
    border_mode='same'))
# reshape output to be board x board
network.add(Flatten())
# softmax makes it into a probability distribution
network.add(Activation('softmax'))

• keras 1.2.0
• python 2.7

Gives the following exception:

ValueError: Error when checking model target: expected activation_1 to have shape (None, 60) but got array with shape (10, 100)

The training data set is a (10, 6, 10, 10) array, 10 x 6 layers , each layer is a 10x10 array(chess board), why model need (None, 60) ? if chagne input_shape=(6, 10, 10) to input_shape=(10, 10, 10), will get:

ValueError: Error when checking model input: expected convolution2d_input_1 to have shape (None, 10, 10, 10) but got array with shape (10, 6, 10, 10)

All the code is here

Answer 1

As Matias is saying in the comments, if you add

network.summary() 

you will probably notice that your convolutions are applied on the first 2 dimensions of your input data (which is (6,10,10)). Your features here are in the first dimension. Keras, by default when you use tensorflow, will assume that the dimension where your features will stand is the third one, not the first. For that reason, when you apply 128 filters on a (6,10,10) array, the output will be (6,10,128) which is not what you want if I understand well.

For that reason, at the output of the last convolution layer , you get a (6,10,1) array, which flattens to (,60) and not (,100) as you were expecting.

There are two ways to fix your network. Either you change your input data to the format (10,10,6). Or you use the data_format="channels_first" parameter of the Convolution2D() layer.

Here is the code for the second option:

from keras.models import Sequential, Model
from keras.layers.core import Activation, Flatten
from keras.layers import convolutional

defaults = {
            "board": 10,
            "filters_per_layer": 128,
            "layers": 12,
            "filter_width_1": 5
}
# copy defaults, but override with anything in kwargs
params = defaults
network = Sequential()
# create first layer
network.add(convolutional.Convolution2D(
      input_shape=(6, 10, 10),
      nb_filter=128,
      nb_row=5,
      nb_col=5,
      init='uniform',
      activation='relu',
      border_mode='same',
      data_format='channels_first'
      ))

# create all other layers
for i in range(2, 13):
     # use filter_width_K if it is there, otherwise use 3
     filter_key = "filter_width_%d" % i
     filter_width = params.get(filter_key, 3)

     # use filters_per_layer_K if it is there, otherwise use default value
     filter_count_key = "filters_per_layer_%d" % i
     filter_nb = params.get(filter_count_key, 128)

     network.add(convolutional.Convolution2D(
         nb_filter=filter_nb,
         nb_row=filter_width,
         nb_col=filter_width,
         init='uniform',
         activation='relu',
         border_mode='same',
         data_format='channels_first'))

# the last layer maps each <filters_per_layer> feature to a number
network.add(convolutional.Convolution2D(
    nb_filter=1,
    nb_row=1,
    nb_col=1,
    init='uniform',
    border_mode='same',
    data_format='channels_first'))
# reshape output to be board x board
network.add(Flatten())
# softmax makes it into a probability distribution
network.add(Activation('softmax'))
# display your network summary
network.summary()

EDIT

Taking into account your version of keras, you should use the parameter "dim_ordering" and set it to "th".

I have found this information in the keras documentation

from keras.models import Sequential, Model
from keras.layers.core import Activation, Flatten
from keras.layers import convolutional

defaults = {
            "board": 10,
            "filters_per_layer": 128,
            "layers": 12,
            "filter_width_1": 5
}
# copy defaults, but override with anything in kwargs
params = defaults
network = Sequential()
# create first layer
network.add(convolutional.Convolution2D(
      input_shape=(6, 10, 10),
      nb_filter=128,
      nb_row=5,
      nb_col=5,
      init='uniform',
      activation='relu',
      border_mode='same',
      dim_ordering='th'
      ))

# create all other layers
for i in range(2, 13):
     # use filter_width_K if it is there, otherwise use 3
     filter_key = "filter_width_%d" % i
     filter_width = params.get(filter_key, 3)

     # use filters_per_layer_K if it is there, otherwise use default value
     filter_count_key = "filters_per_layer_%d" % i
     filter_nb = params.get(filter_count_key, 128)

     network.add(convolutional.Convolution2D(
         nb_filter=filter_nb,
         nb_row=filter_width,
         nb_col=filter_width,
         init='uniform',
         activation='relu',
         border_mode='same',
         dim_ordering='th'))

# the last layer maps each <filters_per_layer> feature to a number
network.add(convolutional.Convolution2D(
    nb_filter=1,
    nb_row=1,
    nb_col=1,
    init='uniform',
    border_mode='same',
    dim_ordering='th'))
# reshape output to be board x board
network.add(Flatten())
# softmax makes it into a probability distribution
network.add(Activation('softmax'))
# display your network summary
network.summary()