Make a custom loss function for mean intersection of union for regression in bounding boxes

Question

I am trying to iterate over the batch one by one to calculate the mean intersection over union. but fit function showing this

Error: An operation has None for the gradient. Please make sure that all of your ops have a gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.

Help as I am new to keras

#y_true shape: (None, 4)
import keras.backend as K
def iou(y_true, y_pred):
    # determine the (x, y)-coordinates of the intersection rectangle
    iou = 0
    for i in range(K.int_shape(y_pred)[0]):
        boxA = y_pred[i]
        boxB = y_true[i]
        xA = K.max(boxA[0], boxB[0])
        yA = K.max(boxA[2], boxB[2])
        xB = K.min(boxA[1], boxB[1])
        yB = K.min(boxA[3], boxB[3])

        interArea = K.max(0, xB - xA + 1) * K.max(0, yB - yA + 1)

        boxAArea = (boxA[1] - boxA[0] + 1) * (boxA[3] - boxA[2] + 1)
        boxBArea = (boxB[1] - boxB[0] + 1) * (boxB[3] - boxB[2] + 1)

        iou += interArea / float(boxAArea + boxBArea - interArea)
    #MEAN
    mean = iou/K.int_shape(y_pred)[0]
    return 1-mean

model.compile(optimizer='adam', loss=iou, metrics=['accuracy'])
model.fit(x_train, y_train, epochs=20, batch_size = 50)

my model works fine with mean squared error as loss function. Model:

input_shape = (180, 240, 3)
model = Sequential([
    Conv2D(32, (3, 3), input_shape=input_shape, padding='same',activation='relu'),
    MaxPooling2D(pool_size=(2, 2), strides=(2, 2)),
    BatchNormalization(),

    Conv2D(64, (3, 3), activation='relu', padding='same'),
    MaxPooling2D(pool_size=(2, 2), strides=(2, 2)),
    BatchNormalization(),

    Conv2D(128, (3, 3), activation='relu', padding='same',),

    Conv2D(256, (3, 3), activation='relu', padding='same',),
    MaxPooling2D(pool_size=(2, 2), strides=(2, 2)),

    Conv2D(128, (3, 3), activation='relu', padding='same',),
    MaxPooling2D(pool_size=(2, 2), strides=(2, 2)),
    BatchNormalization(),
    Flatten(),
    Dense(4096, activation='relu'),
    Dense(4096, activation='relu'),
    Dense(4, activation='relu')
])

Where is your model definition, what is loss=iou (it does not exist in Keras) and where exactly do you use the function you have defined? — desertnaut

Michael Glazunov Michael Glazunov · Accepted Answer · 2019-02-24T22:52:39

It means that all operations inside your custom loss function should be differentiable since otherwise the optimization procedure cannot be executed. To that end, you just need to check one by one which operation is a culprit in your code and substitute it with a Keras differentiable backend analogue or to find some other alternative.

Considering the provided code snippet, there may be several possible suggestions to make it work:

for-loop should be vectorized
since you are using max(0, ...) in order to get an intersection area, it may happen that it is constant 0 and no gradient is available so check if it is not stuck there
for mean calculation there is a ready-to-use Keras backend function K.mean
it is a good practice to bound the values in order to improve your optimization (e.g., to (0,1) range)

Make a custom loss function for mean intersection of union for regression in bounding boxes

1 Answers