keras : Implementing a custom metric with logical operators

Question

I'm trying to create a custom metric to evaluate a keras model. The evaluation consist in returning 1 if y_true and y_pred are both higher or lower than a certain value (in my case 5), 0 otherwise. The following lambda expression is a demo stating what I want to achieve

lambda y_pred, y_true : 1 if y_true > 5  and y_pred > 5 or y_true < 5 and y_pred < 5 else 0

I've tried to implement it on a custom keras model doing the following:

def SAGR(y_true, y_pred):
   maj = K.greater([y_true, y_pred], 5)
   men = K.less([y_true, y_pred], 5)
   aremaj= K.all(maj)
   aremen = K.all(men)
   res = K.any([aremaj, aremen])

  return K.mean(K.cast(res,'float32'))

But the function always return 0.

The output of the last layer is linear with shape [None, 2]. Can anyone please explain me a way to implement the custom metric?

Thanks

Vikash Singh Vikash Singh · Accepted Answer · 2018-04-23T12:29:10

When you are checking aremaj= K.all(maj) you need to check it with axis=0. Same goes for aremen and res

def SAGR(y_true, y_pred):
    maj = K.greater([y_true, y_pred], 5)
    men = K.less([y_true, y_pred], 5)
    aremaj= K.all(maj, axis=0)
    aremen = K.all(men, axis=0)
    res = K.any([aremaj, aremen], axis=0)
    res_final = K.mean(K.cast(res,'float32'))
    print(K.eval(res_final))
    return res_final

Explanation:

K.eval(maj) # Looks like this
[[False False  True  True]
 [False  True  True False]]

K.eval(K.all(maj)) # evaluates to 1 value:
False

print(K.eval(K.all(maj, axis=0)))
# What we actually want is a comparison at axis 0 level.
[False False  True False]

Working code: Link

PS: you can also use more detailed variable names as aremaj is not very descriptive and SAGR is upper case plus not descriptive.

keras : Implementing a custom metric with logical operators

1 Answers