I am using keras to perform binary classification (single label, one or zero).
I would like to create a custom metric that takes the highest 10% from the predictions and calculates the accuracy of those.
So for example with the labels [1, 0, 1, 1, 0, 1, 0, 1, 0, 1]
and the predictions [0.5, 0.7, 0.98, 0.1, 0.2, 0.5, 0.2, 0.1, 0.9, 0.8]
It only computes the accuracy of the 0.98 prediction.
I managed to solve my problem using this code:
import tensorflow as tf
def topacc(y_true, y_pred):
k = tf.cast(len(y_true) // 10, 'int64')
y_true, y_pred = tf.transpose(y_true), tf.transpose(y_pred)
return tf.keras.metrics.top_k_categorical_accuracy(y_true, y_pred, k=k)
This works as a full keras metric
Try this code:
import tensorflow as tf
def accuracy_metric_fn(y_true, y_pred):
l = len(y_pred) // 10
top = tf.math.top_k(y_pred, l)
y_pred = top.values
y_true = tf.gather(y_true, top.indices)
y_true = tf.cast(y_true, tf.float32)
loss = tf.keras.metrics.binary_accuracy(
y_true, y_pred)
return loss
y_true = [1, 0, 1, 1, 0, 1, 0, 1, 0, 1]
y_pred = [0.5, 0.7, 0.98, 0.1, 0.2, 0.5, 0.2, 0.1, 0.9, 0.8]
print(accuracy_metric_fn(y_true, y_pred).numpy())
