Custom Keras metric return 'axis out of bounds' error

Question

I've built a multi-class, multi-label image classification network using Keras. There are 25 classes overall and each image has at least one class in it. I want to implement a custom accuracy metric which tells me how often the highest probability class is in the image (regular accuracy is less meaningful as the true positives are swamped by the true negatives).

I've built a simple function which generates the desired accuracy metric when I manually feed in y_true and y_pred. However when I try and insert this function into the model training process, it produces an error.

def customAcc(y_true, y_pred):
    classPreds = np.array([np.eye(numClasses)[x] for x in  np.argmax(y_pred, axis=1)])
    correctPreds = y_true * classPreds
    return np.mean(np.sum(correctPreds, axis=1))

model.compile(optimizer=tf.keras.optimizers.Adam(lr=0.0001), 
loss='binary_crossentropy', metrics=['accuracy', customAcc])

AxisError: axis 1 is out of bounds for array of dimension 1

Julio Cezar Silva Julio Cezar Silva · Accepted Answer · 2019-05-21T00:22:14

TL;DR

y_pred is 1D, it only has one possible axis. Remove axis=1 from your np.argmax call.

Walkthrough

The problem in this particular case is this line:

classPreds = np.array([np.eye(numClasses)[x] for x in  np.argmax(y_pred, axis=1)])

Specifically: np.argmax(y_pred, axis=1). Your y_pred is a one-dimensional array - such as [0.1, 0.2] - and you're telling np.argmax to look for values across axis=1 which doesn't exist unless you pass arrays with two or more dimensions - such as [[0.1, 0.2], [0.3, 0.4]].

An actionable example:

>>> import numpy as np
>>> num_classes = 25
>>> np.argmax([0.1, 0.5, 0.9]) # max value's index on 1D array
2
>>> np.argmax([0.1, 0.5, 0.9], axis=1) # max value's index on axis 1 of 1D array
AxisError: axis 1 is out of bounds for array of dimension 1

Had y_pred been a 2D array, the axis error wouldn't happen - but np.argmax would then return a list of indexes instead of a scalar, such as below:

>>> np.argmax([
...     [0.1, 0.5, 0.9],
...     [0.9, 0.5, 0.1]
... ], axis=1)
array([2, 0], dtype=int64) # first array's max at index 2, second array's max at index 0

By taking away axis=1 from argmax, you'll then get the correct, scalar index for the maximum value within y_pred.

Custom Keras metric return 'axis out of bounds' error

1 Answers

TL;DR

Walkthrough