Test score vs test accuracy when evaluating model using Keras

Question

Im using a neural network implemented with the Keras library and below is the results during training. At the end it prints a test score and a test accuracy. I can't figure out exactly what the score represents, but the accuracy I assume to be the number of predictions that was correct when running the test.

Epoch 1/15 1200/1200 [==============================] - 4s - loss: 0.6815 - acc: 0.5550 - val_loss: 0.6120 - val_acc: 0.7525

Epoch 2/15 1200/1200 [==============================] - 3s - loss: 0.5481 - acc: 0.7250 - val_loss: 0.4645 - val_acc: 0.8025

Epoch 3/15 1200/1200 [==============================] - 3s - loss: 0.5078 - acc: 0.7558 - val_loss: 0.4354 - val_acc: 0.7975

Epoch 4/15 1200/1200 [==============================] - 3s - loss: 0.4603 - acc: 0.7875 - val_loss: 0.3978 - val_acc: 0.8350

Epoch 5/15 1200/1200 [==============================] - 3s - loss: 0.4367 - acc: 0.7992 - val_loss: 0.3809 - val_acc: 0.8300

Epoch 6/15 1200/1200 [==============================] - 3s - loss: 0.4276 - acc: 0.8017 - val_loss: 0.3884 - val_acc: 0.8350

Epoch 7/15 1200/1200 [==============================] - 3s - loss: 0.3975 - acc: 0.8167 - val_loss: 0.3666 - val_acc: 0.8400

Epoch 8/15 1200/1200 [==============================] - 3s - loss: 0.3916 - acc: 0.8183 - val_loss: 0.3753 - val_acc: 0.8450

Epoch 9/15 1200/1200 [==============================] - 3s - loss: 0.3814 - acc: 0.8233 - val_loss: 0.3505 - val_acc: 0.8475

Epoch 10/15 1200/1200 [==============================] - 3s - loss: 0.3842 - acc: 0.8342 - val_loss: 0.3672 - val_acc: 0.8450

Epoch 11/15 1200/1200 [==============================] - 3s - loss: 0.3674 - acc: 0.8375 - val_loss: 0.3383 - val_acc: 0.8525

Epoch 12/15 1200/1200 [==============================] - 3s - loss: 0.3624 - acc: 0.8367 - val_loss: 0.3423 - val_acc: 0.8650

Epoch 13/15 1200/1200 [==============================] - 3s - loss: 0.3497 - acc: 0.8475 - val_loss: 0.3069 - val_acc: 0.8825

Epoch 14/15 1200/1200 [==============================] - 3s - loss: 0.3406 - acc: 0.8500 - val_loss: 0.2993 - val_acc: 0.8775

Epoch 15/15 1200/1200 [==============================] - 3s - loss: 0.3252 - acc: 0.8600 - val_loss: 0.2960 - val_acc: 0.8775

400/400 [==============================] - 0s

Test score: 0.299598811865

Test accuracy: 0.88

Looking at the Keras documentation, I still don't understand what score is. For the evaluate function, it says:

Returns the loss value & metrics values for the model in test mode.

One thing I noticed is that when the test accuracy is lower, the score is higher, and when accuracy is higher, the score is lower.

@maz I am using Keras 2.0.3 and the code I am experimenting with is this: github.com/fchollet/keras/blob/master/examples/imdb_lstm.py — Stephen Johnson
please check answer to similar question here stackoverflow.com/a/57645003/2851685 — Dev Khadka

maz maz · Accepted Answer · 2017-04-24T14:35:47

For reference, the two relevant parts of the code:

model.compile(loss='binary_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

score, acc = model.evaluate(x_test, y_test,
                            batch_size=batch_size)
print('Test score:', score)
print('Test accuracy:', acc)

Score is the evaluation of the loss function for a given input.

Training a network is finding parameters that minimize a loss function (or cost function).

The cost function here is the binary_crossentropy.

For a target T and a network output O, the binary crossentropy can defined as

f(T,O) = -(T*log(O) + (1-T)*log(1-O) )

So the score you see is the evaluation of that.

If you feed it a batch of inputs it will most likely return the mean loss.

So yeah, if your model has lower loss (at test time), it should often have lower prediction error.

Test score vs test accuracy when evaluating model using Keras

2 Answers