make accuracy appear in my result and interpret the results of the loss and the val_loss

Question

i'm new in tensorflow i'm trying to learning it by examples in github, now i found an example but the results of the loss and val_loss are more than '1' ( you can see bellow the results is between 800 and 700 ) while normally in other examples the loss and val_loss are between 0 and 1)

in addition i would like how to make appear the accuracy. this is the code. https://github.com/simoninithomas/DNN-Speech-Recognizer/blob/master/train_utils.py

thank you !

def train_model(input_to_softmax, pickle_path, save_model_path, train_json='train_corpus.json', valid_json='valid_corpus.json', minibatch_size=20, spectrogram=True, mfcc_dim=13, optimizer=SGD(lr=0.02, decay=1e-6, momentum=0.9, nesterov=True, clipnorm=5), epochs=20, verbose=1, sort_by_duration=False, max_duration=10.0):

# create a class instance for obtaining batches of data
audio_gen = AudioGenerator(minibatch_size=minibatch_size, 
    spectrogram=spectrogram, mfcc_dim=mfcc_dim, max_duration=max_duration,
    sort_by_duration=sort_by_duration)
# add the training data to the generator
audio_gen.load_train_data(train_json)
audio_gen.load_validation_data(valid_json)
# calculate steps_per_epoch
num_train_examples=len(audio_gen.train_audio_paths)
steps_per_epoch = num_train_examples//minibatch_size
# calculate validation_steps
num_valid_samples = len(audio_gen.valid_audio_paths) 
validation_steps = num_valid_samples//minibatch_size

# add CTC loss to the NN specified in input_to_softmax
model = add_ctc_loss(input_to_softmax)

# CTC loss is implemented elsewhere, so use a dummy lambda function for the loss
model.compile(loss={'ctc': lambda y_true, y_pred: y_pred}, optimizer=optimizer)

# make results/ directory, if necessary
if not os.path.exists('results'):
    os.makedirs('results')

# add checkpointer
checkpointer = ModelCheckpoint(filepath='results/'+save_model_path, verbose=0)

# train the model
hist = model.fit_generator(generator=audio_gen.next_train(), steps_per_epoch=steps_per_epoch,
    epochs=epochs, validation_data=audio_gen.next_valid(), validation_steps=validation_steps,
    callbacks=[checkpointer], verbose=verbose)

# save model loss
with open('results/'+pickle_path, 'wb') as f:

pickle.dump(hist.history, f)

Epoch 1/20 106/106 [==============================] - 302s - loss: 839.6881 - val_loss: 744.7609 Epoch 2/20 106/106 [==============================] - 276s - loss: 767.3973 - val_loss: 727.8361 Epoch 3/20 106/106 [==============================] - 272s - loss: 752.6904 - val_loss: 720.8375 Epoch 4/20 106/106 [==============================] - 261s - loss: 751.8432 - val_loss: 728.3446 Epoch 5/20 106/106 [==============================] - 261s - loss: 752.1302 - val_loss: 733.3166 Epoch 6/20 106/106 [==============================] - 264s - loss: 752.3786 - val_loss: 722.4345 Epoch 7/20 106/106 [==============================] - 265s - loss: 752.7827 - val_loss: 723.2651 Epoch 8/20 106/106 [==============================] - 263s - loss: 752.5077 - val_loss: 736.0229 Epoch 9/20 106/106 [==============================] - 263s - loss: 752.5616 - val_loss: 731.2018

Prannay Khosla Prannay Khosla · Accepted Answer · 2019-04-26T11:08:54

The loss that you are using is described in this pdf.

When you say accuracy it could mean a lot of things:

Single unit accuracy (average it over the labels you have. NOTE: you have multiple labels for the same data point since its a temporal classification) [Would be between 0 and 1]
Error rate: Could be defined as edit distance between the predicted labels and true labels [Would be between 0 and MAX_LABELS averaged over data points.
Precision of labels averaged over all timesteps and data points.

There is no reason for it to be between 0 and 1. On the other hand your loss is a connectionist temporal loss. This loss predicts either a label or a blank label at every timestep. Then we use Cross Entropy on top of the labels. The cross entropy of two probability distributions is only a positive quantity, and is not between 0 and 1.

Therefore this is not an issue. If you would like to see the accuracies you will have take some test data and make a prediction. You can calculate accuracy (as defined above) using Tensorflow against the expected labels using whatever metric you want, and use that as your accuracy. You can technically use any metric defined in Tensorflow: https://www.tensorflow.org/api_docs/python/tf/metrics/, after your prediction step.

make accuracy appear in my result and interpret the results of the loss and the val_loss

1 Answers