Confusion matrix and classification report

Question

I successfully create a simple 1D CNN for classification with 3 classes. In the training process, I save the model and weight into yaml and h5 file. Then, in the testing process, I successfully load the model and weight and use it for real-time classification, by returning the class as the output. However, I also test my model with test data, and I want to see it as a confusion matrix. Here's the code that I made:

from keras.models import model_from_yaml
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import classification_report, confusion_matrix
import os
from numpy import array
import numpy as np
import pandas as pd

# load YAML and create model
yaml_file = open('a32.yaml', 'r')
loaded_model_yaml = yaml_file.read()
yaml_file.close()
loaded_model = model_from_yaml(loaded_model_yaml)

# load weights into new model
loaded_model.load_weights("a32.h5")
print("Loaded model from disk")

loaded_model.compile(
    loss='sparse_categorical_crossentropy',
    optimizer='adam',
    metrics=(['accuracy'])
)

#Load data
test_data=pd.read_csv('data/ccpp/t2datn.csv',header=None)
test=test_data.iloc[:,0:2]

#Normalized test set
scaler=StandardScaler().fit(test)
x_test=scaler.transform(test)
y=np.expand_dims(x_test,axis=2)

#Make a prediction
predictions = loaded_model.predict(y)
ynew = loaded_model.predict_classes(y)

yp = np.argmax(predictions, axis=1)
#print(yp)
print("Confusion Matrix")
print(confusion_matrix(ynew, yp))
print("Classification Report")
target_names = ['Too High', 'Normal', 'Too Low']
print(classification_report(ynew,yp, target_names=target_names))

But I always get the output as 100% classified to each class. However, when I evaluate the test data, the accuracy is only around 80%. Can you tell me which part of the confusion matrix's code is wrong?

Output:

Confusion Matrix
[[1967    0    0]
 [   0 3252    0]
 [   0    0 1159]]
Classification Report
              precision    recall  f1-score   support

    Too High       1.00      1.00      1.00      1967
      Normal       1.00      1.00      1.00      3252
     Too Low       1.00      1.00      1.00      1159

    accuracy                           1.00      6378
   macro avg       1.00      1.00      1.00      6378
weighted avg       1.00      1.00      1.00      6378

Valentin Richer Valentin Richer · Accepted Answer · 2020-05-06T09:29:42

In your classification report, you are comparing ynew and yp which are basically the same but computed differently. You should compare ynew with a y_groundtruth.

Confusion matrix and classification report

2 Answers