Calculating Precision and Recall by Class

Question

I am working with a two class dataset, 2 and 4 where 2 is the positive class and 4 is the negative class (regarding sentiment analysis).

I have a set of predictions from my model, and a set of actual values. I need to determine Precision and Recall for each class (the P and R scores for the positive and negative class).

Code is below:

preds = [4, 4,  4,  4,  4,  4,  4,  4,  4,  4,  4,  4,  4,  2,  4,  4,  4,  4,  4,  2,  2,  4,  4,  4,  4,  2]
actuals = [2,   4,  2,  4,  2,  4,  2,  4,  4,  4,  2,  4,  4,  4,  2,  2,  4,  4,  2,  4,  4,  4,  4,  4,  4,  2]

true_pos = 0
true_neg = 0
false_pos = 0
false_neg = 0
for pred, act in zip(preds, actuals):
    # 2 is positive, 4 is negative

    if(pred == 2 & act == 2):
        true_pos += 1

    elif(pred == 4 & act == 4):
        true_neg += 1

    elif(pred == 2 & act == 4):
        false_pos += 1

    elif(pred == 4 & act == 2):
        false_neg += 1

print("True Positive: ", true_pos)
print("True Negative: ", true_neg)
print("False Positive: ", false_neg)
print("False Negative: ", false_neg)

Which yields:

True Positive:  1
True Negative:  14
False Positive:  0
False Negative:  0

However, I am really stuck on how I am supposed to be calculating these metrics by class. This SO post indicates how to do this for an overall, but not by class.

Ideally, I would end up with an output like:

Class 2 P Score: x
Class 2 R Score: x
Class 4 P Score: x 
Class 4 R Score: x

but I am not sure how to compute that.

How can I adapt my current logic to check for Precision and Recall scores for each class with the above data?

Could you show your exact sample inputs and your desired outputs? — ycx
I'm confused as to what you mean by sample inputs. Those would be the actual labels and the predicted labels, I think, which are composed in the two lists in the beginning of the program. The desired output would be Precision and Recall score for each class, but the question relates to not knowing how to build a function or logic to compute that. — wundermahn
I'm confused by what you mean by 'each class'. What exactly is 'each class'? Can you show sample inputs of 'each class'? — ycx
A class is a label...every single value in the preds, or the predicted class list, is a prediction of a class. The actuals is a list of the ACTUAL classes for a dataset. The two lists exist to simplify the program and provide a minimal, complete, reproducible sample. This would equate to running sklearn's train_test_split on an input dataset... — wundermahn
Yeah, the sklearn link for confusion_matrix(preds, actuals).ravel() is all I really needed. From there I can compute the formulas pretty easily. That is what I was having trouble with. Thanks! @ycx — wundermahn

ycx ycx · Accepted Answer · 2019-10-28T16:04:58

I think I know which direction to point you in:
https://scikit-learn.org/stable/modules/generated/sklearn.metrics.confusion_matrix.html
This should be what you are looking for.

Refer to the wikipedia link: https://en.wikipedia.org/wiki/Confusion_matrix
and read up on how to use this.

Calculating Precision and Recall by Class

1 Answers