Are there any better ways to compare nan values in ground truth values and predicted values?

0
votes

I have a validation dataset which has ground truths labelled "nan" values.

The expected ground truth is to return a String, but when nothing is detected from the data, np.nan will be returned.

What I'm doing right now is to pandas replace my predicted values that is np.nan into "nan" thus it is then compared with the "nan" from ground truth so I know that my algorithm is predicting correctly.

Is there a better way to compare/handle nan values?

1
pythonpandasmachine-learning
Can you elaborate on what you’re trying to do, I’m not sure I understand correctly. – AMC
I am trying to ask if comparing "nan" == "nan" is a good practice – Chia Yi

1 Answers

0
votes

Sounds like you're running into the np.nan != np.nan issue. In general, you can use df.equals to get around that:

In [323]: df1 = pd.DataFrame([[np.nan, 1, 2], [2, 3, 4]])

In [324]: df1
Out[324]:
     0  1  2
0  NaN  1  2
1  2.0  3  4

In [325]: df2 = df1

In [326]: df1 == df2
Out[326]:
       0     1     2
0  False  True  True
1   True  True  True

In [327]: df1.equals(df2)
Out[327]: True

In [328]: df1[0].equals(df2[0])
Out[328]: True