44
votes

I'm battling some floating point problems in Pandas read_csv function. In my investigation, I found this:

In [15]: a = 5.9975

In [16]: a
Out[16]: 5.9975

In [17]: np.float64(a)
Out[17]: 5.9974999999999996

Why is builtin float of Python and the np.float64 type from Python giving different results? I thought they were both C++ doubles?

1
Note also that the Pandas read_csv function employs its own super-fast string-to-float conversion that is not correctly rounded. Thus after exporting a value and re-reading it, the recovered value may end up being 1 or 2 ulps different from the original. - Mark Dickinson

1 Answers

39
votes
>>> numpy.float64(5.9975).hex()
'0x1.7fd70a3d70a3dp+2'
>>> (5.9975).hex()
'0x1.7fd70a3d70a3dp+2'

They are the same number. What differs is their representation; the Python native type uses a "sane" representation, and the NumPy type uses an accurate representation.