How precision of floating-point works exactly in practical examples?

Question

I thought that float precision is 7 decimal places/digits (including both integer and decimal part) - here I mean base10 7 digits - I can type those 7 digits in my floating-point literal in my code editor. It means that if I have 7 significant digits (before and after decimal point) in two numbers, those two numbers shall be always different.

But as I see, two numbers with 7 significant digits sometimes differ, and sometimes same!!!

1) Where am I wrong?

2) What is the pattern and principle in examples below? Why same 7-digit-precision combinations sometimes are treated as different, and other times are treated as being the same?

float f01 = 90.000_001f;
float f02 = 90.000_002f;    //  f01 == f02 is TRUE ! (CORRECT RESULT)

float f03 = 90.000_001f;
float f04 = 90.000_003f;    //  f03 == f04 is TRUE ! (CORRECT RESULT)

float f1 = 90.000_001f;     
float f2 = 90.000_004f;     // FALSE (INCORRECT RESULT)

float f3 = 90.000_002f;
float f4 = 90.000_009f;     // FALSE (INCORRECT RESULT)

float f5 = 90.000_009f;
float f6 = 90.000_000f;     // FALSE (INCORRECT RESULT)

float f7 = 90.000_001f;
float f8 = 90.000_009f;     // FALSE (INCORRECT RESULT)

Any rule you have seen about number of significant decimal digits is only a rough rule of thumb. — cpp beginner

hugh hugh · Accepted Answer · 2018-03-31T09:38:50

Seven decimal places is a convenient rule of thumb, but it's not what's really going on. Java's float is a 32-bit binary floating point format, following the IEEE-754 standard. The encoding has 1 sign bit, 23 bits for the mantissa and 8 for the exponent, so your value is in scientific notation, in binary:

f = +/- mantissa * 2^exponent

Converting your values into this format you should be able to see what's happening:

90.000001 = 0(sign) 10000101(exponent) 01101000000000000000000(mantissa)
90.000003 = 0(sign) 10000101(exponent) 01101000000000000000000(mantissa)
90.000004 = 0(sign) 10000101(exponent) 01101000000000000000001(mantissa)

This is a handy tool for comparing encoded values if you'd like to explore further: https://www.h-schmidt.net/FloatConverter/IEEE754.html

In practice, the solution to this is that you should never use the == operator to compare floating point values, always compare floats with a precision:

Math.abs(x - y) < epsilon

How precision of floating-point works exactly in practical examples?

1 Answers