Java: Trigonometry and double inaccuracy causing NaN

1
votes

I have a distance formula using latitude and longitude:

distance = EARTH_MILES_RADIUS
    * Math.acos(Math.sin(lat1 / RADIAN_CONV)
    * Math.sin(lat2 / RADIAN_CONV)
    + Math.cos(lat1 / RADIAN_CONV)
    * Math.cos(lat2 / RADIAN_CONV)
    * Math.cos((lng2 - lng1) / RADIAN_CONV));

lat1,lng1,lat2,lng2 are double primitives. They come to me as double primitives and there is nothing I can do about it.

The problem is that when I have a pair of longitude or latitudes that are the same the formula sometimes returns NaN. I believe this is because I am taking the arc cosine of a number very slightly greater than 1, when in fact it should be exactly 1. I would probably have problems if the points were antipodal as well, where they might be slightly less than -1.

How can I best fix this problem?

3
javafloating-pointgeospatialtrigonometrydouble-precision
If you want to study a better great-circle distance and bearing calculation implementation in Java, here is the place UNICAR NETCDF library svn.unidata.ucar.edu/repos/common/java/trunk/src/main/java/ucar/… Copyright notice from UNICAR on NetCDF: These applications are copyrighted by UCAR and are intended to be freely available with very minimal restriction. – eee
Reformatted code; please revert if incorrect. – trashgod
@eee I see in the linked code the they have if ((lat1 == lat2) && (lon1 == lon2)) in which case the distance is set to zero, so perhaps that is the best solution. Maybe checking for equality within a certain accuracy is unnecessary (puddingfox's solution). This was solution that I tried on my system and it worked great but I wasn't sure if it would work in every case. – Muhd

3 Answers

1
votes

If you are indeed calculating great circle distance as I think you are, you should use the Vincenty formula instead of what you have.

http://en.wikipedia.org/wiki/Great-circle_distance

1
votes

Check for two very close (ideally equal) values in your code. For example:

boolean doubleApproxEqual(double a, double b) {
    double PRECISION = 0.000001;
    if (Math.abs(a-b) < PRECISION) //not sure what the name of the function is
                                   //cannot be bothered to check
        return true;
    return false;
}

if you get a True, do cos(1.0) or whatever

0
votes

Simply checking for equality was enough to fix my problem:

if (lat1 == lat2 && lng1 == lng2) ...