0
votes

I'm trying to understand floating point math in C#. I have read a lot of documents and papers over the entire day , about floating points and sad to say most of the time I have only an incling about the core issues. I'm quite good at English not being a native speaker but sometimes I still miss context. What I still know from my studies is that floating point is just a representation of 4 bytes and that doing math on it will always be wrong because of it. You can compare with an epsilon for error margin but even that is just a close approximation. That being said to point out what I understand this is the case I'm struggling with.

    [Test]
    public void LibHfzCompressedReadTest()
    {           
        LibHfZ lib = new LibHfZ();
        HfzFile file1 = lib.loadFile("c:\\demo2_HF.hf2.gz", HfzFormat.LIBHFZ_FORMAT_HF2_GZ);
        //HfzFile file2 = lib.loadFile("c:\\fractal-16bit.hf2.gz", HfzFormat.LIBHFZ_FORMAT_HF2_GZ);

        List<float> data = file1.getTileData((short)0);

        double HFmin = 0;
        double HFmax = 0;

        for(int i=0;i < data.Count;i++)
        {
            double f = data.ElementAt(i);
            System.Console.WriteLine("float: " + f);
            if(i == 0)
            {
                HFmin = HFmax = f;
            }
            else
            {
                if (f < HFmin && f != 0)
                {
                    System.Console.WriteLine("new min: " + f);
                    HFmin = f;
                }

                if (f > HFmax && f != 0)
                {
                    System.Console.WriteLine("new max: " + f);
                    HFmax = f;
                }
            }
        }

        System.Console.WriteLine("HFmin: " + HFmin);
        System.Console.WriteLine("HFmax: " + HFmax);
        float Precis = file1.header.Precis;
        System.Console.WriteLine("Precis: " + Precis);
        float BlockLevels = (float)((HFmax - HFmin) / Precis) + 1;
        System.Console.WriteLine("BlockLevels: " + BlockLevels);

        // calc scale 
        float VertScale = (float)(HFmax - HFmin) / BlockLevels;
        if(VertScale<=0)
        {
            VertScale = 1.0f;
        }

        System.Console.WriteLine("VertScale: " + VertScale);
        //System.Console.WriteLine(file2);
    }

    ========output==========
    HFmin: 0
    HFmax: 1019,586
    Precis: 0,01
    BlockLevels: 101959,6
    VertScale: 0,009999902

The end result SHOULD be VertScale:0,009999821 but it's VertScale: 0,009999902

I'm sure the input data is is right, cause I just read it from a tested and verified file. My personal hunch is that it has something to do with the comparison on HFmax and HFmin, maybe they are giving false positives or negatives causing the end result of HFmax and HFmin to be wrong.

I've been through a lot of stackoverflows about how you can compare for equality on a floating point with the epsiolon but I haven't came across one for higher or lower ... Any insights and please not another paper :( I've read dozens today already I just want to get this understood and fixed :(

UPDATE I added a check <= 0 on the VertScale to see if it would evaluate and set it to 1.0f however the output is the same. I tried this with doubles as well as recommended in the first comments however this was still the same result.

dataset: http://users.telenet.be/sunspot/hfzcsharpsource/testfloatarray.7z

1
What do you want fixed? Try double instead of float for all variables of type float (leave List<float> alone). Is that better (whatever "better" may mean) ?Kris Vandermotten
float is 32bit, double is 64bit and decimal is 128bit. What Kris Vandermotten should solve your problem as with System.Math it uses double as well.Bauss
I have to add some context it seems I've been trying to write a file of this type bundysoft.com/docs/doku.php?id=l3dt:formats:specs:hf2 . I have been able to write the code to read the entire file into memory, with the tiles part as an array of floats that you see in my example here. Now I want to write just the very first 4 bytes of one single tile in this example. So I use the calculation the author provided and I'm checking the result with the first 4 bytes of the origin file and I just can't get them to match. I'm not sure I can use doubles wouldn't that alter the data inputkenny
doing math on it will always be wrong because of it No, no, no, you have completely misunderstood floating-point arithmetic. What you must understand is that floating-point arithmetic is not the same as real arithmetic (ie the kind of arithmetic you learned at school). F-p arithmetic is used on computers to approximate real arithmetic, but it is not wrong.High Performance Mark
well the array of floats has (256*256) - 255 , entries in it do you want an entire data set ? I could upload one somewhere if you do. I've updated the codeblock with my entire unit testkenny

1 Answers

0
votes

Well I fixed the greater then and smaller then dilemma after reading following article and applying the class created by the writer.

http://www.codeproject.com/Articles/383871/Demystify-Csharp-floating-point-equality-and-relat

However now my equal , smaller and bigger then expressions work, but the results I get are still far to be desired. I decided to run the same program on c++ and on C# with following results

C++
====
Vertscale: 0.00999982096
VertOffset: 53.4127693
LineDepth: '\x2'
FirstVal: 27
Li: 14
LastVal: 41
f: 53.8227615

c#
====
VertScale: 0,009999821
VertOffset: 53,41277
LineDepth: 2
FirstVal: 27
Li: 14
LastVal: 41
f: 53,82276

You notice the small differences in VertScale and f values after calculation. You would consider them insignificant, but those small changes make it impossible for me to write the file to disk cause it ruins the compression algorithm I'm trying to write.

http://www.bundysoft.com/docs/doku.php?id=l3dt:formats:specs:hf2

I'll have to face the fact that it will be impossible to write this in C# due to the float calculations ending up different in c# then c++ where the original designer of the filetype wrote his algorithm.