TOPIC:

Hello
I have a multi dim array with text and float. And i fill it with a dbf. Now when i compare the results. i have a fault.
In fKontoSaldo and aWriteLines is the same value.
What's wrong here ?

I hope this time my sample in the attachemt is Ok

Horst

Horst,
Most likely there is a rounding problem. Some floating point numbers cannot be stored 100% exactly.
You can either add:which tells the runtime to ignore differences smaller than 1/10 of a cent
or compare the numbers with Round(fKontoSaldo,2)

You will see similar problems in all apps that work with floating point numbers.


Robert

Hi Horst,

Yes, the sample is great!

The problem is that floating point data types like REAL and FLOAT cannot represent the decimal part of non-integer numbers precisely (that;s for all computer programming languages), there's always some precision lost which cascades when you do arithmetic operations on them. In your particular sample, the fTotal var ends up having a value of 19106.990000000027 and this is why the comparison with 19106.99 fails.

In order to workaround this, VO introduced (and X# also supports it) the function SetFloatDelta(), which you can use to tell the runtime that it should treat floating point numbers which are different by just a tiny amount as equal. So, in your sample, if you use something like SetFloatDelta(0.000001), then it will work as you expect it.

In .Net, there's a new data type exactly for that, System.Decimal, which is slower than REAL/FLOAT, but does not suffer from this lack of precision. If you change the data type of your locals fTotal and fKontoSaldo from FLOAT to Decimal, you will again get the expected results, without needing to also use SetFloatDelta().

.

Hello
Thanks Robert and Chris for explaining that.
I will now only use Decimal instead of Float, because i will forget that float is making this little misstake

btw: when i am importing some excel data, there also has numbers in the file (on disc) like 15.12200000009 . But the input of the user was 15.122.

Horst

Hi Horst,

Same issue, the value 15.122 cannot be represented exactly in the binary notation that FLOAT/REAL uses. It's a bit similar to that the result of the division 10/3 cannot be represented exactly in decimal notation, binary notation has similar problems as well.