Welcome, Guest
Username: Password: Remember me
Visual Objects

Please use this forum to post questions about Visual Objects and Vulcan.NET

TOPIC:

Float, Real8, or Decimal? 01 Dec 2020 23:37 #16839

  • Chris
  • Chris's Avatar


  • Posts: 3739
  • Hi Otto,

    Yes, VO always treats floating point literals as FLOAT, so I am pretty sure the whole comparison is done with FLOAT rules, not with REAL8 ones. You should get the same results also in X#, if the project option /vo14 (Use FLOAT literals) is enabled.
    XSharp Development Team
    chris(at)xsharp.eu

    Please Log in or Create an account to join the conversation.

    Float, Real8, or Decimal? 03 Dec 2020 21:57 #16883

    • Otto
    • Otto's Avatar


  • Posts: 174
  • I've tested the /vo14 on an individual case and it looks good!
    Now we are going to test it in bulk. I'll hope next week to see the outcome.

    Thanks for all your prompt responses and patience. These are stressful times for us here.

    Please Log in or Create an account to join the conversation.

    Float, Real8, or Decimal? 03 Dec 2020 23:50 #16884

    • Chris
    • Chris's Avatar


  • Posts: 3739
  • Great to hear it works for you Otto! For projects moved from VO, there's no reason really ever having /vo14 disabled, at least in the first stages, since it is providing compatible behavior with the code in VO.

    And you can still use REAL8/4 literals when needed, even when /vo14 is enabled, by using the "d" and"s" postfixes:

    123.45s : System.Single/REAL4 literal
    123.45d : System.Double/REAL8 literal
    XSharp Development Team
    chris(at)xsharp.eu

    Please Log in or Create an account to join the conversation.

    Float, Real8, or Decimal? 04 Dec 2020 15:29 #16889

    • ArneOrtlinghaus
    • ArneOrtlinghaus's Avatar
    • Topic Author


  • Posts: 337
  • We are also working with /VO14 float literals enabled and we left float var definitions where float were.

    Originally we were thinking about converting to Real8. But in the programs there are too many comparisons like f == 0 .or. f <> 0 that are hard to find.

    There were very few cases where we had to change something in Dotnet in comparison to VO for getting the same results as before. Dotnet calculates with about 14 digits and VO with about 16. For working with normal business units as currency or quantities there is no problem. There were some places where very small numbers of less than 0.00000001 had to be compared and there we had to adapt the rounding process a little bit.
    Ntrim and Str(f) without specifying the exact number of digits gives also different results. But in any case there shouldn't be string conversions without specifying what is really wanted.

    Arne

    Please Log in or Create an account to join the conversation.

    Last edit: by ArneOrtlinghaus.