How do you interoperate with other applications using double-precision floating-point?

General FreeBASIC programming questions.
cbruce
Posts: 135
Joined: Sep 12, 2007 19:13
Location: Dallas, Texas

Re: How do you interoperate with other applications using double-precision floating-point?

Postby cbruce » May 21, 2019 17:09

Hi srvaldez,

That's my whole problem... none of them return the exact same results.

So, for now, I'm writing it to have different implementations make a first attempt with a truncated (not rounded) 15 digits of precision. If that doesn't allow for equivalency checks, then we'll drop it down to 14 digits and try again. At the moment I'm thinking that if we can't get at least 12 digits to work, we'll just call it off as "totally incompatible"... but I'll have to calculate what that minimum number of digits really should be for the application.

If the app instances pass the equivalency checks... we'll go ahead with normal operations.

Return to “General”

Who is online

Users browsing this forum: No registered users and 13 guests