How do you interoperate with other applications using double-precision floating-point?

General FreeBASIC programming questions.
cbruce
Posts: 166
Joined: Sep 12, 2007 19:13
Location: Dallas, Texas

Re: How do you interoperate with other applications using double-precision floating-point?

Post by cbruce »

Hi srvaldez,

That's my whole problem... none of them return the exact same results.

So, for now, I'm writing it to have different implementations make a first attempt with a truncated (not rounded) 15 digits of precision. If that doesn't allow for equivalency checks, then we'll drop it down to 14 digits and try again. At the moment I'm thinking that if we can't get at least 12 digits to work, we'll just call it off as "totally incompatible"... but I'll have to calculate what that minimum number of digits really should be for the application.

If the app instances pass the equivalency checks... we'll go ahead with normal operations.
Post Reply