KeyPgFormat wiki page

Forum for discussion about the documentation project.
Post Reply
badidea
Posts: 2586
Joined: May 24, 2007 22:10
Location: The Netherlands

KeyPgFormat wiki page

Post by badidea »

I am trying to reproduce the 'examples table' on KeyPgFormat so that the actual example code can be added.
My code for the 'Sample numeric formats' part seems to give slightly different results however:

Code: Select all

#include "string.bi"

dim as double numberVal(...) = {5, -5, .5}
dim as string formatStr(...) = {"","0","0.00","#,##0","#,##0.00","0%", "0.00%", "0.00E+00", "0.00E-00"}

? "Format (fmt)", str(numberVal(0)), str(numberVal(1)), str(numberVal(2))
for iFormat as integer = 0 to ubound(formatStr)
	? formatStr(iFormat),
	for iNumber as integer = 0 to ubound(numberVal)
		? format(numberVal(iNumber), formatStr(iFormat)),
	next
	?
next
My output:

Code: Select all

Format (fmt)  5             -5            0.5
              5             -5            .5            
0             5             -5            0             
0.00          5.00          -5.00         0.50          
#,##0         5             -5            0             
#,##0.00      5.00          -5.00         0.50          
0%            500%          -500%         50%           
0.00%         500.00%       -500.00%      50.00%        
0.00E+00      5.00E+00      -5.00E+00     5.00E-01      
0.00E-00      5.00E00       -5.00E00      5.00E-01
On wiki:

Code: Select all

Format (fmt)  5             -5            .5
Null String   5             -5            0.5
0             5             -5            1   <-- different
0.00          5.00          -5.00         0.50
#,##0         5             -5            1   <-- different
#,##0.00      5.00          -5.00         0.50
0%            500%          -500%         50%
0.00%         500.00%       -500.00%      50.00%
0.00E+00      5.00E+00      -5.00E+00     5.00E-01
0.00E-00      5.00E00       -5.00E00      5.00E-01
Has the 'format' command changed, or did I make a mistake?
fxm
Moderator
Posts: 12083
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: KeyPgFormat wiki page

Post by fxm »

It seems that for the exact value 0.5, CINT and FORMAT round down while USING rounds up:

Code: Select all

#include "string.bi"

Print Cint(0.4999999999), Format(0.4999999999, "0"), Using "#"; 0.4999999999
Print Cint(0.5)         , Format(0.5, "0")         , Using "#"; 0.5
Print Cint(0.5000000001), Format(0.5000000001, "0"), Using "#"; 0.5000000001
For 1.5, CINT and USING round up while FORMAT rounds down.
fxm
Moderator
Posts: 12083
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: KeyPgFormat wiki page

Post by fxm »

Code: Select all

#include "string.bi"

Print Cint(0.5)         , Format(0.5, "0")         , Using "#"; 0.5
Print Cint(1.5)         , Format(1.5, "0")         , Using "#"; 1.5
Print Cint(2.5)         , Format(2.5, "0")         , Using "#"; 2.5
Print Cint(3.5)         , Format(3.5, "0")         , Using "#"; 3.5
Print Cint(4.5)         , Format(4.5, "0")         , Using "#"; 4.5
Print Cint(5.5)         , Format(5.5, "0")         , Using "#"; 5.5
Print Cint(6.5)         , Format(6.5, "0")         , Using "#"; 6.5

Code: Select all

 0            0             1
 2            1             2
 2            2             3
 4            3             4
 4            4             5
 6            5             6
 6            6             7
CINT uses the round-to-even method, while FORMAT rounds down and USING rounds up.

CINT : Round half to even
FORMAT : Round half down
USING : Round half up
badidea
Posts: 2586
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: KeyPgFormat wiki page

Post by badidea »

fxm wrote:FORMAT : Round half down
Yes, but not in 2008, when the page was created?
Anyway, I will update the page to current behaviour.
Post Reply