(2^64) - 1 = 0 but numeric literal is ok?

General FreeBASIC programming questions.
cbruce
Posts: 136
Joined: Sep 12, 2007 19:13
Location: Dallas, Texas

(2^64) - 1 = 0 but numeric literal is ok?

Postby cbruce » Apr 06, 2018 21:59

Any idea why the numeric literal is acceptable where the calculation is not?

Code: Select all

' Prints 0
const X as ulongint = (2^64) - 1ull
print  "X = "; X

' Prints 18446744073709551615
const Z as ulongint = 18446744073709551615ull   ' same as (2^64) - 1ull ... right?
print  "Z = "; Z
MrSwiss
Posts: 3657
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: (2^64) - 1 = 0 but numeric literal is ok?

Postby MrSwiss » Apr 06, 2018 22:14

Its probably causing an 'overflow' before the '- 1ull' is even calculated.

All below versions appear to work OK, FBC 32 -gen gas / FBC 64 -gen gcc:

Code: Select all

Const As ULongInt x = &hFFFFFFFFFFFFFFFFull, y = 18446744073709551615ull, z = -1

Print x, y, z

Sleep
dodicat
Posts: 6761
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: (2^64) - 1 = 0 but numeric literal is ok?

Postby dodicat » Apr 06, 2018 22:18

2^64 is a double (the ^ operator)
change to

Code: Select all

' Prints 0
const X as ulongint = culngint(2^64) - 1
print  "X = "; X

' Prints 18446744073709551615
const Z as ulongint = 18446744073709551615ull   ' same as (2^64) - 1ull ... right?
print  "Z = "; Z
cbruce
Posts: 136
Joined: Sep 12, 2007 19:13
Location: Dallas, Texas

Re: (2^64) - 1 = 0 but numeric literal is ok?

Postby cbruce » Apr 06, 2018 23:23

Darn... thanks, @dodicat. I probably never would have thought to RTM about the '^' operator - it just seems so obvious what it is supposed to do...

"^ works with double float numbers only, operands of other types will be converted into double before performing the exponentiation"

I had tried with a ULONG and (2^32) - 1, which gave the correct answer, so it looked like an issue. But just an operator (pun intended) error. [smile]
adele
Posts: 47
Joined: Jun 13, 2015 19:33

Re: (2^64) - 1 = 0 but numeric literal is ok?

Postby adele » Apr 07, 2018 7:03

Hi,
just to make it "solved" ??

Code: Select all

const y as ulongint = (2 Shl 63) - 1ull
print  "y = ";y; " (shifted)"
Sleep

Adi
badidea
Posts: 2179
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: (2^64) - 1 = 0 but numeric literal is ok?

Postby badidea » Apr 07, 2018 9:03

Both (2 Shl 63) and (1 Shl 64) give warning 33(0): Shift value greater than or equal to number of bits in data type
Which makes sense, but they give different results however?:

Code: Select all

const y as ulongint = (2 Shl 63) - 1ull
print  "y = "; hex(y)
const x as ulongint = (1 Shl 64) - 1ull
print  "x = "; hex(x)
const z as ulongint = not(0)
print  "z = "; hex(z)
Sleep
counting_pine
Site Admin
Posts: 6237
Joined: Jul 05, 2005 17:32
Location: Manchester, Lancs

Re: (2^64) - 1 = 0 but numeric literal is ok?

Postby counting_pine » Apr 07, 2018 12:10

If 'shl 63' gives a warning, that's a bug, because shifting by 63 is well-defined.
Shifting by 64 is not well-defined. You would expect the result (after truncation) to be 0 for n>=64, but usually the CPU shifts by (n mod 64u), meaning '1 shl 64' is the same as '1 shl 0'.
fxm
Posts: 10046
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: (2^64) - 1 = 0 but numeric literal is ok?

Postby fxm » Apr 07, 2018 13:16

No bug:
'2 shl 63' gives a warning in fbc-32bit, and not in fbc-64bit. => OK
but '2ull shl 63' does not give a warning in fbc-32bit. => OK

Return to “General”

Who is online

Users browsing this forum: Gunslinger and 6 guests