Shared 3.9GB array too big in FB64bits with 8GB RAM

[Luis Babboni]

Thats!

I want to do with FB 64 bits in my Windows 7 64 bits with 8GB RAM this:

Code: Select all

Dim Shared P(1 To 10, 1 To 20, 1 To 20, 0 To 63, 0 To 63, 0 To 63) As UInteger<32> 

Thats I understood have 3,9GB size:
10*20^2*64^3*4 / (1024^3) = 3,90625

And when try to compile FBEdit says me:
"...error 49: Array too big..."

Any hint about why I can´t do it?

Thanks.

[Tourist Trap]

[Edit] I count 33554432000/8 bytes, or approximatively 34/8 Gigabytes. Storing a so big amount of data is very rarely needed, unless you store hi-res multimedia, what is this array for?

[RockTheSchock]

Code: Select all

reDim Shared P(1 To 10, 1 To 20, 1 To 20, 0 To 63, 0 To 63, 0 To 63) As UInteger<32> 

[fxm]

See at CompilerFeatures:

.....
Arrays

• Fixed- and variable- length arrays are supported, up to 2 GB in size.
• Up to eight dimensions, including arrays with unknown dimensions.
• Any lower and upper boundaries.
• Element data can be preserved during a re-size of variable-length arrays with Redim using the new Preserve specifier.

[Luis Babboni]

Thanks Rock! Now works! :-)

Why you count as much Tourist?
I puted my calculation in my post. UInterger<32> has 4 bytes I undestood.

[Tourist Trap]

Thanks Rock! Now works! :-)

Why you count as much Tourist?
I puted my calculation in my post. UInterger<32> has 4 bytes I undestood.

Right sorry. But you dont have to divide by 1024, you're already in octet=byte from what I understand.
Your array remains impressive.

[Luis Babboni]

I read what fxm said but still do not understood what kind of risk I have using REDIM instead of Dim :-/
I could lost the values?
As far as I understood, all I read about "preserve" is referred to use ReDim with previous Dimed array, I´m right?
So if you use ReDim to define an array for the first time, you do not need to be worried about lost data?
But not sure then why to not use always ReDim instead of Dim.... is cause that´s about variable lenghts arrays I still do not understand?
Wait.... if I have no risk using ReDim , please do not waste your time trying to explain me more... at least for the moment. :-D

Tourist, I´m trying to do some stats about chess, those 0 to 63 for example are the numbers of squares in a chess board.

[Tourist Trap]

Wait.... if I have no risk using ReDim , please do not waste your time trying to explain me more... at least for the moment. :-D

From what I've always understood: use redim when you declare a variable-length array. Use redim preserve each time you change the array if you don't want it reinitialized.

Tourist, I´m trying to do some stats about chess, those 0 to 63 for example are the numbers of squares in a chess board.

Ok. I've read about that years ago. People where using trees, it's more complicated than an array, but more appropriate for chess I guess.

[fxm]

.....
Arrays

• Fixed- and variable- length arrays are supported, up to 2 GB in size.

Have you tested, with the compiler option -exx, the right to writting/reading in your big array up to the end addresses (and that the right data are memorized)?
If it works, the documentation should be updated to suppress or modify this barrier of 2 GB (if a limit remains due to the compiler).

[RockTheSchock]

It's less about the capabilities of FB but more about which type of memory is used to store the array. With ReDim an array is always dynamic. With only Dim it depends!

With qb or fblite dialekt there is also '$Dynamic or Option Dynamic to force Dim to use dynamic memory segment.

Static memory should be a bit faster at least when allocating. Thats only a guess.

from https://software.intel.com/en-us/articl ... ns-windows

32-bit
Static data - 2GB
Dynamic data - 2GB
Stack data - 1GB (the stack size is set by the linker, the default is 1MB. This can be increased using the Linker property System > Stack Reserve Size)

Note that on 32-bit Windows, the sum of all types of data must be 2GB or less. The practical limit is about 1.75GB due to space used by Windows itself

64-bit

Static data - 2GB
Dynamic data - 8TB
Stack data - 1GB (the stack size is set by the linker, the default is 1MB. This can be increased using the Linker property System > Stack Reserve Size)

[MichaelW]

The memory limits vary with the version of Windows, see Memory Limits for Windows and Windows Server Releases

[Provoni]

The memory limits vary with the version of Windows, see Memory Limits for Windows and Windows Server Releases

So under Win 8 Pro 64-bit it's possible to allocate arrays up to 512gb with FreeBASIC 64-bit?

[MichaelW]

Running on my cheap 64-bit, 4GB laptop under Windows 8.1 (probably the base edition), whether I test a dynamic array or memory allocated from the CRT (I think memory for dynamic arrays is also allocated from the CRT), I get a limit of ~15GB.

Code: Select all

dim as longint x
for i as longint = 1000000 to 512000000000 step 1000000 
    print  i * 8
    redim as longint a(i)
    x = a(i)    
next    
sleep

Code: Select all

#include "crt.bi"
dim as longint x
dim as longint ptr p
for i as longint = 1000000 to 512000000000 step 1000000 
    p = malloc(i*8)
    if p = 0 then exit for
    x = p[i-8]
    print i*8
    free(p)
next    
sleep

[fxm]

(I think memory for dynamic arrays is also allocated from the CRT)

Yes I also think this, as for (C)ALLOCATE and NEW:

Code: Select all

dim as longint x
dim as longint ptr p
for i as longint = 1000000 to 512000000000 step 1000000
    p = allocate(i*8)
    if p = 0 then exit for
    x = p[i-1]
    print  i*8
    deallocate(p)
next    
sleep

Code: Select all

dim as longint x
dim as longint ptr p
for i as longint = 1000000 to 512000000000 step 1000000
    p = new longint[i] {any}
    if p = 0 then exit for
    x = p[i-1]
    print  i*8
    delete [] p
next    
sleep