Shared 3.9GB array too big in FB64bits with 8GB RAM

New to FreeBASIC? Post your questions here.
Luis Babboni
Posts: 303
Joined: Mar 15, 2015 12:41

Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby Luis Babboni » Jul 21, 2015 19:03

Thats!

I want to do with FB 64 bits in my Windows 7 64 bits with 8GB RAM this:

Code: Select all

Dim Shared P(1 To 10, 1 To 20, 1 To 20, 0 To 63, 0 To 63, 0 To 63) As UInteger<32>


Thats I understood have 3,9GB size:
10*20^2*64^3*4 / (1024^3) = 3,90625

And when try to compile FBEdit says me:
"...error 49: Array too big..."

Any hint about why I can´t do it?

Thanks.
Tourist Trap
Posts: 2815
Joined: Jun 02, 2015 16:24

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby Tourist Trap » Jul 21, 2015 19:20

[Edit] I count 33554432000/8 bytes, or approximatively 34/8 Gigabytes. Storing a so big amount of data is very rarely needed, unless you store hi-res multimedia, what is this array for?
Last edited by Tourist Trap on Jul 21, 2015 19:54, edited 1 time in total.
RockTheSchock
Posts: 226
Joined: Mar 12, 2006 16:25

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby RockTheSchock » Jul 21, 2015 19:29

Code: Select all

reDim Shared P(1 To 10, 1 To 20, 1 To 20, 0 To 63, 0 To 63, 0 To 63) As UInteger<32>
fxm
Posts: 9529
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby fxm » Jul 21, 2015 19:45

See at /wiki/CompilerFeatures:
.....
Arrays
  • Fixed- and variable- length arrays are supported, up to 2 GB in size.
  • Up to eight dimensions, including arrays with unknown dimensions.
  • Any lower and upper boundaries.
  • Element data can be preserved during a re-size of variable-length arrays with Redim using the new Preserve specifier.
Luis Babboni
Posts: 303
Joined: Mar 15, 2015 12:41

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby Luis Babboni » Jul 21, 2015 19:46

Thanks Rock! Now works! :-)

Why you count as much Tourist?
I puted my calculation in my post. UInterger<32> has 4 bytes I undestood.
Tourist Trap
Posts: 2815
Joined: Jun 02, 2015 16:24

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby Tourist Trap » Jul 21, 2015 19:52

Luis Babboni wrote:Thanks Rock! Now works! :-)

Why you count as much Tourist?
I puted my calculation in my post. UInterger<32> has 4 bytes I undestood.

Right sorry. But you dont have to divide by 1024, you're already in octet=byte from what I understand.
Your array remains impressive.
Luis Babboni
Posts: 303
Joined: Mar 15, 2015 12:41

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby Luis Babboni » Jul 21, 2015 20:12

I read what fxm said but still do not understood what kind of risk I have using REDIM instead of Dim :-/
I could lost the values?
As far as I understood, all I read about "preserve" is referred to use ReDim with previous Dimed array, I´m right?
So if you use ReDim to define an array for the first time, you do not need to be worried about lost data?
But not sure then why to not use always ReDim instead of Dim.... is cause that´s about variable lenghts arrays I still do not understand?
Wait.... if I have no risk using ReDim , please do not waste your time trying to explain me more... at least for the moment. :-D

Tourist, I´m trying to do some stats about chess, those 0 to 63 for example are the numbers of squares in a chess board.
Tourist Trap
Posts: 2815
Joined: Jun 02, 2015 16:24

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby Tourist Trap » Jul 21, 2015 20:18

Luis Babboni wrote:Wait.... if I have no risk using ReDim , please do not waste your time trying to explain me more... at least for the moment. :-D

From what I've always understood: use redim when you declare a variable-length array. Use redim preserve each time you change the array if you don't want it reinitialized.
Luis Babboni wrote:Tourist, I´m trying to do some stats about chess, those 0 to 63 for example are the numbers of squares in a chess board.

Ok. I've read about that years ago. People where using trees, it's more complicated than an array, but more appropriate for chess I guess.
fxm
Posts: 9529
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby fxm » Jul 22, 2015 9:06

http://www.freebasic.net/wiki/CompilerFeatures wrote:.....
Arrays
  • Fixed- and variable- length arrays are supported, up to 2 GB in size.

Have you tested, with the compiler option -exx, the right to writting/reading in your big array up to the end addresses (and that the right data are memorized)?
If it works, the documentation should be updated to suppress or modify this barrier of 2 GB (if a limit remains due to the compiler).
RockTheSchock
Posts: 226
Joined: Mar 12, 2006 16:25

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby RockTheSchock » Jul 22, 2015 9:36

It's less about the capabilities of FB but more about which type of memory is used to store the array. With ReDim an array is always dynamic. With only Dim it depends!

With qb or fblite dialekt there is also '$Dynamic or Option Dynamic to force Dim to use dynamic memory segment.

Static memory should be a bit faster at least when allocating. Thats only a guess.

from https://software.intel.com/en-us/articl ... ns-windows
32-bit
Static data - 2GB
Dynamic data - 2GB
Stack data - 1GB (the stack size is set by the linker, the default is 1MB. This can be increased using the Linker property System > Stack Reserve Size)

Note that on 32-bit Windows, the sum of all types of data must be 2GB or less. The practical limit is about 1.75GB due to space used by Windows itself

64-bit

Static data - 2GB
Dynamic data - 8TB
Stack data - 1GB (the stack size is set by the linker, the default is 1MB. This can be increased using the Linker property System > Stack Reserve Size)
MichaelW
Posts: 3500
Joined: May 16, 2006 22:34
Location: USA

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby MichaelW » Aug 08, 2015 2:27

The memory limits vary with the version of Windows, see Memory Limits for Windows and Windows Server Releases
Provoni
Posts: 347
Joined: Jan 05, 2014 12:33
Location: Belgium

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby Provoni » Aug 22, 2015 18:47

MichaelW wrote:The memory limits vary with the version of Windows, see Memory Limits for Windows and Windows Server Releases

So under Win 8 Pro 64-bit it's possible to allocate arrays up to 512gb with FreeBASIC 64-bit?
MichaelW
Posts: 3500
Joined: May 16, 2006 22:34
Location: USA

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby MichaelW » Aug 23, 2015 4:33

Running on my cheap 64-bit, 4GB laptop under Windows 8.1 (probably the base edition), whether I test a dynamic array or memory allocated from the CRT (I think memory for dynamic arrays is also allocated from the CRT), I get a limit of ~15GB.

Code: Select all

dim as longint x
for i as longint = 1000000 to 512000000000 step 1000000
    print  i * 8
    redim as longint a(i)
    x = a(i)   
next   
sleep

Code: Select all

#include "crt.bi"
dim as longint x
dim as longint ptr p
for i as longint = 1000000 to 512000000000 step 1000000
    p = malloc(i*8)
    if p = 0 then exit for
    x = p[i-8]
    print i*8
    free(p)
next   
sleep
fxm
Posts: 9529
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Shared 3.9GB array too big in FB64bits with 8GB RAM

Postby fxm » Aug 23, 2015 6:56

MichaelW wrote:(I think memory for dynamic arrays is also allocated from the CRT)

Yes I also think this, as for (C)ALLOCATE and NEW:

Code: Select all

dim as longint x
dim as longint ptr p
for i as longint = 1000000 to 512000000000 step 1000000
    p = allocate(i*8)
    if p = 0 then exit for
    x = p[i-1]
    print  i*8
    deallocate(p)
next   
sleep

Code: Select all

dim as longint x
dim as longint ptr p
for i as longint = 1000000 to 512000000000 step 1000000
    p = new longint[i] {any}
    if p = 0 then exit for
    x = p[i-1]
    print  i*8
    delete [] p
next   
sleep

Return to “Beginners”

Who is online

Users browsing this forum: No registered users and 2 guests