Freebasic 1.20.0 Development

General discussion for topics related to the FreeBASIC project or its community.
Post Reply
Lost Zergling
Posts: 577
Joined: Dec 02, 2011 22:51
Location: France

Re: Freebasic 1.20.0 Development

Post by Lost Zergling »

I am interested and surprised by this behavior where REALLOCATE fails, but another separate ALLOCATE (with the same size) succeeds ?
This was an old test, i'll try to repeat it. Did not remember exactly fb version.
Context : I needed a pre allocated zstring available for being used very fast (no need to reallocate)
The size shall be known, logically fixed, but with possible resize to bigger size (never smaller)
The zstring is in a class.
The zstring resize request is inside a property of this class.
As the property was used many and many times, random crashes occured.
Replacing reallocate by allocate + swap did solve the problem. (Smell like a sticky hack)
=> I supposed the new allocated could find available memory outside the class wheras the reallocate was trying finding space into the same area inside class definition ??
( Or reallocate did break class desc integrity)

Usage : you handle a list of key-values, but.., at one moment, because an error or anything else, the key is becoming huge just for a few keys, and this 'exception' shall be handled as a standard key, for robustness.
( ou sinon, c'est la clef qui pète la classe, .. ;- )
Last edited by Lost Zergling on Jun 18, 2024 20:34, edited 1 time in total.
fxm
Moderator
Posts: 12238
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Freebasic 1.20.0 Development

Post by fxm »

Lost Zergling wrote: Jun 18, 2024 19:18 I supposed the new allocated could find available memory outside the class wheras the reallocate was trying finding space into the same area inside class definition ??
(Or reallocate did break class desc integrity)

REALLOCATE first attempts to resize the memory already allocated without moving its starting address in order to avoid recopying the data to be kept (returned pointer value unchanged).
If this is not possible, a new separate memory area is allocated with recopy of the data to be kept before deallocating the old memory area (returned pointer value modified).

So I assume you are using in the property body something like:
This.myPtr = Reallocate(This.myPtr, ...)
and not only:
Reallocate(This.myPtr, ...)
Lost Zergling
Posts: 577
Joined: Dec 02, 2011 22:51
Location: France

Re: Freebasic 1.20.0 Development

Post by Lost Zergling »

Well I don't remember exactly but probably was this.myptr=reallocate..
shadow008
Posts: 90
Joined: Nov 26, 2013 2:43

Re: Freebasic 1.20.0 Development

Post by shadow008 »

Lost Zergling wrote: Jun 18, 2024 20:37 Well I don't remember exactly but probably was this.myptr=reallocate..
Use of reallocate in situations where memory integrity is guaranteed requires a bit more than just that. Specifically you need to assign the reallocate result to a temporary pointer and check the result. If it failed, you do not assign your this.myptr (or whatever) to the temp value or you will have a dangling pointer which is a memory leak.
This is how to properly use reallocate.

Code: Select all

dim tmp as any ptr = reallocate(...)
if tmp then
    this.myptr = tmp
endif
Lost Zergling
Posts: 577
Joined: Dec 02, 2011 22:51
Location: France

Re: Freebasic 1.20.0 Development

Post by Lost Zergling »

That's the issue...but not a leak issue.
No fail due to intrinsic scopes or memory fragmentation issues shall be admitted.
Admitted fail shall only be running out of memory. Would require new test...allocate+swap work.
Replacing allocate+swap by reallocate is not a problem, very small benefit expected.
fxm
Moderator
Posts: 12238
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Freebasic 1.20.0 Development

Post by fxm »

Example of exhaustive testing after REALLOCATE:

Code: Select all

Sub resize(Byval p As Any Ptr, Byval size As Longint)
    Dim As Any Ptr tp = Reallocate(p, size)
    If tp > 0 Then
        Print "REALLOCATE succeeded for " & size & " bytes"
        If tp <> p Then
            Print "memory moved"
'            Deallocate(myPtr)  '' No because previous memory already deallocated under the hood
            p = tp
        Else
            Print "memory not moved"
        End If
    Else
        Print "REALLOCATE failed for " & size & " bytes"
        Deallocate(p)  '' Yes otherwise memory leak
        End
    End If
End Sub

Dim As Any Ptr myPtr = Allocate(10)

resize(myPtr, 9)
Print
resize(myPtr, 1000000)
Print
resize(myPtr, 1000000000000)

Deallocate(myPtr)  '' Yes otherwise memory leak
Sleep
REALLOCATE succeeded for 9 bytes
memory not moved

REALLOCATE succeeded for 1000000 bytes
memory moved

REALLOCATE failed for 1000000000000 bytes
fxm
Moderator
Posts: 12238
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Freebasic 1.20.0 Development

Post by fxm »

Lost Zergling wrote: Jun 18, 2024 19:18 Context : I needed a pre allocated zstring available for being used very fast (no need to reallocate)
The size shall be known, logically fixed, but with possible resize to bigger size (never smaller)
The zstring is in a class.
The zstring resize request is inside a property of this class.
As the property was used many and many times, random crashes occured.
Replacing reallocate by allocate + swap did solve the problem. (Smell like a sticky hack)
=> I supposed .....
( Or reallocate did break class desc integrity)

I have just reworded (in more detail as for REDIM) in the documentation the special case where REALLOCATE use inside a member procedure can induce malfunction if the pointer argument (of REALLOCATE) points to the instance itself of the object (on which the member procedure is called):
In REALLOCATE documentation page, the 'note' wrote: Reallocating a pointer inside an object function, when that pointer contains the parent object of the function, is undefined, and will likely result in horrible crashes.
Reallocate cannot be used inside a member procedure if pointer points to the instance itself of the object, because that could cause horrible crashes:
- If pointer is modified by Reallocate (object data moved into memory), the passed This reference becomes inconsistent, in the same way as a dangling pointer.
- In that case, all subsequent accesses to any non-static member field from this member procedure will be erroneous, except if the passed This reference would be readjusted (by means of @This = new pointer value) immediately after executing Reallocate in the body of this member procedure.
Very simple example for REALLOCATE (with @THIS readjustment if necessary):

Code: Select all

Type UDT
    number As Uinteger
    Declare Sub resize(Byval size As Integer)
End Type

Dim Shared pU As UDT Ptr = 0

Sub UDT.resize(Byval size As Integer)
    pU = Reallocate(pU, size * Len(UDT))
    If pU <> @This Then
        Print "memory moved for " & size & " instances"
        @This = pU  '' mandatory when memory is moved, for future use of non-static member field in procedure body
    Else
        Print "memory not moved for " & size & " instances"
    End If
    This.number = size
End Sub


pU = Allocate(5 * Len(UDT))
pU->number = 5

pU->resize(4)
Print pU->number
Print
pU->resize(1000000)
Print pU->number

Deallocate(pU)
Sleep
memory not moved for 4 instances
4

memory moved for 1000000 instances
1000000

Similar example for REDIM [PRESERVE] (with @THIS readjustment if necessary):

Code: Select all

Type UDT
    number As Uinteger
    Declare Sub resize(Byval size As Integer)
End Type

Dim Shared U() As UDT

Sub UDT.resize(Byval size As Integer)
    ReDim  U(size - 1)
    If @U(0) <> @This Then
        Print "memory moved for " & size & " elements"
        @This = @U(0)  '' mandatory when memory is moved, for future use of member field in procedure body
    Else
        Print "memory not moved for " & size & " elements"
    End If
    This.number = size
End Sub


ReDim U(4)
U(0).number = 5

U(0).resize(4)
Print U(0).number
Print
U(0).resize(1000000)
Print U(0).number

Erase U
Sleep
memory not moved for 4 elements
4

memory moved for 1000000 elements
1000000
Last edited by fxm on Jun 19, 2024 18:11, edited 5 times in total.
Reason: Added similar example for REDIM [PRESERVE]
fxm
Moderator
Posts: 12238
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Freebasic 1.20.0 Development

Post by fxm »

Lost Zergling wrote: Jun 19, 2024 5:35 Replacing allocate+swap by reallocate is not a problem, very small benefit expected.

But, concept consequence in a member procedural body:
- something like (1 line):
This.myPtr = Reallocate(This.myPtr, ...)
- instead of (4 lines):
Dim As Any Ptr p = Allocate(...)
Swap This.myPtr, p
FB_memcopy(Byval This.myPtr, Byval p, ...)
Deallocate(p)
Lost Zergling
Posts: 577
Joined: Dec 02, 2011 22:51
Location: France

Re: Freebasic 1.20.0 Development

Post by Lost Zergling »

Noted. Thank you.
Nb : in my use case, I do not need to keep previous value. 'Reallocation' is just for size.
fxm
Moderator
Posts: 12238
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Freebasic 1.20.0 Development

Post by fxm »

Lost Zergling wrote: Jun 19, 2024 13:36 Nb : in my use case, I do not need to keep previous value. 'Reallocation' is just for size.

In your specific case (no data to keep), the 3 lines:
Dim As Any Ptr p = Allocate(...)
Swap This.myPtr, p
Deallocate(p)

are at execution faster or equal to the single line:
This.myPtr = Reallocate(This.myPtr, ...)
Lost Zergling
Posts: 577
Joined: Dec 02, 2011 22:51
Location: France

Re: Freebasic 1.20.0 Development

Post by Lost Zergling »

@fxm. As you have understand it, this section using swap is critical, handling scalability while trying to keep safe from memory fragmentation.
Ps added :
In retrospect, my post may seem quite pretentious. At the time and in the context in which I wrote it, I mainly had in mind the fact that my code was particularly difficult to read and a concern about the sustainability of the 'swap' kwd, there was not an intention to against you from me.
Post Reply