Pure FB Runtime Library (in progress)

User projects written in or related to FreeBASIC.
Post Reply
miilvyxg
Posts: 193
Joined: Dec 07, 2021 6:51

Re: Pure FB Runtime Library (in progress)

Post by miilvyxg »

Working yet? I wanted to try.
adeyblue
Posts: 299
Joined: Nov 07, 2019 20:08

Re: Pure FB Runtime Library (in progress)

Post by adeyblue »

coderJeff wrote: And if all that is done, then I kind of think that if the rtlib function doesn't actually operate on FBSTRING structure internals, then could potentially be declared as a "normal" function returning STRING.
There's might be a problem with this that's causing the compiler to complain and crash. This is on fbrt master (so none of the string changes) with a build of the compiler from current master

For instance:

Code: Select all

#include "C:/fbc/src/fbrt/fb.bi"

dim as any ptr kernel32 = DyLibLoad("kernel32")
C:\shared\FBSamples>%fbc32% -c array.bas
array.bas(3) error 58: Type mismatch, at parameter 1 of FB_STRALLOCTEMPDESCZEX()
in 'dim as any ptr kernel32 = DyLibLoad("kernel32")'

Aborting due to runtime error 12 ("segmentation violation" signal)

C:\shared\FBSamples>%fbc64% -c array.bas
array.bas(3) error 58: Type mismatch, at parameter 1 of FB_STRALLOCTEMPDESCZEX()
in 'dim as any ptr kernel32 = DyLibLoad("kernel32")'

C:\shared\FBSamples>
The 64 bit compiler crashes with a windows "fbc has stopped working" so it doesn't print the crash message

Comment out the include and it compiles fine.

It happens with using SWAP for the string descriptors too, which is why that's done with the new fb_StrSwapDesc because swap wouldn't compile

Code: Select all

#include "C:/fbc/src/fbrt/fb.bi"

dim as FBSTRING one, two
SWAP one, two
C:\shared\FBSamples>%fbc64% -c array.bas
array.bas(4) error 58: Type mismatch, at parameter 1 of FB_MEMSWAP() in 'SWAP on
e, two'
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Pure FB Runtime Library (in progress)

Post by coderJeff »

adeyblue wrote:
coderJeff wrote: And if all that is done, then I kind of think that if the rtlib function doesn't actually operate on FBSTRING structure internals, then could potentially be declared as a "normal" function returning STRING.
There's might be a problem with this that's causing the compiler to complain and crash. This is on fbrt master (so none of the string changes) with a build of the compiler from current master
Yeah, that makes sense. Just wishful thinking on an old exchange in this thread:
viewtopic.php?p=237909&237909#p237909
viewtopic.php?p=237929&237929#p237929

To use the STRING type, the compiler needs to be in control of any calls to functions in the rtlib, since the compiler is going to translate the user visible STRING type to the internal FBSTRING type under the hood. I have a feeling this will extend to many fbc built-ins if tried to use directly within the rtlib. For example using OPEN directly to read a file.

I think could get this to work with some tricky naming and aliasing of prototypes.

- on fbc side
- every string function gets a new name but is aliased to the original rtlib function
- e.g. declare function fbc_fb_StrAllocTempDescZEx alias "fb_StrAllocTempDescZEx" ( byval str as const zstring ptr, byval len as const integer ) as string

- on rtlib side
- don't #undef built-ins in the rtlib headers
- declare / define the internal version using the rtlib internal types.
- e.g. declare function fb_StrAllocTempDescZEx FBCALL ( str as const ubyte ptr, len as ssize_t ) as FBSTRING ptr
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Pure FB Runtime Library (in progress)

Post by coderJeff »

Close, but no dice. gcc backend craps out.

Fake fbc side:

Code: Select all

declare function fbc_F alias "F" ( byref s as string, byref result as string ) as string

'' note, 'result' should be generated internal to fbc only
dim as string x, ret, result

x = "hello"

'' for some built-in 'F', compiler calls 'fbc_F'

ret = fbc_F(x, result)

? result

'' careful, because of current fbc internals
'' both 'ret' and 'result' point to the same
'' string data even though they are different
'' descriptors (i.e. fbc will try to double free)
Fake rtlib side:

Code: Select all

declare function fbc_F alias "F" ( byref s as string, byref result as string ) as string

type fbstring
	data as ubyte ptr
	len_ as integer
	size as integer
end type

function F alias "F" ( byval s as fbstring ptr, byval result as fbstring ptr ) as fbstring ptr

	print "data = 0x" & hex(s->data)
	print "len_ = " & s->len_
	print "size = " & s->size

	'' danger! - no checks for valid data or errors

	result->data = callocate( s->size * 2 )
	fb_memcopy( *result->data, *s->data, s->len_ )
	fb_memcopy( *(result->data + s->len_), *s->data, s->len_ )
	result->data[ s->len_ + s->len_ ] = 0
	result->len_ = s->len_ * 2
	result->size = s->size * 2
	return result

end function


/' -- uncomment to see the gcc errors
sub someRtlibProc()
	'' All good until here, except can't actually use fbc_F()
	'' due the incompatable types in gcc backend
	fbc_F( "", "" )
end sub
'/
adeyblue
Posts: 299
Joined: Nov 07, 2019 20:08

Re: Pure FB Runtime Library (in progress)

Post by adeyblue »

Eh, if it's a non-zero amount of work. then I wouldn't bother. Not to just be able to prettify one project.
Considering fb.bi makes it not compile, it'd probably work OK if the files had finer grained includes rather than all of them including everything. But again, that'd be a lot of work just to prettify it
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Pure FB Runtime Library (in progress)

Post by coderJeff »

It's not zero work. But it is close if I maybe handle through a mangle modifier, so it's not starting from scratch either. Maybe another time I will look at it again, because assembly could handle it without issue.
adeyblue
Posts: 299
Joined: Nov 07, 2019 20:08

Re: Pure FB Runtime Library (in progress)

Post by adeyblue »

I finished out the rest of the modifications,
removed all the temp descriptior deletions
deleted all the string lock functions and macros
and updated all of the usages of the changed functions in rtlib

https://github.com/adeyblue/fbrtLib/commits/new_strings

Since temporary descriptors aren't a thing anymore, I renamed the str_tempdescf/v/z.bas files to remove the 'temp' (like I did the function names within). This has caused the makefile to start compiling the str_tempdescf/v/z.c files from the C rtlib. The only file (for Windows) that's needed from the C rtlib now is file_datetime.c because crt/stat.bi causes issues when included with fb.bi. Maybe that behaviour can be disabled or turned off on a per-platform basis? Everything else in the makefile blacklist compiles fine.

Anyway, I'm not sure which functions the compiler uses or generates calls to, so I kept a rather repetitive list of which functions declares changed.

Code: Select all

fb_hStrAllocTmpDesc - deleted
fb_hStrFreeTmpDesc - deleted
fb_hStrAllocTemp_NoLock - deleted
fb_hStrAllocTemp - deleted
fb_hStrDelTemp_NoLock - deleted
fb_hStrDelTemp - deleted
fb_StrAllocTempResult - deleted

fb_StrSwapDesc - new

fb_ConReadLine - added string output
fb_Dir - added string output
fb_Dir64 - added string output
fb_DirNext - added string output
fb_DirNext64 - added string output
fb_FileStrInput - added string output
fb_inkey - add output parameter to end of arg list
fb_StrDelete - Updated to not free static strings
fb_ctx.hooks.inkeyproc() add output parameter
fb_IntlGetMonthName - added string output
fb_IntlGetWeekdayName - added string output
fb_hMakeInkeyStr - added string output
fb_CHR - output param added as first param as its a vararg function
fb_LCASE - added string output
fb_UCASE - added string output
fb_InkeyQB - added string output
fb_IntToStrQB - added string output
fb_UintToStrQB - added string output
fb_DoubleToStrQB - added string output
fb_FloatToStrQB - added string output
fb_UlongIntToStrQB - added string output
fb_LongIntToStrQB - added string output
fb_WstrToStr - added string output
fb_BIN, fb_BINEx_l, fb_BINEx_p - added string output
fb_OCT, fb_OCTEx_l, fb_OCTEx_p - added string output
fb_HEX, fb_HEXEx_l, fb_HEXEx_p - added string output
fb_CurDir - added string output
fb_Command - added string output
fb_GetEnviron - added string output
fb_ExePath - added string output
fb_Date - added string output
fb_MonthName - added string output
fb_Time - added string output
fb_WeekdayName - added string output
fb_IntToStr,  fb_UIntToStr - added string output
fb_BoolToStr - added string output
fb_FloatToStr - added string output
fb_DoubleToStr - added string output
fb_LongintToStr - added string output
fb_ULongintToStr - added string output
fb_MK? - added string output
fb_StrFill1 - added string output
fb_StrFill2 - added string output
fb_StrFormat - added string output
fb_hStrFormat - added string output
fb_StrLCase2 - added string output
fb_LEFT - added string output
fb_LTRIM - added string output
fb_LTRIMAny - added string output
fb_LTRIMEx - added string output
fb_StrMid - added string output
fb_Right - added string output
fb_RTRIM - added string output
fb_RTRIMAny - added string output
fb_RTRIMEx - added string output
fb_TRIM - added string output
fb_TRIMAny - added string output
fb_TRIMEx - added string output
fb_StrAllocTempDescF - renamed to fb_StrAllocDescF, added string output
fb_StrAllocTempDescV - renamed to fb_StrAllocDescV, added string output
fb_StrAllocTempDescZEx - renamed to fb_StrAllocDescZEx, added string output
fb_StrAllocTempDescZ - renamed to fb_StrAllocDescZ, added string output

Win32\
fb_DrvIntlGetMonthName - added string output
fb_DrvIntlGetWeekdayName - added string output
_GetLocaleString - added function, de-duplicates common ending from above two functions
fb_hIntlConvertString - added string output
fb_Dir - added string output

unix\
fb_DrvIntlGetMonthName - added string output
fb_DrvIntlGetWeekdayName - added string output
_GetLocaleString - added function, de-duplicates common ending from above two functions
fb_Dir - added string output
If the compiler ever digs into the FBSTRING structure, I did make one behavioural change. The StrAlloc functions that don't allocate and own the data pointer (basically, the StrAllocDesc functions), set the size member to 0 instead of what they used to (len is unchanged). This hasn't lost any information (they all just set size to the same value as len) but it does mean any FBSTRING can be passed as the result parameter now regardless of what it holds and the right thing will happen.

Without this there'd still need to be a way to distinguish between those FBSTRINGs that need actually DeAllocating and those that don't because of the descriptor swapping and destructable_string.
dodicat
Posts: 7976
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Pure FB Runtime Library (in progress)

Post by dodicat »

Hi adeyblue.
Will the proposed changes affect the fast len method which I use in my quicksorts?.

Code: Select all


dim as string a="abcdefghijklnmopqrstuvwxyz (and a merry Christmas)"
dim as integer l
dim as double t
dim as long looplen =100000000

t=timer
for n as long=1 to looplen
l= Cast(Integer Ptr,@a)[1]
next
print timer-t,l
t=timer
for n as long=1 to looplen
l= len(a)
next
print timer-t,l
sleep
 
adeyblue
Posts: 299
Joined: Nov 07, 2019 20:08

Re: Pure FB Runtime Library (in progress)

Post by adeyblue »

dodicat wrote:Hi adeyblue.
Will the proposed changes affect the fast len method which I use in my quicksorts?.
Should be OK, size is the third member not the second.

I don't know anything about how the compiler transforms FB statements into rtlib calls, but if it's possible for temporary string descriptors to escape rtlib into user code, then that code won't work with them. Temporary descriptors set the high bit of the string length to indicate the temporary-ness so that code would return negative length strings.

Code: Select all

fb_string.h
/** Flag to identify a string as a temporary string.
 *
 * This flag is stored in struct _FBSTRING::len so it's absolutely required
 * to use FB_STRSIZE(s) to query a strings length.
 */
#ifdef HOST_64BIT
	#define FB_TEMPSTRBIT ((long long)0x8000000000000000ll)
#else
	#define FB_TEMPSTRBIT ((int)0x80000000)
#endif
Ironically, this is part of the stuff that I just removed, so it's a problem you may face currently, but not in the new order.

This sort of thing though (changes to how things work internally) is exactly why that sort of thing (relying on internal representations and mechanisms) is generally a bad idea. If these internal things are documented on the wiki (I don't know if it is), then it's the sort of thing that shouldn't be IMO.
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Pure FB Runtime Library (in progress)

Post by fxm »

The most precise in the documentation can be only found in the Programmer's Guide : Strings (string, zstring, and wstring)
.....(string referenced by an internal descriptor of 1 pointer + 2 uinteger length)
but with an example accessing the number of characters allocated in memory for the string (3rd field of the descriptor).
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Pure FB Runtime Library (in progress)

Post by fxm »

Is the temporary string flag placed on the second (len) or third (size) string descriptor structure member ?
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Pure FB Runtime Library (in progress)

Post by coderJeff »

fxm wrote:Is the temporary string flag placed on the second (len) or third (size) string descriptor structure member ?
For temporary strings, the most significant bit is set on the 'len' member which is the second member of the string descriptor type.

In normal user code you should never see the temporary bit set. The temporary bit gets cleared by the time the string is returned to a descriptor that the user can access.

Code: Select all

function salut() as string
	'' temporary string returned 
	return "salut"
end function

sub proc( byref s as string )
	print s
	'' hack to get length member
	print hex( cast( integer ptr, @s)[1], sizeof(integer)*2 )
end sub

'' under the hood, a temporary descriptor is allocated to hold the result of salut() and pass to proc()
'' when the temporary string is assigned to the temporary descriptor, the temporary bit is cleared.

proc( salut() )
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Pure FB Runtime Library (in progress)

Post by fxm »

coderJeff wrote:For temporary strings, the most significant bit is set on the 'len' member which is the second member of the string descriptor type.
Finally, I thus found there an explanation on the maximum size of variable strings: 2 ^ (31 or 63) - 1
:-)
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Pure FB Runtime Library (in progress)

Post by fxm »

coderJeff wrote:For temporary strings, the most significant bit is set on the 'len' member which is the second member of the string descriptor type.

In normal user code you should never see the temporary bit set. The temporary bit gets cleared by the time the string is returned to a descriptor that the user can access.

Code: Select all

function salut() as string
	'' temporary string returned 
	return "salut"
end function

sub proc( byref s as string )
	print s
	'' hack to get length member
	print hex( cast( integer ptr, @s)[1], sizeof(integer)*2 )
end sub

'' under the hood, a temporary descriptor is allocated to hold the result of salut() and pass to proc()
'' when the temporary string is assigned to the temporary descriptor, the temporary bit is cleared.

proc( salut() )
With hacking code, we can see the temporary string flag set on the len field:

Code: Select all

function salut() as string
   '' temporary string returned
   return "salut"
end function

dim f as function() as any ptr = cast(function() as any ptr, @salut)

print *cptr(string ptr, f())                                 '' salut
print hex(cptr(uinteger ptr, f())[1], sizeof(uinteger) * 2)  '' 8000...005
print hex(cptr(uinteger ptr, f())[2], sizeof(uinteger) * 2)  '' 0000...020

sleep
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Pure FB Runtime Library (in progress)

Post by coderJeff »

fxm wrote:With hacking code, we can see the temporary string flag set on the len field:
Clever. I challenge your notion of normal! :P

Calling the function through a differently typed pointer disregards all the usual memory management that fbc will add for string descriptors and string data. Careful of the memory leaks. Every time f() is called, a new temporary string descriptor and data is allocated. If you are going to use this hack in any code, then save the pointer, or pass it to an rtlib function to automatically clean it up.

Code: Select all

function salut() as string
   '' temporary string returned
   return "salut"
end function

dim f as function() as any ptr = cast(function() as any ptr, @salut)

dim as STRING ptr tmp =  cptr(string ptr, f())

'' passing a temporary string to PRINT will automatically delete it
'' print *tmp '' salut

'' print it as a zstring instead
print *cptr(zstring ptr, cptr(uinteger ptr, tmp)[0])         '' salut                                                   

print hex(cptr(uinteger ptr, tmp)[1], sizeof(uinteger) * 2)  '' 8000...005
print hex(cptr(uinteger ptr, tmp)[2], sizeof(uinteger) * 2)  '' 0000...020

'' clean-up the temporary (if we didn't already pass it to some other rlib function)
fb_hStrDelTemp( *tmp )
Post Reply