Pure FB Runtime Library (in progress)
Re: Pure FB Runtime Library (in progress)
Working yet? I wanted to try.
Re: Pure FB Runtime Library (in progress)
There's might be a problem with this that's causing the compiler to complain and crash. This is on fbrt master (so none of the string changes) with a build of the compiler from current mastercoderJeff wrote: And if all that is done, then I kind of think that if the rtlib function doesn't actually operate on FBSTRING structure internals, then could potentially be declared as a "normal" function returning STRING.
For instance:
Code: Select all
#include "C:/fbc/src/fbrt/fb.bi"
dim as any ptr kernel32 = DyLibLoad("kernel32")
The 64 bit compiler crashes with a windows "fbc has stopped working" so it doesn't print the crash messageC:\shared\FBSamples>%fbc32% -c array.bas
array.bas(3) error 58: Type mismatch, at parameter 1 of FB_STRALLOCTEMPDESCZEX()
in 'dim as any ptr kernel32 = DyLibLoad("kernel32")'
Aborting due to runtime error 12 ("segmentation violation" signal)
C:\shared\FBSamples>%fbc64% -c array.bas
array.bas(3) error 58: Type mismatch, at parameter 1 of FB_STRALLOCTEMPDESCZEX()
in 'dim as any ptr kernel32 = DyLibLoad("kernel32")'
C:\shared\FBSamples>
Comment out the include and it compiles fine.
It happens with using SWAP for the string descriptors too, which is why that's done with the new fb_StrSwapDesc because swap wouldn't compile
Code: Select all
#include "C:/fbc/src/fbrt/fb.bi"
dim as FBSTRING one, two
SWAP one, two
C:\shared\FBSamples>%fbc64% -c array.bas
array.bas(4) error 58: Type mismatch, at parameter 1 of FB_MEMSWAP() in 'SWAP on
e, two'
Re: Pure FB Runtime Library (in progress)
Yeah, that makes sense. Just wishful thinking on an old exchange in this thread:adeyblue wrote:There's might be a problem with this that's causing the compiler to complain and crash. This is on fbrt master (so none of the string changes) with a build of the compiler from current mastercoderJeff wrote: And if all that is done, then I kind of think that if the rtlib function doesn't actually operate on FBSTRING structure internals, then could potentially be declared as a "normal" function returning STRING.
viewtopic.php?p=237909&237909#p237909
viewtopic.php?p=237929&237929#p237929
To use the STRING type, the compiler needs to be in control of any calls to functions in the rtlib, since the compiler is going to translate the user visible STRING type to the internal FBSTRING type under the hood. I have a feeling this will extend to many fbc built-ins if tried to use directly within the rtlib. For example using OPEN directly to read a file.
I think could get this to work with some tricky naming and aliasing of prototypes.
- on fbc side
- every string function gets a new name but is aliased to the original rtlib function
- e.g. declare function fbc_fb_StrAllocTempDescZEx alias "fb_StrAllocTempDescZEx" ( byval str as const zstring ptr, byval len as const integer ) as string
- on rtlib side
- don't #undef built-ins in the rtlib headers
- declare / define the internal version using the rtlib internal types.
- e.g. declare function fb_StrAllocTempDescZEx FBCALL ( str as const ubyte ptr, len as ssize_t ) as FBSTRING ptr
Re: Pure FB Runtime Library (in progress)
Close, but no dice. gcc backend craps out.
Fake fbc side:
Fake rtlib side:
Fake fbc side:
Code: Select all
declare function fbc_F alias "F" ( byref s as string, byref result as string ) as string
'' note, 'result' should be generated internal to fbc only
dim as string x, ret, result
x = "hello"
'' for some built-in 'F', compiler calls 'fbc_F'
ret = fbc_F(x, result)
? result
'' careful, because of current fbc internals
'' both 'ret' and 'result' point to the same
'' string data even though they are different
'' descriptors (i.e. fbc will try to double free)
Code: Select all
declare function fbc_F alias "F" ( byref s as string, byref result as string ) as string
type fbstring
data as ubyte ptr
len_ as integer
size as integer
end type
function F alias "F" ( byval s as fbstring ptr, byval result as fbstring ptr ) as fbstring ptr
print "data = 0x" & hex(s->data)
print "len_ = " & s->len_
print "size = " & s->size
'' danger! - no checks for valid data or errors
result->data = callocate( s->size * 2 )
fb_memcopy( *result->data, *s->data, s->len_ )
fb_memcopy( *(result->data + s->len_), *s->data, s->len_ )
result->data[ s->len_ + s->len_ ] = 0
result->len_ = s->len_ * 2
result->size = s->size * 2
return result
end function
/' -- uncomment to see the gcc errors
sub someRtlibProc()
'' All good until here, except can't actually use fbc_F()
'' due the incompatable types in gcc backend
fbc_F( "", "" )
end sub
'/
Re: Pure FB Runtime Library (in progress)
Eh, if it's a non-zero amount of work. then I wouldn't bother. Not to just be able to prettify one project.
Considering fb.bi makes it not compile, it'd probably work OK if the files had finer grained includes rather than all of them including everything. But again, that'd be a lot of work just to prettify it
Considering fb.bi makes it not compile, it'd probably work OK if the files had finer grained includes rather than all of them including everything. But again, that'd be a lot of work just to prettify it
Re: Pure FB Runtime Library (in progress)
It's not zero work. But it is close if I maybe handle through a mangle modifier, so it's not starting from scratch either. Maybe another time I will look at it again, because assembly could handle it without issue.
Re: Pure FB Runtime Library (in progress)
I finished out the rest of the modifications,
removed all the temp descriptior deletions
deleted all the string lock functions and macros
and updated all of the usages of the changed functions in rtlib
https://github.com/adeyblue/fbrtLib/commits/new_strings
Since temporary descriptors aren't a thing anymore, I renamed the str_tempdescf/v/z.bas files to remove the 'temp' (like I did the function names within). This has caused the makefile to start compiling the str_tempdescf/v/z.c files from the C rtlib. The only file (for Windows) that's needed from the C rtlib now is file_datetime.c because crt/stat.bi causes issues when included with fb.bi. Maybe that behaviour can be disabled or turned off on a per-platform basis? Everything else in the makefile blacklist compiles fine.
Anyway, I'm not sure which functions the compiler uses or generates calls to, so I kept a rather repetitive list of which functions declares changed.
If the compiler ever digs into the FBSTRING structure, I did make one behavioural change. The StrAlloc functions that don't allocate and own the data pointer (basically, the StrAllocDesc functions), set the size member to 0 instead of what they used to (len is unchanged). This hasn't lost any information (they all just set size to the same value as len) but it does mean any FBSTRING can be passed as the result parameter now regardless of what it holds and the right thing will happen.
Without this there'd still need to be a way to distinguish between those FBSTRINGs that need actually DeAllocating and those that don't because of the descriptor swapping and destructable_string.
removed all the temp descriptior deletions
deleted all the string lock functions and macros
and updated all of the usages of the changed functions in rtlib
https://github.com/adeyblue/fbrtLib/commits/new_strings
Since temporary descriptors aren't a thing anymore, I renamed the str_tempdescf/v/z.bas files to remove the 'temp' (like I did the function names within). This has caused the makefile to start compiling the str_tempdescf/v/z.c files from the C rtlib. The only file (for Windows) that's needed from the C rtlib now is file_datetime.c because crt/stat.bi causes issues when included with fb.bi. Maybe that behaviour can be disabled or turned off on a per-platform basis? Everything else in the makefile blacklist compiles fine.
Anyway, I'm not sure which functions the compiler uses or generates calls to, so I kept a rather repetitive list of which functions declares changed.
Code: Select all
fb_hStrAllocTmpDesc - deleted
fb_hStrFreeTmpDesc - deleted
fb_hStrAllocTemp_NoLock - deleted
fb_hStrAllocTemp - deleted
fb_hStrDelTemp_NoLock - deleted
fb_hStrDelTemp - deleted
fb_StrAllocTempResult - deleted
fb_StrSwapDesc - new
fb_ConReadLine - added string output
fb_Dir - added string output
fb_Dir64 - added string output
fb_DirNext - added string output
fb_DirNext64 - added string output
fb_FileStrInput - added string output
fb_inkey - add output parameter to end of arg list
fb_StrDelete - Updated to not free static strings
fb_ctx.hooks.inkeyproc() add output parameter
fb_IntlGetMonthName - added string output
fb_IntlGetWeekdayName - added string output
fb_hMakeInkeyStr - added string output
fb_CHR - output param added as first param as its a vararg function
fb_LCASE - added string output
fb_UCASE - added string output
fb_InkeyQB - added string output
fb_IntToStrQB - added string output
fb_UintToStrQB - added string output
fb_DoubleToStrQB - added string output
fb_FloatToStrQB - added string output
fb_UlongIntToStrQB - added string output
fb_LongIntToStrQB - added string output
fb_WstrToStr - added string output
fb_BIN, fb_BINEx_l, fb_BINEx_p - added string output
fb_OCT, fb_OCTEx_l, fb_OCTEx_p - added string output
fb_HEX, fb_HEXEx_l, fb_HEXEx_p - added string output
fb_CurDir - added string output
fb_Command - added string output
fb_GetEnviron - added string output
fb_ExePath - added string output
fb_Date - added string output
fb_MonthName - added string output
fb_Time - added string output
fb_WeekdayName - added string output
fb_IntToStr, fb_UIntToStr - added string output
fb_BoolToStr - added string output
fb_FloatToStr - added string output
fb_DoubleToStr - added string output
fb_LongintToStr - added string output
fb_ULongintToStr - added string output
fb_MK? - added string output
fb_StrFill1 - added string output
fb_StrFill2 - added string output
fb_StrFormat - added string output
fb_hStrFormat - added string output
fb_StrLCase2 - added string output
fb_LEFT - added string output
fb_LTRIM - added string output
fb_LTRIMAny - added string output
fb_LTRIMEx - added string output
fb_StrMid - added string output
fb_Right - added string output
fb_RTRIM - added string output
fb_RTRIMAny - added string output
fb_RTRIMEx - added string output
fb_TRIM - added string output
fb_TRIMAny - added string output
fb_TRIMEx - added string output
fb_StrAllocTempDescF - renamed to fb_StrAllocDescF, added string output
fb_StrAllocTempDescV - renamed to fb_StrAllocDescV, added string output
fb_StrAllocTempDescZEx - renamed to fb_StrAllocDescZEx, added string output
fb_StrAllocTempDescZ - renamed to fb_StrAllocDescZ, added string output
Win32\
fb_DrvIntlGetMonthName - added string output
fb_DrvIntlGetWeekdayName - added string output
_GetLocaleString - added function, de-duplicates common ending from above two functions
fb_hIntlConvertString - added string output
fb_Dir - added string output
unix\
fb_DrvIntlGetMonthName - added string output
fb_DrvIntlGetWeekdayName - added string output
_GetLocaleString - added function, de-duplicates common ending from above two functions
fb_Dir - added string output
Without this there'd still need to be a way to distinguish between those FBSTRINGs that need actually DeAllocating and those that don't because of the descriptor swapping and destructable_string.
Re: Pure FB Runtime Library (in progress)
Hi adeyblue.
Will the proposed changes affect the fast len method which I use in my quicksorts?.
Will the proposed changes affect the fast len method which I use in my quicksorts?.
Code: Select all
dim as string a="abcdefghijklnmopqrstuvwxyz (and a merry Christmas)"
dim as integer l
dim as double t
dim as long looplen =100000000
t=timer
for n as long=1 to looplen
l= Cast(Integer Ptr,@a)[1]
next
print timer-t,l
t=timer
for n as long=1 to looplen
l= len(a)
next
print timer-t,l
sleep
Re: Pure FB Runtime Library (in progress)
Should be OK, size is the third member not the second.dodicat wrote:Hi adeyblue.
Will the proposed changes affect the fast len method which I use in my quicksorts?.
I don't know anything about how the compiler transforms FB statements into rtlib calls, but if it's possible for temporary string descriptors to escape rtlib into user code, then that code won't work with them. Temporary descriptors set the high bit of the string length to indicate the temporary-ness so that code would return negative length strings.
Code: Select all
fb_string.h
/** Flag to identify a string as a temporary string.
*
* This flag is stored in struct _FBSTRING::len so it's absolutely required
* to use FB_STRSIZE(s) to query a strings length.
*/
#ifdef HOST_64BIT
#define FB_TEMPSTRBIT ((long long)0x8000000000000000ll)
#else
#define FB_TEMPSTRBIT ((int)0x80000000)
#endif
This sort of thing though (changes to how things work internally) is exactly why that sort of thing (relying on internal representations and mechanisms) is generally a bad idea. If these internal things are documented on the wiki (I don't know if it is), then it's the sort of thing that shouldn't be IMO.
Re: Pure FB Runtime Library (in progress)
The most precise in the documentation can be only found in the Programmer's Guide : Strings (string, zstring, and wstring)
but with an example accessing the number of characters allocated in memory for the string (3rd field of the descriptor)......(string referenced by an internal descriptor of 1 pointer + 2 uinteger length)
Re: Pure FB Runtime Library (in progress)
Is the temporary string flag placed on the second (len) or third (size) string descriptor structure member ?
Re: Pure FB Runtime Library (in progress)
For temporary strings, the most significant bit is set on the 'len' member which is the second member of the string descriptor type.fxm wrote:Is the temporary string flag placed on the second (len) or third (size) string descriptor structure member ?
In normal user code you should never see the temporary bit set. The temporary bit gets cleared by the time the string is returned to a descriptor that the user can access.
Code: Select all
function salut() as string
'' temporary string returned
return "salut"
end function
sub proc( byref s as string )
print s
'' hack to get length member
print hex( cast( integer ptr, @s)[1], sizeof(integer)*2 )
end sub
'' under the hood, a temporary descriptor is allocated to hold the result of salut() and pass to proc()
'' when the temporary string is assigned to the temporary descriptor, the temporary bit is cleared.
proc( salut() )
Re: Pure FB Runtime Library (in progress)
Finally, I thus found there an explanation on the maximum size of variable strings: 2 ^ (31 or 63) - 1coderJeff wrote:For temporary strings, the most significant bit is set on the 'len' member which is the second member of the string descriptor type.
:-)
Re: Pure FB Runtime Library (in progress)
With hacking code, we can see the temporary string flag set on the len field:coderJeff wrote:For temporary strings, the most significant bit is set on the 'len' member which is the second member of the string descriptor type.
In normal user code you should never see the temporary bit set. The temporary bit gets cleared by the time the string is returned to a descriptor that the user can access.
Code: Select all
function salut() as string '' temporary string returned return "salut" end function sub proc( byref s as string ) print s '' hack to get length member print hex( cast( integer ptr, @s)[1], sizeof(integer)*2 ) end sub '' under the hood, a temporary descriptor is allocated to hold the result of salut() and pass to proc() '' when the temporary string is assigned to the temporary descriptor, the temporary bit is cleared. proc( salut() )
Code: Select all
function salut() as string
'' temporary string returned
return "salut"
end function
dim f as function() as any ptr = cast(function() as any ptr, @salut)
print *cptr(string ptr, f()) '' salut
print hex(cptr(uinteger ptr, f())[1], sizeof(uinteger) * 2) '' 8000...005
print hex(cptr(uinteger ptr, f())[2], sizeof(uinteger) * 2) '' 0000...020
sleep
Re: Pure FB Runtime Library (in progress)
Clever. I challenge your notion of normal! :Pfxm wrote:With hacking code, we can see the temporary string flag set on the len field:
Calling the function through a differently typed pointer disregards all the usual memory management that fbc will add for string descriptors and string data. Careful of the memory leaks. Every time f() is called, a new temporary string descriptor and data is allocated. If you are going to use this hack in any code, then save the pointer, or pass it to an rtlib function to automatically clean it up.
Code: Select all
function salut() as string
'' temporary string returned
return "salut"
end function
dim f as function() as any ptr = cast(function() as any ptr, @salut)
dim as STRING ptr tmp = cptr(string ptr, f())
'' passing a temporary string to PRINT will automatically delete it
'' print *tmp '' salut
'' print it as a zstring instead
print *cptr(zstring ptr, cptr(uinteger ptr, tmp)[0]) '' salut
print hex(cptr(uinteger ptr, tmp)[1], sizeof(uinteger) * 2) '' 8000...005
print hex(cptr(uinteger ptr, tmp)[2], sizeof(uinteger) * 2) '' 0000...020
'' clean-up the temporary (if we didn't already pass it to some other rlib function)
fb_hStrDelTemp( *tmp )