Array Descriptor (split from Wiki Improvements)

[Lost Zergling]

@dodicat. Thank you.

[coderJeff]

OK, it's declared outside of namespace "FBC", thanks, should have seen that myself ...

Actually, I would like to have it in the namespace, which I tried in an earlier test.

For concept, I would like, conveniently, to be able to do something like:

Code: Select all

namespace FBC
	'' ...
	extern "rtlib"
		declare function ArrayDescriptorPtr alias "fb_ArrayGetDesc" ( array() as any ) as FBARRAY ptr
	end extern
end namespace

The intent is that from the compiler's point of view, the symbol would be in the FBC namespace, but from the linker's point of view, link to a regular C function in rtlib.

Alas, we don't have anything currently that makes it convenient, due the default cdecl/stdcall depending on the extern block, and nothing to built in to follow FBCALL as in the rtlib. So currently would need to add a number of compile time conditionals, maybe another header. Another path for further development...

[coderJeff]

@dodicat. Most code in your runtime tester might be very close to what I was looking for to improve lzae (overload redundancy, udts handling). It is simple & elegant.
#macro setptr(datatype)
...

Sure thing Lost Zerling, good luck with your project.

@Lost Zergling, thank's for pointing that out.

@dodicat, sorry, I missed all what you were doing there ...

I like the setptr() code generator, giving a compile-time type check and typed pointer return. Very nice.

With the fbc-int/array.bi include that we are playing around with this could be something like:

Code: Select all

#include "fbc-int/array.bi"

#macro setptr(datatype)
function arrayptr overload(a() As datatype) As datatype ptr
	return fb_ArrayGetDesc( a() )->base_ptr
end function
#endmacro	

A couple things I noticed:
- the fb_ArrayGetDesc() won't allow a const array, it could allow const & non-const if it takes a 'array() const any' parameter
- Maybe separate function name for const arrays, returning a const pointer would be safer
- because having both const and non-const array in not allowed for overload in fbc anyway (duplicate defintion)

Otherwise, can't get much better, I think, other than adding ARRAYPTR() keyword to fbc compiler itself.

Thanks, dodicat.

[Juergen Kuehlwein]

@Jeff,

should i start preparing a pull request for my array features with what we have now, or should i wait for you to merge #169 ?


JK

[coderJeff]

@Jeff,

should i start preparing a pull request for my array features with what we have now, or should i wait for you to merge #169 ?

I am almost done with #169:
- exploring the extern "rtlib" idea just now. Not specifically needed for array descriptor, however in the long term it helps with the extension of ./inc/fbc-int/ API's and other feature requests from users in general.
- returning an appropriate pointer for const arrays, and adding the tests for it.

After those 2 things I will update the PR and merge it in.

So, you could probably start working on a PR, knowing that you (or I) will have to rebase after #169 gets merged in. And just for fair warning, I have not reviewed your array code extensively, but it does appear that you are adding many new things, so as usual will pick apart and analyze each component.

[MrSwiss]

- returning an appropriate pointer for const arrays

I'm actually more interested in: appropriate pointer for "dynamic" array, because I'll
want to check that, in my procedure(s), requiring "resizable" array, as argument.
(it doesn't even need to be dimensioned/sized, when passed)

[Juergen Kuehlwein]

@Jeff,

thanks, i will wait for the merge and i will try to break it into digestable commits.

BTW did you notice my reply to your comments on #158 ? Having a solution for this problem is not a requirement, but it would make my additions more complete.


@MrSwiss

There will be a method of detecting dynamic arrays vs. fixed size arrays too.


JK

[coderJeff]

BTW did you notice my reply to your comments on #158 ? Having a solution for this problem is not a requirement, but it would make my additions more complete.

I did notice your comments. Sorry, I just haven't had time to go in to it deeper. I like that you attack problems head on to arrive at a solution, and I understand what you are going for as an end result. My initial reaction to the fbc code changes is that I'd like to look a little deeper to see the other effects that may not be immediately obvious. There are some related bugs, and a related note in 'TODO.txt' file, so there is a larger picture I would like to examine before merging in the changes as-is.

[coderJeff]

I added the extern "rtlib" feature.

For that discussion, please go to Extern "rtlib"

[coderJeff]

The next step is to add the new "flags" field to the array descriptor and break everything.

The new flags field for the array descriptor is merged in to fbc/master. When compiling fbc from sources, I recommend building it from a previous release, and then rebuilding it again with itself.

I've updated documentation at Arrays in the wiki.

Please keep in mind, this makes it possible to fix the ERASE/REDIM related bugs, however, no fixes have yet been applied.

New, in this topic, from what we have already discussed is:

Code: Select all

namespace FBC
extern "rtlib"
	declare function ArrayDescriptorPtr alias "fb_ArrayGetDesc" _
		( array() as any ) as FBC.FBARRAY ptr
	declare function ArrayConstDescriptorPtr alias "fb_ArrayGetDesc" _
		( array() as const any ) as const FBC.FBARRAY ptr
end extern
end namespace

For example, to determine if array is fixed-length or variable-length in a subroutine:

Code: Select all

#include "fbc-int/array.bi"

sub proc( array() as integer )
	dim ap as const FBC.FBARRAY ptr = FBC.ArrayConstDescriptorPtr( array() ) 
	if( (ap->flags and FBC.FBARRAY_FLAGS_FIXED_LEN) <> 0 ) then
		print "fixed-length"
	else
		print "variable-length"
	end if 
end sub

dim a(1 to 10) as integer
print "a() : "; : proc( a() )

dim b() as integer
print "b() : "; : proc( b() )

redim c(1 to 10) as integer 
print "c() : "; : proc( c() )

OUTPUT:

Code: Select all

a() : fixed-length
b() : variable-length
c() : variable-length

[fxm]

Why not replace:

Code: Select all

Const FB_MAXDIMENSIONS As Integer = 8

Type FBARRAYDIM
    Dim As UInteger elements     '' number of elements
    Dim As Integer LBound        '' dimension lower bound
    Dim As Integer UBound        '' dimension upper bound
End Type

Const FBARRAY_FLAGS_DIMENSIONS = &h0000000f    '' number of entries allocated in dimTb()
Const FBARRAY_FLAGS_FIXED_DIM  = &h00000010    '' array has fixed number of dimensions
Const FBARRAY_FLAGS_FIXED_LEN  = &h00000020    '' array points to fixed-length memory
Const FBARRAY_FLAGS_RESERVED   = &hffffffc0    '' reserved, do not use

Type FBARRAY
    Dim As Any Ptr index_ptr     '' @array(0, 0, 0, ... )
    Dim As Any Ptr base_ptr      '' start of memory at array lowest bounds
    Dim As UInteger size         '' byte size of allocated contents
    Dim As UInteger element_len  '' byte size of single element
    Dim As UInteger dimensions   '' number of dimensions
    Dim As UInteger flags        '' FBARRAY_FLAGS_*

    '' take care with number of dimensions; fbc may allocate
    '' a smaller descriptor with fewer than FB_MAXDIMENSIONS
    '' in dimTb() if it is known at compile time that they
    '' are never needed.  Always respect number of
    '' dimensions when accessing dimTb()

    Dim As FBARRAYDIM dimTb(0 To FB_MAXDIMENSIONS-1)
End Type

with:

Code: Select all

Const FB_MAXDIMENSIONS As Integer = 8

Type FBARRAYDIM
    Dim As UInteger elements     '' number of elements
    Dim As Integer LBound        '' dimension lower bound
    Dim As Integer UBound        '' dimension upper bound
End Type

Type FBARRAY
    Dim As Any Ptr index_ptr               '' @array(0, 0, 0, ... )
    Dim As Any Ptr base_ptr                '' start of memory at array lowest bounds
    Dim As UInteger size                   '' byte size of allocated contents
    Dim As UInteger element_len            '' byte size of single element
    Dim As UInteger dimensions             '' number of dimensions
    Dim As Uinteger FLAGS_DIMENSIONS :  4  '' number of entries allocated in dimTb()
    Dim As Uinteger FLAGS_FIXED_DIM  :  1  '' array has fixed number of dimensions
    Dim As Uinteger FLAGS_FIXED_LEN  :  1  '' array points to fixed-length memory
    Dim As Uinteger FLAGS_RESERVED   : 26  '' reserved, do not use

    '' take care with number of dimensions; fbc may allocate
    '' a smaller descriptor with fewer than FB_MAXDIMENSIONS
    '' in dimTb() if it is known at compile time that they
    '' are never needed.  Always respect number of
    '' dimensions when accessing dimTb()

    Dim As FBARRAYDIM dimTb(0 To FB_MAXDIMENSIONS-1)
End Type

[Juergen Kuehlwein]

@Jeff,


thanks for pushing thing forward!

Sorry, I just haven't had time to go in to it deeper. I like that you attack problems head on to arrive at a solution, and I understand what you are going for as an end result. My initial reaction to the fbc code changes is that I'd like to look a little deeper to see the other effects that may not be immediately obvious. There are some related bugs, and a related note in 'TODO.txt' file, so there is a larger picture I would like to examine before merging in the changes as-is.

Everyone can see, that you are very busy currently, so take your time. I just wanted to make sure, that you see my point, why it should be a run time solution and not a compile time solution. As usual this PR is a proposal, it shows a possible way for fixing the bug. As far as i can tell from own tests, it fixes the bug without raising other problems. I don´t know what to do with fixed lenght strings (STRING * ...), so i left it as it is (you still cannot pass arrays of fixed length strings, but z/wstring arrays work). Feel free to modify the code as needed.


JK

[MrSwiss]

The Array Descriptor works correctly, as of today (not so, with yesterdays build).
For anybody wanting to test it, get the latest FBC 1.08.0-DEV build from: here.

Then run the test-code below:

Code: Select all

' NewArrayFeat-test2.bas -- (c) 2019-09-02, MrSwiss
'
' ATTENTION: needs FBC 1.08.0-DEV, St_W build: >= 2019-09-03 !!!
'
' compile: -s console
'
#Include "fbc-int/array.bi"

#Macro SetTypeArrDynInfo(DataType)      ' define datatype for overloaded Function
Function ArrDynInfo OverLoad( _         ' check for dynamic array
    a() As DataType _
    ) As Boolean                        ' TRUE = dynamic | FALSE = static
    Dim As FBC.FBARRAY Ptr pa = _
        FBC.ArrayDescriptorPtr(a())     ' get array descriptor ptr (pa)

    If (pa->flags And FBC.FBARRAY_FLAGS_FIXED_LEN) _    ' mask 'static' = &h20
        Then Return FALSE               ' if TRUE = static (return FALSE)
    Return TRUE                         ' otherwise: dynamic (return TRUE)
End Function
#EndMacro


Dim As String   sa(), sa1(9)            ' sa() = dynamic | sa1(9) = static (10 elements)
SetTypeArrDynInfo(String)               ' define datatype for overloaded Function

If ArrDynInfo(sa()) Then                ' check for dynamic array (is dynamic)
    Print "sa() is dynamic"
Else
    Print "sa() is static"
End If

If ArrDynInfo(sa1()) Then               ' check for dynamic array (is static)
    Print "sa1() is dynamic"
Else
    Print "sa1() is static"
End If

Sleep

[fxm]

The Array Descriptor works correctly, as of today (not so, with yesterdays build).
For anybody wanting to test it, get the latest FBC 1.08.0-DEV build from: here

For people who make recurring revisons from this site, do not forget to also take into account the latest version of the file "freebasic_additional_headers.zip".

[coderJeff]

Why not replace:
....

Partly old habits, partly because I think that bitfields generate unnecessary code due the masking and shifting when assigning or accessing. I haven't looked the asm generation for bitfields in years, though.

The advantage of bitfields is that it makes for much cleaner interface. Disadvantage, I guess, would accessing all bits at once. So I would perhaps still have full flags field, but in a union with the bitfields.