Array Descriptor (split from Wiki Improvements)
Re: Array Descriptor (split from Wiki Improvements)
The size of the array descriptor may depend on the number of dimensions.
If you increase the number of dimensions, the descriptor should be reallocated in memory, which should often change its address, but change the referencing of an array is not possible.
I think that only if you declare the dynamic array as:
{Dim|Static} [Shared] As datatype array()
(without any indication of sizing)
the allocated descriptor is sized for the maximum number of dimensions (8).
If you increase the number of dimensions, the descriptor should be reallocated in memory, which should often change its address, but change the referencing of an array is not possible.
I think that only if you declare the dynamic array as:
{Dim|Static} [Shared] As datatype array()
(without any indication of sizing)
the allocated descriptor is sized for the maximum number of dimensions (8).
-
- Posts: 284
- Joined: Mar 07, 2018 13:59
- Location: Germany
Re: Array Descriptor (split from Wiki Improvements)
Yes, that´s true. Fixed sized arrays get allocated a descriptor only for the dimensions needed. Dynamic arrays get always allocated room for 8 dimensions (and i would leave it this way at any rate). Currently the RTL cannot distinguish fixed sized from dynamic arrays. Therefore it doesn´t allow dynamic arrays (once dimmed) to increase their number of dimensions with REDIM(). Once knowing fixed size or not, this issue can be addressed too.
JK
JK
Re: Array Descriptor (split from Wiki Improvements)
But if you declare the dynamic array as:fxm wrote:I think that only if you declare the dynamic array as:
{Dim|Static} [Shared] As datatype array()
(without any indication of sizing)
the allocated descriptor is sized for the maximum number of dimensions (8).
{Dim|Static} [Shared] As datatype array(Any, Any)
or (1st declaration):
Redim [Shared] As datatype array(1, 2)
the allocated array descriptor is sized for only 2 dimensions.
-
- Posts: 284
- Joined: Mar 07, 2018 13:59
- Location: Germany
Re: Array Descriptor (split from Wiki Improvements)
Code: Select all
... As datatype array(Any, Any)
But for a dynamic array
Code: Select all
... As datatype array ()
Code: Select all
REDIM array(10)
...
REDIM array(10, 10)
...
REDIM array(10, 10, 10)
JK
Re: Array Descriptor (split from Wiki Improvements)
For this to be complete, we must also allow in UDTs:
- 'As datatype array()' in UDTs (in this case, the descriptor is allocated for 8 dimensions),
- in addition to 'As datatype array(Any, ...)' (where the allocated descriptor is sized depending on the number of 'Any' declared),
- and also 'Redim As datatype array(...)' (where the allocated descriptor is sized depending on the number of dimensions declared through the bounds).
- 'As datatype array()' in UDTs (in this case, the descriptor is allocated for 8 dimensions),
- in addition to 'As datatype array(Any, ...)' (where the allocated descriptor is sized depending on the number of 'Any' declared),
- and also 'Redim As datatype array(...)' (where the allocated descriptor is sized depending on the number of dimensions declared through the bounds).
Re: Array Descriptor (split from Wiki Improvements)
With -gen gas you can undefine a variable then re define it again.
For example you could undefine an array with one dimension then redefine it again with two dimensions.
But not with -gen gcc
Same goes for a simple variable.
However you can still undefine a label then re-define it again with -gen gcc.
(The gcc errors are cryptic!)
For example you could undefine an array with one dimension then redefine it again with two dimensions.
But not with -gen gcc
Same goes for a simple variable.
However you can still undefine a label then re-define it again with -gen gcc.
Code: Select all
print"hello ";
if false then goto label: else print"hello ";
label:
print
#undef label
print "Goodbye"
if true then goto label: else print "Goodbye"
label:
print
#undef label
dim as integer i=6
print i
#undef i
dim as integer i=5
print i
sleep
Re: Array Descriptor (split from Wiki Improvements)
OK, so we have a number of array related bugs reported on sf.net, plus some new ones reported in this topic, and maybe some others hidden in other topics on the forum.
First step, I think, is to expose the existing array descriptor for developers and advanced users. This will give us a record of our starting point before we attempt to fix bugs and add some features that JK is proposing.
- in github PR #165 I have added a very small API to expose the array descriptor in a relatively stable way. It also kind of establishes the pattern I would like to see if we expand on this idea in future adding other API's that expose fbc internals.
- I added a wiki page DevArrays in the developer section to document array and array descriptor behaviours from the point of view of the developer. It gives us a place to record all the details of arrays that don't fit in the reference or programmer's guide.
So, a very simple example of usage for fbc-int/array.bi
I've merged the changes in to fbc/master as it won't affect any existing code. I appreciate any comments on the addition.
- in __FB_LANG__="fb" declaring the new symbols in namespace FBC, thus reserving "FBC" for internals. This maybe is a poor choice as we are already using "fbc as fbcctx" in the compiler sources, so we have to rename one of "fbc" symbol names.
- header in a separate directory "./inc/fbc-int" which will be for internal stuff, but exposed to the user.
First step, I think, is to expose the existing array descriptor for developers and advanced users. This will give us a record of our starting point before we attempt to fix bugs and add some features that JK is proposing.
- in github PR #165 I have added a very small API to expose the array descriptor in a relatively stable way. It also kind of establishes the pattern I would like to see if we expand on this idea in future adding other API's that expose fbc internals.
- I added a wiki page DevArrays in the developer section to document array and array descriptor behaviours from the point of view of the developer. It gives us a place to record all the details of arrays that don't fit in the reference or programmer's guide.
So, a very simple example of usage for fbc-int/array.bi
Code: Select all
#include "fbc-int/array.bi"
#macro printArrayDesc( title, array )
scope
dim ap as FBC.FBARRAY ptr = fb_ArrayGetDesc( array )
print title
print , "index_ptr : " & hex( cint( ap->index_ptr ) )
print , "base_ptr : " & hex( cint( ap->base_ptr ) )
print , "size : " & ap->size
print , "element_len: " & ap->element_len
print , "dimensions : " & ap->dimensions
for i as integer = 0 to ap->dimensions-1
print , "dimTb(" & i & ").elements: " & ap->dimTb(i).elements
print , "dimTb(" & i & ").lbound : " & ap->dimTb(i).lbound
print , "dimTb(" & i & ").ubound : " & ap->dimTb(i).ubound
next
end scope
#endmacro
dim a() as integer
printArrayDesc( "a() as integer", a() )
dim b(1 to 2, 3 to 5, 7 to 13) as byte
printArrayDesc( "b() as byte", b() )
- in __FB_LANG__="fb" declaring the new symbols in namespace FBC, thus reserving "FBC" for internals. This maybe is a poor choice as we are already using "fbc as fbcctx" in the compiler sources, so we have to rename one of "fbc" symbol names.
- header in a separate directory "./inc/fbc-int" which will be for internal stuff, but exposed to the user.
Re: Array Descriptor (split from Wiki Improvements)
I like the approach: nicely, step by step
The naming used currently is however, not very clear (ambiguous/misleading).
FBC-int -- C's 32 bit signed integer data type, comes to mind instantly, instead of referencing 'internal'.
(not considering the name of the namespace proposed)
FBintern -- whould clear the issue ... just a idea, up for discussion.
The naming used currently is however, not very clear (ambiguous/misleading).
FBC-int -- C's 32 bit signed integer data type, comes to mind instantly, instead of referencing 'internal'.
(not considering the name of the namespace proposed)
FBintern -- whould clear the issue ... just a idea, up for discussion.
Re: Array Descriptor (split from Wiki Improvements)
Thanks, MrSwiss. Yes, the changes I added immediately create 2 issues that I will lazily defer to later to resolve.MrSwiss wrote: FBintern -- whould clear the issue ... just a idea, up for discussion.
1) namespace FBC, I really would like to reserve the FBC namespace for compiler internals, so, should the need arise, I will rename the 'fbc' variable in fbc compiler sources to 'fbcompiler' to resolve the conflict. It's not needed just now, but if we ever use './inc/fbc-int/*.bi' directly in compiler sources, this is my solution.
2) __FB_LANG__="??". As it stands, the API will have namespace FBC in lang "fb", no namespace in lang "fblite", and fail in lang "qb" due the differences in integer sizes for lang "qb". I'm going to leave the solution open ended for now. The solution probably involves creating a "./inc/fbc-int/types.bi" kind of include. Though, for comparison, "./inc/fbgfx.bi" deals with namespace FB in the same way as current addition for array descriptor API, so basically I don't know what is best long term solution, at the moment.
Thank-you. Your comment helps to justify the time I will take to explain each step we take on this development path. Nice to know community is listening. :)I like the approach: nicely, step by step
-
- Posts: 284
- Joined: Mar 07, 2018 13:59
- Location: Germany
Re: Array Descriptor (split from Wiki Improvements)
@Jeff,
so you decided to expose the descriptor itself!
Ok, no problem - this makes some of my functions obsolete, because you now can take the officially declared FBARRAY type.
As already said above when you expand the FBARRAY type, add one or more "reserved" members. This gives room for future ideas, without breaking compatibility with every new step.
For the additions i wrote, i need at least one bit field as a flag for one of the features. It could of course be an entire member variable to be used as a flag then too.
I appreciate your initiative in this area!
JK
so you decided to expose the descriptor itself!
Ok, no problem - this makes some of my functions obsolete, because you now can take the officially declared FBARRAY type.
As already said above when you expand the FBARRAY type, add one or more "reserved" members. This gives room for future ideas, without breaking compatibility with every new step.
For the additions i wrote, i need at least one bit field as a flag for one of the features. It could of course be an entire member variable to be used as a flag then too.
I appreciate your initiative in this area!
JK
-
- Posts: 606
- Joined: Nov 28, 2012 1:27
- Location: CA, USA moving to WA, USA
- Contact:
Re: Array Descriptor (split from Wiki Improvements)
Does this mean that all arrays will have descriptors, exposed and available, in this future version?
david
david
Re: Array Descriptor (split from Wiki Improvements)
Yes, I read through the code for your proposed additions. From your code and other comments in this topic, the "flags" field is looking like the following (so far):Juergen Kuehlwein wrote:As already said above when you expand the FBARRAY type, add one or more "reserved" members. This gives room for future ideas, without breaking compatibility with every new step.
Code: Select all
enum FBARRAY_FLAGS
FBARRAY_FLAGS_DIMENSIONS = &h0000000f '' number of entries allocated in dimTb()
FBARRAY_FLAGS_STATIC = &h00000010 '' array points to static/fixed-length memory
FBARRAY_FLAGS_ATTACHED = &h00000020 '' array points to memory managed by another descriptor
FBARRAY_FLAGS_LOCKED = &h00000040 '' array in use: do not allow redim/erase
FBARRAY_FLAGS_RESERVED = &hffffff80 '' reserved, do not use
end enum
fbc will allocate the smallest array descriptor possible, if it can determine the dimension count at compile time otherwise it allocates room for FB_MAXDIMENSIONS. Haven't decided on this one as the compiler and runtime are currently set up to try and dimension an array once and then not allow number of dimensions to change. Allowing number of dimensions to change during run time is counter productive to compile time array bounds checks and advice noted in the sources with respect to dimensions. Needs more thought
FBARRAY_STATIC:
if the descriptor points to data with static allocation, then REDIM should be disallowed. ERASE could still be used to clear the array. This is the first flag that I will be focusing on.
FBARRAY_ATTACHED:
this is a feature I saw in your proposed code where an array descriptor can point to arbitrary data, basically providing an array(index, ...) interface, rather than cast( datatype ptr, @array(0) ) + index ... pointer maths. Have to be careful with this one, especially with complex UDT's. It might look like a safe interface, but has all the dangers of pointers under the hood. So may want to think of incorporating some help from the compiler. Most simple syntax would be something like "var byref a() = b()" to create an alias to an existing array, plus other extensions for different dimensions or data types.
FBARRAY_LOCKED:
not sure about this one, based on a comment in this topic, though the intent would be that REDIM should return a run-time error, if array is "locked". Why an array would be locked; yet to be defined.
FBARRAY_RESERVED:
don't use.
Re: Array Descriptor (split from Wiki Improvements)
Yes, that is the intent. Plus, also trying to create a bit of a structure for other additions, like string descriptor, or file i/o, or whatever. To note, I am trying to keep a little separation from the main language, to give continued development half a chance, rather than immediately locked in for next 10 years.. i.e. the separate ./inc/fbc-int folder and documented in the developer section.speedfixer wrote:Does this mean that all arrays will have descriptors, exposed and available, in this future version?
david
Also, "all arrays will have descriptors" is true only if the array is passed as argument to a procedure. This includes the procedure to get the array descriptor itself. Technically, there are some cases where arrays won't have descriptors because fbc optimizes them out, but if the descriptor is never accessed, the user will never know.
Re: Array Descriptor (split from Wiki Improvements)
yes, new field will be uinteger, just like all the other fields being one of integer, uinteger, or pointer.fxm wrote:I hope this will be an Integer (32/64 bits) like all other fields, so 32 bits useful?coderJeff wrote:The field will have 32 bits available for array descriptor options.
Where exactly do you intend to add this field in the array descriptor structure?
(perhaps just before the number of dimensions)
For new code it probably doesn't matter exactly where the new "flags" field is placed. In my tests I have placed it just after the dimensions field. Is placing it just before better?
Re: Array Descriptor (split from Wiki Improvements)
A remark a little off-topic about the syntax to use to designate an array (in its entirety):
- We must use either 'array()' or 'array' (without parentheses) depending on the keyword used.
- This constraint is a nuisance when one wants to define a macro with such a parameter, because it is not known (to my knowledge) how to truncate a macro parameter of its final parentheses (only argument concatenation exists).
- We must use either 'array()' or 'array' (without parentheses) depending on the keyword used.
- This constraint is a nuisance when one wants to define a macro with such a parameter, because it is not known (to my knowledge) how to truncate a macro parameter of its final parentheses (only argument concatenation exists).