Array Descriptor (split from Wiki Improvements)

Forum for discussion about the documentation project.
fxm
Posts: 9123
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Array Descriptor (split from Wiki Improvements)

Postby fxm » Aug 20, 2019 13:16

The size of the array descriptor may depend on the number of dimensions.
If you increase the number of dimensions, the descriptor should be reallocated in memory, which should often change its address, but change the referencing of an array is not possible.

I think that only if you declare the dynamic array as:
{Dim|Static} [Shared] As datatype array()
(without any indication of sizing)
the allocated descriptor is sized for the maximum number of dimensions (8).
Juergen Kuehlwein
Posts: 210
Joined: Mar 07, 2018 13:59
Location: Germany

Re: Array Descriptor (split from Wiki Improvements)

Postby Juergen Kuehlwein » Aug 20, 2019 15:20

Yes, that´s true. Fixed sized arrays get allocated a descriptor only for the dimensions needed. Dynamic arrays get always allocated room for 8 dimensions (and i would leave it this way at any rate). Currently the RTL cannot distinguish fixed sized from dynamic arrays. Therefore it doesn´t allow dynamic arrays (once dimmed) to increase their number of dimensions with REDIM(). Once knowing fixed size or not, this issue can be addressed too.


JK
fxm
Posts: 9123
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Array Descriptor (split from Wiki Improvements)

Postby fxm » Aug 20, 2019 15:26

fxm wrote:I think that only if you declare the dynamic array as:
{Dim|Static} [Shared] As datatype array()
(without any indication of sizing)
the allocated descriptor is sized for the maximum number of dimensions (8).

But if you declare the dynamic array as:
{Dim|Static} [Shared] As datatype array(Any, Any)
or (1st declaration):
Redim [Shared] As datatype array(1, 2)
the allocated array descriptor is sized for only 2 dimensions.
Juergen Kuehlwein
Posts: 210
Joined: Mar 07, 2018 13:59
Location: Germany

Re: Array Descriptor (split from Wiki Improvements)

Postby Juergen Kuehlwein » Aug 20, 2019 16:23

Code: Select all

... As datatype array(Any, Any)

is somehow a hybrid for me. I can tell the number of dimensions beforehand, but i want to set the index bounds later. I suppose the compiler allocates two dimensions. This syntax definitely fixes the number of dimensions, so i wouldn´t call it dynamic.

But for a dynamic array

Code: Select all

... As datatype array ()

it would be nice to be allowed to

Code: Select all

REDIM array(10)
...
REDIM array(10, 10)
...
REDIM array(10, 10, 10)

which currently isn´t possible


JK
fxm
Posts: 9123
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Array Descriptor (split from Wiki Improvements)

Postby fxm » Aug 20, 2019 17:47

For this to be complete, we must also allow in UDTs:
- 'As datatype array()' in UDTs (in this case, the descriptor is allocated for 8 dimensions),
- in addition to 'As datatype array(Any, ...)' (where the allocated descriptor is sized depending on the number of 'Any' declared),
- and also 'Redim As datatype array(...)' (where the allocated descriptor is sized depending on the number of dimensions declared through the bounds).
dodicat
Posts: 5913
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Array Descriptor (split from Wiki Improvements)

Postby dodicat » Aug 21, 2019 10:17

With -gen gas you can undefine a variable then re define it again.
For example you could undefine an array with one dimension then redefine it again with two dimensions.
But not with -gen gcc

Same goes for a simple variable.
However you can still undefine a label then re-define it again with -gen gcc.

Code: Select all



print"hello ";
if false then goto label: else print"hello ";

 label:
print
#undef label

print "Goodbye"
if true then goto label: else print "Goodbye"

 label:
print
#undef label


dim as integer i=6
print i
#undef i

dim as integer i=5
print i

sleep


 

(The gcc errors are cryptic!)
coderJeff
Site Admin
Posts: 3016
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Array Descriptor (split from Wiki Improvements)

Postby coderJeff » Aug 22, 2019 11:42

OK, so we have a number of array related bugs reported on sf.net, plus some new ones reported in this topic, and maybe some others hidden in other topics on the forum.

First step, I think, is to expose the existing array descriptor for developers and advanced users. This will give us a record of our starting point before we attempt to fix bugs and add some features that JK is proposing.

- in github PR #165 I have added a very small API to expose the array descriptor in a relatively stable way. It also kind of establishes the pattern I would like to see if we expand on this idea in future adding other API's that expose fbc internals.
- I added a wiki page DevArrays in the developer section to document array and array descriptor behaviours from the point of view of the developer. It gives us a place to record all the details of arrays that don't fit in the reference or programmer's guide.

So, a very simple example of usage for fbc-int/array.bi

Code: Select all

#include "fbc-int/array.bi"

#macro printArrayDesc( title, array )
scope
   dim ap as FBC.FBARRAY ptr = fb_ArrayGetDesc( array )

   print title
   print , "index_ptr  : " & hex( cint( ap->index_ptr ) )
   print , "base_ptr   : " & hex( cint( ap->base_ptr ) )
   print , "size       : " & ap->size
   print , "element_len: " & ap->element_len
   print , "dimensions : " & ap->dimensions

   for i as integer = 0 to ap->dimensions-1
   print , "dimTb(" & i & ").elements: " & ap->dimTb(i).elements
   print , "dimTb(" & i & ").lbound  : " & ap->dimTb(i).lbound
   print , "dimTb(" & i & ").ubound  : " & ap->dimTb(i).ubound
   next
end scope
#endmacro


dim a() as integer
printArrayDesc( "a() as integer", a() )

dim b(1 to 2, 3 to 5, 7 to 13) as byte
printArrayDesc( "b() as byte", b() )


I've merged the changes in to fbc/master as it won't affect any existing code. I appreciate any comments on the addition.

- in __FB_LANG__="fb" declaring the new symbols in namespace FBC, thus reserving "FBC" for internals. This maybe is a poor choice as we are already using "fbc as fbcctx" in the compiler sources, so we have to rename one of "fbc" symbol names.
- header in a separate directory "./inc/fbc-int" which will be for internal stuff, but exposed to the user.
MrSwiss
Posts: 3213
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: Array Descriptor (split from Wiki Improvements)

Postby MrSwiss » Aug 22, 2019 13:52

I like the approach: nicely, step by step

The naming used currently is however, not very clear (ambiguous/misleading).
FBC-int -- C's 32 bit signed integer data type, comes to mind instantly, instead of referencing 'internal'.
(not considering the name of the namespace proposed)
FBintern -- whould clear the issue ... just a idea, up for discussion.
coderJeff
Site Admin
Posts: 3016
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Array Descriptor (split from Wiki Improvements)

Postby coderJeff » Aug 22, 2019 14:04

MrSwiss wrote:FBintern -- whould clear the issue ... just a idea, up for discussion.


Thanks, MrSwiss. Yes, the changes I added immediately create 2 issues that I will lazily defer to later to resolve.

1) namespace FBC, I really would like to reserve the FBC namespace for compiler internals, so, should the need arise, I will rename the 'fbc' variable in fbc compiler sources to 'fbcompiler' to resolve the conflict. It's not needed just now, but if we ever use './inc/fbc-int/*.bi' directly in compiler sources, this is my solution.

2) __FB_LANG__="??". As it stands, the API will have namespace FBC in lang "fb", no namespace in lang "fblite", and fail in lang "qb" due the differences in integer sizes for lang "qb". I'm going to leave the solution open ended for now. The solution probably involves creating a "./inc/fbc-int/types.bi" kind of include. Though, for comparison, "./inc/fbgfx.bi" deals with namespace FB in the same way as current addition for array descriptor API, so basically I don't know what is best long term solution, at the moment.

I like the approach: nicely, step by step

Thank-you. Your comment helps to justify the time I will take to explain each step we take on this development path. Nice to know community is listening. :)
Juergen Kuehlwein
Posts: 210
Joined: Mar 07, 2018 13:59
Location: Germany

Re: Array Descriptor (split from Wiki Improvements)

Postby Juergen Kuehlwein » Aug 22, 2019 20:58

@Jeff,

so you decided to expose the descriptor itself!
Ok, no problem - this makes some of my functions obsolete, because you now can take the officially declared FBARRAY type.

As already said above when you expand the FBARRAY type, add one or more "reserved" members. This gives room for future ideas, without breaking compatibility with every new step.

For the additions i wrote, i need at least one bit field as a flag for one of the features. It could of course be an entire member variable to be used as a flag then too.

I appreciate your initiative in this area!


JK
speedfixer
Posts: 375
Joined: Nov 28, 2012 1:27
Location: California

Re: Array Descriptor (split from Wiki Improvements)

Postby speedfixer » Aug 22, 2019 22:36

Does this mean that all arrays will have descriptors, exposed and available, in this future version?

david
coderJeff
Site Admin
Posts: 3016
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Array Descriptor (split from Wiki Improvements)

Postby coderJeff » Aug 23, 2019 1:13

Juergen Kuehlwein wrote:As already said above when you expand the FBARRAY type, add one or more "reserved" members. This gives room for future ideas, without breaking compatibility with every new step.


Yes, I read through the code for your proposed additions. From your code and other comments in this topic, the "flags" field is looking like the following (so far):

Code: Select all

enum FBARRAY_FLAGS
   FBARRAY_FLAGS_DIMENSIONS = &h0000000f      '' number of entries allocated in dimTb()
   FBARRAY_FLAGS_STATIC     = &h00000010      '' array points to static/fixed-length memory
   FBARRAY_FLAGS_ATTACHED   = &h00000020      '' array points to memory managed by another descriptor
   FBARRAY_FLAGS_LOCKED     = &h00000040      '' array in use: do not allow redim/erase
   FBARRAY_FLAGS_RESERVED   = &hffffff80      '' reserved, do not use
end enum


FBARRAY_FLAGS_DIMENSIONS:
fbc will allocate the smallest array descriptor possible, if it can determine the dimension count at compile time otherwise it allocates room for FB_MAXDIMENSIONS. Haven't decided on this one as the compiler and runtime are currently set up to try and dimension an array once and then not allow number of dimensions to change. Allowing number of dimensions to change during run time is counter productive to compile time array bounds checks and advice noted in the sources with respect to dimensions. Needs more thought

FBARRAY_STATIC:
if the descriptor points to data with static allocation, then REDIM should be disallowed. ERASE could still be used to clear the array. This is the first flag that I will be focusing on.

FBARRAY_ATTACHED:
this is a feature I saw in your proposed code where an array descriptor can point to arbitrary data, basically providing an array(index, ...) interface, rather than cast( datatype ptr, @array(0) ) + index ... pointer maths. Have to be careful with this one, especially with complex UDT's. It might look like a safe interface, but has all the dangers of pointers under the hood. So may want to think of incorporating some help from the compiler. Most simple syntax would be something like "var byref a() = b()" to create an alias to an existing array, plus other extensions for different dimensions or data types.

FBARRAY_LOCKED:
not sure about this one, based on a comment in this topic, though the intent would be that REDIM should return a run-time error, if array is "locked". Why an array would be locked; yet to be defined.

FBARRAY_RESERVED:
don't use.
coderJeff
Site Admin
Posts: 3016
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Array Descriptor (split from Wiki Improvements)

Postby coderJeff » Aug 23, 2019 1:24

speedfixer wrote:Does this mean that all arrays will have descriptors, exposed and available, in this future version?

david


Yes, that is the intent. Plus, also trying to create a bit of a structure for other additions, like string descriptor, or file i/o, or whatever. To note, I am trying to keep a little separation from the main language, to give continued development half a chance, rather than immediately locked in for next 10 years.. i.e. the separate ./inc/fbc-int folder and documented in the developer section.

Also, "all arrays will have descriptors" is true only if the array is passed as argument to a procedure. This includes the procedure to get the array descriptor itself. Technically, there are some cases where arrays won't have descriptors because fbc optimizes them out, but if the descriptor is never accessed, the user will never know.
coderJeff
Site Admin
Posts: 3016
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Array Descriptor (split from Wiki Improvements)

Postby coderJeff » Aug 23, 2019 2:20

fxm wrote:
coderJeff wrote:The field will have 32 bits available for array descriptor options.

I hope this will be an Integer (32/64 bits) like all other fields, so 32 bits useful?

Where exactly do you intend to add this field in the array descriptor structure?
(perhaps just before the number of dimensions)


yes, new field will be uinteger, just like all the other fields being one of integer, uinteger, or pointer.

For new code it probably doesn't matter exactly where the new "flags" field is placed. In my tests I have placed it just after the dimensions field. Is placing it just before better?
fxm
Posts: 9123
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Array Descriptor (split from Wiki Improvements)

Postby fxm » Aug 23, 2019 6:13

A remark a little off-topic about the syntax to use to designate an array (in its entirety):
- We must use either 'array()' or 'array' (without parentheses) depending on the keyword used.
- This constraint is a nuisance when one wants to define a macro with such a parameter, because it is not known (to my knowledge) how to truncate a macro parameter of its final parentheses (only argument concatenation exists).

Return to “Documentation”

Who is online

Users browsing this forum: No registered users and 2 guests