Study of symbol lookups in namespaces and types

Forum for discussion about the documentation project.
Post Reply
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Study of symbol lookups in namespaces and types

Post by marcov »

fxm wrote:
marcov wrote:
But if a symbol is marked overload, also search deeper ?
In the single scope (if no ambiguity) retained after a procedure name look-up, then, an overload resolution is applied in addition if necessary.
The no ambiguity will be harder as you start to introduce more conversions.
I think the current process (similar to the one of C ++ for example) simplifies the problem by dealing with it in two successive phases:
- selection of a single scope among all those which are accessible by only taking into account the symbol name regardless of the arguments supplied by the caller (if at least two scopes containing the symbol name have equivalent accessibility priority => ambiguity).
- within this single scope retained, one final overload resolution taking into account the full signature.

The problem is thus simplified to one single full overload resolution within a same scope (instead of a full overload resolution within each scope candidate, and then the difficult choice of the best result among the scope candidates).
Such a process uses the principle of 'name hiding'.
But then you are limited in cross scope overloading. This means that it is very hard to e.g. add an overloaded variant of a function to a set that is e.g. already defined in system and let it be transparent to the user (without detailed scope setup).

But yes, if you don't do it, you search much more scopes, and the operation is more expensive/slower. But only if you don't find an exact match. Then it reduces to making finding an exact match cheaper (e.g. compare with hash of a normalized signature), and only when not found do full resolution
Lost Zergling
Posts: 534
Joined: Dec 02, 2011 22:51
Location: France

Re: Study of symbol lookups in namespaces and types

Post by Lost Zergling »

Hypothetically. Would'nt generic 'cross scope overloading' require a specific extended instruction set ? I mean new concepts using syntax, otherwise may lead to some new spaguetti mix ?
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Study of symbol lookups in namespaces and types

Post by marcov »

Lost Zergling wrote:Hypothetically. Would'nt generic 'cross scope overloading' require a specific extended instruction set ? I mean new concepts using syntax, otherwise may lead to some new spaguetti mix ?
No. Basically it is all in the translation of HLL SUB names to the assembler labels to call from the various overloaded subs. There is no code generated, no spaghetti, nothing.

However it can be confusing, assume DoObject() is a major function in, say, the RTS, and a "using mydatatype" pulls in a DoObject() for the datatype defined by the "mydatatype" module. Now if there are conversions, users might think the RTS, but due to conversions some corner cases ends up at the mydatatype variant.

Rare, but realistic. Adding to the RTS, even just scope wise, is a thing to be careful about.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Study of symbol lookups in namespaces and types

Post by coderJeff »

Munair wrote:What makes it complicated IMO, both for the compiler and programmer, is the keyword "using", which is somewhat similar to "with". Perhaps "using .. end using" would make things easier?
marcov wrote:Using is done after the definitions, so its scopes are on the scope stack before older definitions.

But more logical would be to require using to be valid only at the top.
@Munair, @marcov, thank-you for your suggestions.

namespace definition as way of organizing scopes and code is fairly solid in fbc. Or at least it's getting there. Some of the bugs I fixed deal with explicitly qualifying namespace scopes. So it's improving. The issues with enums are still a mess though.

Yes, the use of 'using' to import a namespace in to another namespace is problematic. 'using' as implemented in fbc is a rather low level operation. So it is kind of like c++ in that it is convenient when it works but also gives the programmer an extra way to shoot themselves in the foot when it doesn't. fbc is more similar to c++ when there are no name collisions.

Because there is an existing user code base, I don't think we will get very far with the 'using' statement as a way enforce a hierarchy or dependency model and should leave it up to the user to sort out the ambiguities if they are going to use 'using'.

By contrast 'type X extends Y' does intend to enforce an inheritance model with well defined rules and if user code gets broken because of those improvements, then that's just what needs to happen.

Maybe in future can expand on namespaces in a different way through a different syntax. 'namespace X extends Y,Z', or some kind of 'import namespace X' with stricter rules. Maybe future jeff can work on that, I don't know.

There's multiple issues to solve and in my mind I think I have them separated. More on that soon.

---
@Munair, the detractors and and criticisms on the forum get to me sometimes. My apologies for my curtness towards you elsewhere.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Study of symbol lookups in namespaces and types

Post by coderJeff »

fxm wrote:I consider this following as OK ('dim A as integer = 1' and 'print A' in the same scope):

Code: Select all

dim A as integer = 1

namespace N
   dim A as integer = 2
end namespace

using N

print A  '' 1
Why by adding 'Shared' in the first declaration should modify the behavior ?
You are correct. Adding shared should not change the behaviour.

I believe the rule should be, for namespaces+using:
- The symbol defined in the ancestry of the current context, i.e. directly reachable from the current scope within the namespace, is preferred
- otherwise, check the imports (using).

I'm planning to add '#pragma lookup108' which will allow selection of the old lookup rules for unqualified symbols. This should allow users to compile older code if they are really having trouble. It will also help with development of fbc since it will be easier to check differences as we try to move forward.

I have a few new changes based on our discussions so far that I'm going to try and push out soon.

Should be something like...

Namespaces in priority order:
- symbols in the direct ancestry of the current scope (namespace), most local scope first
- symbols in the imports

Types (methods) - a work in progress:
- symbols in the most local scope
- symbols in the ancestry of the TYPE
- symbols in the ancestry of the namespace
- symbols in the other imports

The main issue I have is that under hood, fbc doesn't quite propagate enough context information for the look-ups in Type.methods to work exactly correct. So it will probably still have some bugs, unfortunately. If it's a real mess '#pragma lookup108' can bail users out by reverting to the old lookup rules for types.
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Study of symbol lookups in namespaces and types

Post by fxm »

coderJeff wrote:I believe the rule should be, for namespaces+using:
- The symbol defined in the ancestry of the current context, i.e. directly reachable from the current scope within the namespace, is preferred
- otherwise, check the imports (using).
.....
Namespaces in priority order:
- symbols in the direct ancestry of the current scope (namespace), most local scope first
- symbols in the imports
I think it already works like that basically.

But the question remains for the specific case of the global namespace:
- if we consider that this is the oldest of the ancestors, then it would nevertheless have priority (just above) over the imported namespaces (by using) as with fbc 1.08, which is not the case for the moment with fbc 1.09.
- if we make it a special case, it could then remain in last priority as it currently is (or somewhere else).
Munair
Posts: 1286
Joined: Oct 19, 2017 15:00
Location: Netherlands
Contact:

Re: Study of symbol lookups in namespaces and types

Post by Munair »

coderJeff wrote: @Munair, the detractors and and criticisms on the forum get to me sometimes. My apologies for my curtness towards you elsewhere.
I figured as much and I understand. I might have been too quick in responding to the detractors and decided to largely ignore them.
Munair
Posts: 1286
Joined: Oct 19, 2017 15:00
Location: Netherlands
Contact:

Re: Study of symbol lookups in namespaces and types

Post by Munair »

marcov wrote:But more logical would be to require using to be valid only at the top.
That would be more like Pascal structure and I concur it would make it easier for such compiler, but (Free)BASIC is different in that it doesn't require that kind of structure, so I imagine it makes it more of a challenge. To be honest, I wouldn't know how to approach the problem other than what I suggested earlier with a strictly defined using-block similar to a scope- or with-block. Personally, I never use the using statement because IMO it makes the code less readable.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Study of symbol lookups in namespaces and types

Post by coderJeff »

fxm wrote:I think it already works like that basically.

But the question remains for the specific case of the global namespace:
- if we consider that this is the oldest of the ancestors, then it would nevertheless have priority (just above) over the imported namespaces (by using) as with fbc 1.08, which is not the case for the moment with fbc 1.09.
- if we make it a special case, it could then remain in last priority as it currently is (or somewhere else).
I think the global namespace should not be a special case. If it were then we get different behaviour depending on if we are in the global namespace or in a nested namespace.

I need to make a change in fbc-1.09 that makes namespaces a little more like fbc-1.08 again but keeps the improvements for types as in fbc-1.09.

My apologies. The journey through this problem has been challenging. Challenging because of the need to abandon the similarities between namespaces and types - both goals of each and the common treatment by the compiler.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Study of symbol lookups in namespaces and types

Post by coderJeff »

A few tests cases, just for namespaces. In current fbc-1.09 from the daily build, there is a couple of failures in the third example, which I intend to fix. I will post tests for types as well.
checks.bi

Code: Select all

#macro checkType( arg1, arg2 )
	? #arg1,
	? #arg2,
	#if( typeof(arg1) = typeof(arg2) )
		? "OK"
	#else
		? "FAIL"
	#endif
#endmacro

#macro checkValue( arg1, arg2 )
	? #arg1,
	? #arg2,
	if( arg1 = arg2 ) then
		? "OK"
	else
		? "FAIL"
	endif
#endmacro
test case 1

Code: Select all

#include "checks.bi"

namespace N1
	type T
		__ as long = 1
	end type

	sub proc()
		checkType( T, N1.T )
	end sub
end namespace

namespace N2
	using N1

	type T
		__ as long = 2
	end type

	dim as T foo

	sub proc()
		checkType( T, N2.T )
	end sub
end namespace

N1.proc()
N2.proc()

checkType( N1.T, N1.T )
checkType( N2.T, N2.T )

using N2

checkType( T, N2.T )

checkValue( foo.__, 2 )
test case 2

Code: Select all

#include "checks.bi"

namespace N1
  type T
    __ as long = 1
  end type
  
  sub proc()
    checkType( T, N1.T )
  end sub
end namespace

namespace N2
  using N1
  
  dim as T foo
  
  sub proc()
    checkType( T, N2.T )
  end sub
end namespace

N1.proc()
N2.proc()

checkType( N1.T, N1.T )
checkType( N2.T, N2.T )

using N2

checkType( T, N2.T )

checkValue( foo.__, 1 )
test case 3

Code: Select all

#include "checks.bi"

type T
	__ as long = 0
end type

namespace N1
  type T
    __ as long = 1
  end type
  
  sub proc()
    checkType( T, N1.T )
  end sub
end namespace

namespace N2
  using N1
  
  dim as T foo
  
  sub proc()
    checkType( T, ..T  )
  end sub
end namespace

N1.proc()
N2.proc()

checkType( T, ..T )

checkType( N1.T, N1.T )
checkType( N2.T, N2.T )

using N2

checkType( T, ..T )

checkValue( foo.__, 0 )
Munair
Posts: 1286
Joined: Oct 19, 2017 15:00
Location: Netherlands
Contact:

Re: Study of symbol lookups in namespaces and types

Post by Munair »

I always use TYPES as a way of structuring module code. That's why I was wondering about the necessity of an additional namespace feature. Are there declarations that can be part of namespaces but not TYPES?
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Study of symbol lookups in namespaces and types

Post by fxm »

Jeff,

Agree for these 2 faults (viewtopic.php?p=288434#p288434), to be corrected.

On the other hand.
To sum up, the unqualified name look-ups for future fbc 1.09 version should verify the priority hierarchy as follows:
- (1) current namespace/type
- (2) base types (by 'Extends'), ranked from closest to furthest in inheritance hierarchy
- (3) parent namespaces, ranked from closest to furthest in ancestry hierarchy (including global namespace: always furthest in hierarchy)
- (4) imported namespaces (by 'Using'), without hierarchy between them
SARG
Posts: 1756
Joined: May 27, 2005 7:15
Location: FRANCE

Re: Study of symbol lookups in namespaces and types

Post by SARG »

Just a warning for users of 1.09.

This morning I got an error using 1.09. It took me a bit of time before understanding it.

By error there was a dot before a variable name : '.typ'.
No error with 1.08 and it's working fine but with 1.09 this error happens : undefined symbol.
I guess that changes made for namespaces/types cause this new behaviour.
Munair
Posts: 1286
Joined: Oct 19, 2017 15:00
Location: Netherlands
Contact:

Re: Study of symbol lookups in namespaces and types

Post by Munair »

SARG wrote:Just a warning for users of 1.09.

This morning I got an error using 1.09. It took me a bit of time before understanding it.

By error there was a dot before a variable name : '.typ'.
No error with 1.08 and it's working fine but with 1.09 this error happens : undefined symbol.
I guess that changes made for namespaces/types cause this new behaviour.
I will stick to 1.08 for the time being.
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Study of symbol lookups in namespaces and types

Post by fxm »

SARG wrote:Just a warning for users of 1.09.

This morning I got an error using 1.09. It took me a bit of time before understanding it.

By error there was a dot before a variable name : '.typ'.
No error with 1.08 and it's working fine but with 1.09 this error happens : undefined symbol.
I guess that changes made for namespaces/types cause this new behaviour.
changelog.txt (today):
Version 1.09.0
.....
[added]
.....
- sf.net #645: respect global namespace '.' and '..' prefixes in REDIM
.....
- sf.net #947: enum declaration in a namespace incorrectly checks for parent
.....
[fixed]
.....
- sf.net #645: don't access locals if explicit namespace is given on identifier
- sf.net #645: don't access locals if explicit global namespace '.' and '..' prefixes are given on identifier
.....
- sf.net #945: regression: Scoping rules have changed inside methods - allow globals to shadow non-explicit enums implicitly imported in to the current namespace
.....
and this is just the start, but all to improve qualified/unqualified name look-ups.

Also, to access duplicate symbols defined as global outside of local blocks, always adding two dots as a prefix is now preferable: '..SomeSymbol' (as mandatory required inside a 'With...End With' block).
Post Reply