Question about nested types and name mangling

General FreeBASIC programming questions.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Question about nested types and name mangling

Post by angros47 »

I understand the limits of using OOP C++ libraries. Point is, when I tried to convert FLTK headers to FreeBasic, I got a lot of features to work:

https://www.mediafire.com/file/iwoz5z42 ... i.zip/file

It would disappoint me if I had to stop now, after what I already managed to get to work
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Question about nested types and name mangling

Post by coderJeff »

caseih wrote:No amount of name mangling is going to make FB work completely with all C++ objects. The problem is that FB's object model does not match C++ 1:1 and thus it's really impossible to interface with all C++ classes directly from FB.
100% agree.

Adding named types inside types is a hard addition to make. The machinery just isn't there for it in the compiler. And if you are only using fbc to write your program, it doesn't really add much utility to have it.

Anyway, next thing to finish is:
- deleting destructor is still incomplete and not well tested
- __THISCALL is incomplete and not well tested
I'd have a tough time justifying starting another change without dealing with some of these other open issues first.

----
Usually, when it comes to inter-operating with g++, I'm going for the low hanging fruit; only what's necessary or easy.
For example, mapping fbc's standard types to c++ 1:1 was a hard thing to do but something I felt was necessary. See Alias Modifer.

But with that out of the way, there's machinery within the compiler now that should make this relatively easy:

Code: Select all

type int128_t
	lo as ulongint
	hi as longint
end type

extern "c++"
	declare function add128 _
		( _
			byval x as int128_t alias "__int128",
			byval y as int128_t alias "__int128"
		) as int128_t alias "__int128"
end extern
I don't really want to spent 3 months adding 128 bit support to fbc, but I would spend a weekend add this kind of interop for c++.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Question about nested types and name mangling

Post by coderJeff »

angros47 wrote:It would disappoint me if I had to stop now, after what I already managed to get to work
The bottom line is, I need help; and it's just not there. By myself, it will take forever - no one is interested in the effort needed to actually take an idea to completion. Help with the time consuming tasks of writing tests and investigating bugs - getting a high level of detail on a problem. Proposing a solution instead of 'thing X doesn't work'.

----
As mentioned named types inside types is going to be a hard thing to add to the compiler.

Code: Select all

type outer
	type inner
		x as integer
	end type
	y as integer
end type
To force it, there would need to be some kind of extra syntax <somethin-somethin> ... I don't know what it should be.

Code: Select all

namespace private_stuff '' put in a namespace so it doesn't clutter global namespace
	type inner
		x as integer				
	end type
end namespace

type outer
	inner as private_stuff.inner <somethin-somethin> 
end type
That would tell fbc that not only is 'inner' a member of 'outer' but that name decoration should be modified as well.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Question about nested types and name mangling

Post by angros47 »

coderJeff wrote: The bottom line is, I need help; and it's just not there. By myself, it will take forever - no one is interested in the effort needed to actually take an idea to completion. Help with the time consuming tasks of writing tests and investigating bugs - getting a high level of detail on a problem. Proposing a solution instead of 'thing X doesn't work'.
I would love to help, but I don't know enough of the internal working of the compiler yet. I try to provide all the information I have been able to figure out, hoping it could help.
Anyway, I really admired what you have already done with the constructor and destructor, and the level of compatibility already achieved. I sent the headers I already managed to convert as a proof of what already achieved: some years ago, the ability to use C++ OOP libraries from FreeBasic looked like an unreachable goal. Now, it looks at least possible. So, I think that the best thing is trying to use it, as if it were supposed to work, so we can find what doesn't work yet, and why.

VANYA used that approach when I was struggling to get the Emscripten branch to work on modern Emscripten. And in the end, that approach helped.

From the few things I managed to figure by looking at the compiler code, I have the impression (I might be wrong) that the way to get things to work is close, perhaps the compiler already has what we need (in the past I complained about the lack of functions inside a class, until someone explained me that "private function" replaced them perfectly. Now, I still have the feeling it might be something like that.

Anyway, in case it wasn't clear: a big thank you, for all you have already achieved
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Question about nested types and name mangling

Post by coderJeff »

I appreciate what you are trying to do. Thank-you.
angros47 wrote:I would love to help, but I don't know enough of the internal working of the compiler yet. I try to provide all the information I have been able to figure out, hoping it could help.
Please, let me try to make my point again. I'd rather not have potential contributors go away with the idea that delving in to the compiler source is the only way to make a contribution.

For example,
A few years ago there were a number of problems with new/delete operators. fxm really helped to push improvements forward, not because he was in to the compiler internals, but because of the testing, discussions, document writing, and really trying to understand the problems from the user's point of view and how it works (or doesn't work) at a high level and providing the details through test cases. It was a very productive partnership.
It is very possible to contribute to the project without necessarily contributing directly to the compiler code base.

----
The compiler is an object oriented design (using classes of a primitive sort) with none of the safety net of a formal OOP declaration. There's a lot of global state so everyting is potentially and unsafely connected to everything. It's a big hairy mess and to work on it requires keeping most of the code and data relationships in your head.

I've had my nose in the code for years and have dedicated quite a bit of mental space to the compiler (and rtlib) source. It would be a lot to ask a contributor to build the same level of knowledge before even considering how to make contribution to the project. But if they really want to learn how to contribute directly to the compiler I don't mind helping them along. However, they should expect to spend more (of their) time learning during the process of code review and addressing suggestions for improvement with contributions.

----
emscripten - also incomplete - it's kind of hacked in to windows at best, not well tested, not well documented, is there a lack of interest now that it "works"?
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Question about nested types and name mangling

Post by angros47 »

Well... more or less, it was what I was trying to say, as well!

About your comment on emscripten: yes, the freebasic port is kind of hacked, but actually, that happens because emscripten itself is a sort of huge hack: it is not designed to develop the best web programs: its goal seems to be to allow porting existing programs, developed for Windows or Linux, to Web with minimal or no changes.

Personally, I think that the Emscripten port of FreeBasic can be useful even if few people will use it: in fact, since its goal is not developing new programs, but porting existing ones, in theory even just one person using it in the whole world would be enough
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Question about nested types and name mangling

Post by angros47 »

coderJeff wrote: As mentioned named types inside types is going to be a hard thing to add to the compiler.

Code: Select all

type outer
	type inner
		x as integer
	end type
	y as integer
end type
To force it, there would need to be some kind of extra syntax <somethin-somethin> ... I don't know what it should be.

Code: Select all

namespace private_stuff '' put in a namespace so it doesn't clutter global namespace
	type inner
		x as integer				
	end type
end namespace

type outer
	inner as private_stuff.inner <somethin-somethin> 
end type
That would tell fbc that not only is 'inner' a member of 'outer' but that name decoration should be modified as well.
I thought about it... and I have a suggestion, for the syntax, to make it consistent with what we already have: how about a "private type"?

Code: Select all

	private type inner
		x as integer				
	end type

type outer
	inner as inner <somethin-somethin> 
end type
Since members that in C++ are defined inside the class, in FreeBasic should be defined as private members, why not doing the same for types?
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Question about nested types and name mangling

Post by fxm »

@Jeff,

I see in the code above that the qualifier 'Private'/'Public' can be placed before a non-nested 'Type' (or 'Union') without a syntax error being detected.
Does this work and does it allow specifying an internal/external link for the entire 'Type' (or 'Union')'?
In any case, this syntax is not described in the documentation.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Question about nested types and name mangling

Post by coderJeff »

That's an interesting find. That behaviour goes back to fbc 0.17.0 (2006) when static member procedures were added.
It will be difficult to change something without investigating and understanding what we've got first. Maybe it was an unfinished change 15 years ago and only surfacing now.

Under the hood, in addition to all the kinds of procedures, parser allows PRIVATE/PUBLIC for many declarations and causes the symbol to have the PUBLIC or PRIVATE attribute. Seems only REDIM / EXTERN / COMMON have an explicit check (later in the parser) to generate an invalid use error.

Code: Select all

'' parser allows PRIVATE or PUBLIC for many declarations

public const  C = 1

public static S as integer = 1

public dim        I1 as integer = 1
public dim shared I2 as integer = 1

public var V1 = 1
public var shared V2 = 1

namespace ns
	private dim x as integer
	public dim y as integer
end namespace

public enum E
	e1 =  1
end enum

public type T
	__ as integer
end type

public union U
	__ as integer
end union

public redim A() as integer

public extern EX as integer

public common XX as integer

sub proc()
	private dim a as integer
	public dim b as integer
end sub
I only have had a brief look:
- an EXTERN (unallocated) variable becomes PUBLIC when DIM'ed and implies SHARED
- at a glance, PRIVATE seems to only affect procedures and has no purpose for other symbols
- there is overlap between EXTERN, SHARED, STATIC, COMMON, PUBLIC depending on use: name mangling, initialization, scope, etc.

Not sure, but I think this PRIVATE / PUBLIC usage for various declarations was just never fully developed.
fxm
Moderator
Posts: 12082
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Question about nested types and name mangling

Post by fxm »

I did a little test, and apparently the 'Private' qualifier in front of a 'Type' has no effect on the visibility of UDT members between modules (even for member procedures).
On the other hand, 'Private' at the level of the definition of the member procedure plays its role well.
dodicat
Posts: 7976
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Question about nested types and name mangling

Post by dodicat »

For the record, private: udt method doesn't travel from fb to c++ via a .dll, you only have to declare public in the mirror c++ class
It doesn't travel the other way either, so that's OK then, after all, 10 million ++ c++ users can't be wrong.
The optimisations are OK from fb to c++ now, tested -Wc -O3 with prime numbers.
(I choose c++ as the other language because I see discussions about c++ in this thread.)
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Question about nested types and name mangling

Post by angros47 »

I wonder if my idea of using "private type" do specify that a type has to be used inside other types can work. As far as I know, a class itself doesn't need to be private in C++ either, since it can be defined in all modules that have the same include file.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Question about nested types and name mangling

Post by coderJeff »

angros47 wrote:I wonder if my idea of using "private type" do specify that a type has to be used inside other types can work.
'private|public' has well established meaning for procedures. And is different from 'private:', 'protected:', 'public:', visibility specifiers in a type.

I kind of buy in to the idea that 'private type' might mean to indicate a type declaration that is used in some private way.

However, it only gets us part of the way there. 'private type inner' is marked as something special so that when solving the name mangling for a parameter passed as 'inner' a decision can be made to do something different. But what? This single bit of information alone does not make the relationship between 'private type inner' and 'type outer' or the member in 'type outer'.

Currently 'private|public' has no expectation of meaning for all of the declarations where it's allowed, so first would probably be to disallow all the cases that don't make sense. At module level, it doesn't make much sense except some investigation is needed to see the overlap with shared|extern|common etc. Possibly, if 'private|public' were allowed to have meaning, then maybe in a namespace would have meaning so that 'using' somewhere later would import the namespace to the current one except for anything marked as private.

I don't think there is a quick fix / hack for this. Adding named inner types will be hard.
- Maybe slightly less hard would be using 'import' keyword which currently only has meaning for win32 DLL's. Maybe that could be used inside a TYPE to indicate that some type is intended to be a child of the outer type.
- Or possibly an extension of the alias modifier. Something like 'member as inner alias ".manglename"'. It's still a hack but as far as I know, inter-operating with c++ does not require mangling of data only members.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: Question about nested types and name mangling

Post by angros47 »

Just a silly suggestion: how about using the "::" symbol, as it's done for procedures?

Like, instead of having:

Code: Select all

type foo
	type inner
		a as long
		b as long
	end type
	c as inner
end type
Could a syntax like this make sense?

Code: Select all

type foo::inner
	a as long
	b as long
end type

type foo
	c as inner
end type
Would it be easier or harder to parse?
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Question about nested types and name mangling

Post by coderJeff »

coderJeff wrote: Aug 07, 2021 22:45 I think, the effort would need to be made much earlier in the parser to allow named nested types and storing the required information in the symbol database, so that later when member lookups are made and name mangling is calculated, there's enough information to get what you want.
I recently revisited this because I was writing some code where I wanted this feature; not necessarily for g++ but at least just within fbc to put a TYPE scoped inside another TYPE. I also noticed some users were using TYPE's to instance a collection of functionality over NAMESPACE's, so extending TYPE's within TYPE's seems like a natural progression.

I have created something that "works" and wrote unit tests to check that it handles the g++ name mangling correctly -- at least for the simple cases I created.

Syntax:

Code: Select all

type T
	dim a as integer
	type P
		dim b as integer
		dim c as integer
	end type
	d as P
end type
Also allowed are method procedure declaration and the like within inner types, but there is so much I haven't tested. No doubt this will cause havoc with lookup rules which we explored a few months ago.

I am wondering, how "finished" should the feature be before I merge in to master because I know there's bugs; visibility rules (private/protected) and object (extends) will have bugs. However, In a sense it is usable.

Thoughts?
Post Reply