Question about nested types and name mangling
Re: Question about nested types and name mangling
I understand the limits of using OOP C++ libraries. Point is, when I tried to convert FLTK headers to FreeBasic, I got a lot of features to work:
https://www.mediafire.com/file/iwoz5z42 ... i.zip/file
It would disappoint me if I had to stop now, after what I already managed to get to work
https://www.mediafire.com/file/iwoz5z42 ... i.zip/file
It would disappoint me if I had to stop now, after what I already managed to get to work
Re: Question about nested types and name mangling
100% agree.caseih wrote:No amount of name mangling is going to make FB work completely with all C++ objects. The problem is that FB's object model does not match C++ 1:1 and thus it's really impossible to interface with all C++ classes directly from FB.
Adding named types inside types is a hard addition to make. The machinery just isn't there for it in the compiler. And if you are only using fbc to write your program, it doesn't really add much utility to have it.
Anyway, next thing to finish is:
- deleting destructor is still incomplete and not well tested
- __THISCALL is incomplete and not well tested
I'd have a tough time justifying starting another change without dealing with some of these other open issues first.
----
Usually, when it comes to inter-operating with g++, I'm going for the low hanging fruit; only what's necessary or easy.
For example, mapping fbc's standard types to c++ 1:1 was a hard thing to do but something I felt was necessary. See Alias Modifer.
But with that out of the way, there's machinery within the compiler now that should make this relatively easy:
Code: Select all
type int128_t
lo as ulongint
hi as longint
end type
extern "c++"
declare function add128 _
( _
byval x as int128_t alias "__int128",
byval y as int128_t alias "__int128"
) as int128_t alias "__int128"
end extern
Re: Question about nested types and name mangling
The bottom line is, I need help; and it's just not there. By myself, it will take forever - no one is interested in the effort needed to actually take an idea to completion. Help with the time consuming tasks of writing tests and investigating bugs - getting a high level of detail on a problem. Proposing a solution instead of 'thing X doesn't work'.angros47 wrote:It would disappoint me if I had to stop now, after what I already managed to get to work
----
As mentioned named types inside types is going to be a hard thing to add to the compiler.
Code: Select all
type outer
type inner
x as integer
end type
y as integer
end type
Code: Select all
namespace private_stuff '' put in a namespace so it doesn't clutter global namespace
type inner
x as integer
end type
end namespace
type outer
inner as private_stuff.inner <somethin-somethin>
end type
Re: Question about nested types and name mangling
I would love to help, but I don't know enough of the internal working of the compiler yet. I try to provide all the information I have been able to figure out, hoping it could help.coderJeff wrote: The bottom line is, I need help; and it's just not there. By myself, it will take forever - no one is interested in the effort needed to actually take an idea to completion. Help with the time consuming tasks of writing tests and investigating bugs - getting a high level of detail on a problem. Proposing a solution instead of 'thing X doesn't work'.
Anyway, I really admired what you have already done with the constructor and destructor, and the level of compatibility already achieved. I sent the headers I already managed to convert as a proof of what already achieved: some years ago, the ability to use C++ OOP libraries from FreeBasic looked like an unreachable goal. Now, it looks at least possible. So, I think that the best thing is trying to use it, as if it were supposed to work, so we can find what doesn't work yet, and why.
VANYA used that approach when I was struggling to get the Emscripten branch to work on modern Emscripten. And in the end, that approach helped.
From the few things I managed to figure by looking at the compiler code, I have the impression (I might be wrong) that the way to get things to work is close, perhaps the compiler already has what we need (in the past I complained about the lack of functions inside a class, until someone explained me that "private function" replaced them perfectly. Now, I still have the feeling it might be something like that.
Anyway, in case it wasn't clear: a big thank you, for all you have already achieved
Re: Question about nested types and name mangling
I appreciate what you are trying to do. Thank-you.
For example,
A few years ago there were a number of problems with new/delete operators. fxm really helped to push improvements forward, not because he was in to the compiler internals, but because of the testing, discussions, document writing, and really trying to understand the problems from the user's point of view and how it works (or doesn't work) at a high level and providing the details through test cases. It was a very productive partnership.
It is very possible to contribute to the project without necessarily contributing directly to the compiler code base.
----
The compiler is an object oriented design (using classes of a primitive sort) with none of the safety net of a formal OOP declaration. There's a lot of global state so everyting is potentially and unsafely connected to everything. It's a big hairy mess and to work on it requires keeping most of the code and data relationships in your head.
I've had my nose in the code for years and have dedicated quite a bit of mental space to the compiler (and rtlib) source. It would be a lot to ask a contributor to build the same level of knowledge before even considering how to make contribution to the project. But if they really want to learn how to contribute directly to the compiler I don't mind helping them along. However, they should expect to spend more (of their) time learning during the process of code review and addressing suggestions for improvement with contributions.
----
emscripten - also incomplete - it's kind of hacked in to windows at best, not well tested, not well documented, is there a lack of interest now that it "works"?
Please, let me try to make my point again. I'd rather not have potential contributors go away with the idea that delving in to the compiler source is the only way to make a contribution.angros47 wrote:I would love to help, but I don't know enough of the internal working of the compiler yet. I try to provide all the information I have been able to figure out, hoping it could help.
For example,
A few years ago there were a number of problems with new/delete operators. fxm really helped to push improvements forward, not because he was in to the compiler internals, but because of the testing, discussions, document writing, and really trying to understand the problems from the user's point of view and how it works (or doesn't work) at a high level and providing the details through test cases. It was a very productive partnership.
It is very possible to contribute to the project without necessarily contributing directly to the compiler code base.
----
The compiler is an object oriented design (using classes of a primitive sort) with none of the safety net of a formal OOP declaration. There's a lot of global state so everyting is potentially and unsafely connected to everything. It's a big hairy mess and to work on it requires keeping most of the code and data relationships in your head.
I've had my nose in the code for years and have dedicated quite a bit of mental space to the compiler (and rtlib) source. It would be a lot to ask a contributor to build the same level of knowledge before even considering how to make contribution to the project. But if they really want to learn how to contribute directly to the compiler I don't mind helping them along. However, they should expect to spend more (of their) time learning during the process of code review and addressing suggestions for improvement with contributions.
----
emscripten - also incomplete - it's kind of hacked in to windows at best, not well tested, not well documented, is there a lack of interest now that it "works"?
Re: Question about nested types and name mangling
Well... more or less, it was what I was trying to say, as well!
About your comment on emscripten: yes, the freebasic port is kind of hacked, but actually, that happens because emscripten itself is a sort of huge hack: it is not designed to develop the best web programs: its goal seems to be to allow porting existing programs, developed for Windows or Linux, to Web with minimal or no changes.
Personally, I think that the Emscripten port of FreeBasic can be useful even if few people will use it: in fact, since its goal is not developing new programs, but porting existing ones, in theory even just one person using it in the whole world would be enough
About your comment on emscripten: yes, the freebasic port is kind of hacked, but actually, that happens because emscripten itself is a sort of huge hack: it is not designed to develop the best web programs: its goal seems to be to allow porting existing programs, developed for Windows or Linux, to Web with minimal or no changes.
Personally, I think that the Emscripten port of FreeBasic can be useful even if few people will use it: in fact, since its goal is not developing new programs, but porting existing ones, in theory even just one person using it in the whole world would be enough
Re: Question about nested types and name mangling
I thought about it... and I have a suggestion, for the syntax, to make it consistent with what we already have: how about a "private type"?coderJeff wrote: As mentioned named types inside types is going to be a hard thing to add to the compiler.To force it, there would need to be some kind of extra syntax <somethin-somethin> ... I don't know what it should be.Code: Select all
type outer type inner x as integer end type y as integer end type
That would tell fbc that not only is 'inner' a member of 'outer' but that name decoration should be modified as well.Code: Select all
namespace private_stuff '' put in a namespace so it doesn't clutter global namespace type inner x as integer end type end namespace type outer inner as private_stuff.inner <somethin-somethin> end type
Code: Select all
private type inner
x as integer
end type
type outer
inner as inner <somethin-somethin>
end type
Re: Question about nested types and name mangling
@Jeff,
I see in the code above that the qualifier 'Private'/'Public' can be placed before a non-nested 'Type' (or 'Union') without a syntax error being detected.
Does this work and does it allow specifying an internal/external link for the entire 'Type' (or 'Union')'?
In any case, this syntax is not described in the documentation.
I see in the code above that the qualifier 'Private'/'Public' can be placed before a non-nested 'Type' (or 'Union') without a syntax error being detected.
Does this work and does it allow specifying an internal/external link for the entire 'Type' (or 'Union')'?
In any case, this syntax is not described in the documentation.
Re: Question about nested types and name mangling
That's an interesting find. That behaviour goes back to fbc 0.17.0 (2006) when static member procedures were added.
It will be difficult to change something without investigating and understanding what we've got first. Maybe it was an unfinished change 15 years ago and only surfacing now.
Under the hood, in addition to all the kinds of procedures, parser allows PRIVATE/PUBLIC for many declarations and causes the symbol to have the PUBLIC or PRIVATE attribute. Seems only REDIM / EXTERN / COMMON have an explicit check (later in the parser) to generate an invalid use error.
I only have had a brief look:
- an EXTERN (unallocated) variable becomes PUBLIC when DIM'ed and implies SHARED
- at a glance, PRIVATE seems to only affect procedures and has no purpose for other symbols
- there is overlap between EXTERN, SHARED, STATIC, COMMON, PUBLIC depending on use: name mangling, initialization, scope, etc.
Not sure, but I think this PRIVATE / PUBLIC usage for various declarations was just never fully developed.
It will be difficult to change something without investigating and understanding what we've got first. Maybe it was an unfinished change 15 years ago and only surfacing now.
Under the hood, in addition to all the kinds of procedures, parser allows PRIVATE/PUBLIC for many declarations and causes the symbol to have the PUBLIC or PRIVATE attribute. Seems only REDIM / EXTERN / COMMON have an explicit check (later in the parser) to generate an invalid use error.
Code: Select all
'' parser allows PRIVATE or PUBLIC for many declarations
public const C = 1
public static S as integer = 1
public dim I1 as integer = 1
public dim shared I2 as integer = 1
public var V1 = 1
public var shared V2 = 1
namespace ns
private dim x as integer
public dim y as integer
end namespace
public enum E
e1 = 1
end enum
public type T
__ as integer
end type
public union U
__ as integer
end union
public redim A() as integer
public extern EX as integer
public common XX as integer
sub proc()
private dim a as integer
public dim b as integer
end sub
- an EXTERN (unallocated) variable becomes PUBLIC when DIM'ed and implies SHARED
- at a glance, PRIVATE seems to only affect procedures and has no purpose for other symbols
- there is overlap between EXTERN, SHARED, STATIC, COMMON, PUBLIC depending on use: name mangling, initialization, scope, etc.
Not sure, but I think this PRIVATE / PUBLIC usage for various declarations was just never fully developed.
Re: Question about nested types and name mangling
I did a little test, and apparently the 'Private' qualifier in front of a 'Type' has no effect on the visibility of UDT members between modules (even for member procedures).
On the other hand, 'Private' at the level of the definition of the member procedure plays its role well.
On the other hand, 'Private' at the level of the definition of the member procedure plays its role well.
Re: Question about nested types and name mangling
For the record, private: udt method doesn't travel from fb to c++ via a .dll, you only have to declare public in the mirror c++ class
It doesn't travel the other way either, so that's OK then, after all, 10 million ++ c++ users can't be wrong.
The optimisations are OK from fb to c++ now, tested -Wc -O3 with prime numbers.
(I choose c++ as the other language because I see discussions about c++ in this thread.)
It doesn't travel the other way either, so that's OK then, after all, 10 million ++ c++ users can't be wrong.
The optimisations are OK from fb to c++ now, tested -Wc -O3 with prime numbers.
(I choose c++ as the other language because I see discussions about c++ in this thread.)
Re: Question about nested types and name mangling
I wonder if my idea of using "private type" do specify that a type has to be used inside other types can work. As far as I know, a class itself doesn't need to be private in C++ either, since it can be defined in all modules that have the same include file.
Re: Question about nested types and name mangling
'private|public' has well established meaning for procedures. And is different from 'private:', 'protected:', 'public:', visibility specifiers in a type.angros47 wrote:I wonder if my idea of using "private type" do specify that a type has to be used inside other types can work.
I kind of buy in to the idea that 'private type' might mean to indicate a type declaration that is used in some private way.
However, it only gets us part of the way there. 'private type inner' is marked as something special so that when solving the name mangling for a parameter passed as 'inner' a decision can be made to do something different. But what? This single bit of information alone does not make the relationship between 'private type inner' and 'type outer' or the member in 'type outer'.
Currently 'private|public' has no expectation of meaning for all of the declarations where it's allowed, so first would probably be to disallow all the cases that don't make sense. At module level, it doesn't make much sense except some investigation is needed to see the overlap with shared|extern|common etc. Possibly, if 'private|public' were allowed to have meaning, then maybe in a namespace would have meaning so that 'using' somewhere later would import the namespace to the current one except for anything marked as private.
I don't think there is a quick fix / hack for this. Adding named inner types will be hard.
- Maybe slightly less hard would be using 'import' keyword which currently only has meaning for win32 DLL's. Maybe that could be used inside a TYPE to indicate that some type is intended to be a child of the outer type.
- Or possibly an extension of the alias modifier. Something like 'member as inner alias ".manglename"'. It's still a hack but as far as I know, inter-operating with c++ does not require mangling of data only members.
Re: Question about nested types and name mangling
Just a silly suggestion: how about using the "::" symbol, as it's done for procedures?
Like, instead of having:
Could a syntax like this make sense?
Would it be easier or harder to parse?
Like, instead of having:
Code: Select all
type foo
type inner
a as long
b as long
end type
c as inner
end type
Code: Select all
type foo::inner
a as long
b as long
end type
type foo
c as inner
end type
Re: Question about nested types and name mangling
I recently revisited this because I was writing some code where I wanted this feature; not necessarily for g++ but at least just within fbc to put a TYPE scoped inside another TYPE. I also noticed some users were using TYPE's to instance a collection of functionality over NAMESPACE's, so extending TYPE's within TYPE's seems like a natural progression.coderJeff wrote: ↑Aug 07, 2021 22:45 I think, the effort would need to be made much earlier in the parser to allow named nested types and storing the required information in the symbol database, so that later when member lookups are made and name mangling is calculated, there's enough information to get what you want.
I have created something that "works" and wrote unit tests to check that it handles the g++ name mangling correctly -- at least for the simple cases I created.
Syntax:
Code: Select all
type T
dim a as integer
type P
dim b as integer
dim c as integer
end type
d as P
end type
I am wondering, how "finished" should the feature be before I merge in to master because I know there's bugs; visibility rules (private/protected) and object (extends) will have bugs. However, In a sense it is usable.
Thoughts?