FreeBASIC 1.08 Development

[SARG]

Another way to go would be to port the entire gfxlib C source to FB. I have no idea how difficult or what kind of performance you would get, since gcc is a much better optimizer than fbc. But you might have more people willing to improve or extend the library if it weren't in C.

Many years ago I though about that (porting) but there are so many files that's a bit discouraging.
However do you think (first glance) it would possible to port file by file or at least a group of related files and getting something that works ?
Mixing of .o files from C and from Fbc.
If yes, the task seems a "bit" more achievable.

[coderJeff]

Mixing of .o files from C and from Fbc.

fbc has a high degree of compatibility with C. Regardless of what anyone says [yes, I'm thinking of your opinions, marcov, :) ], it is a goal of the project. In theory, you should be able to freely mix C and fbc code. Some limitations apply when you start getting in to OOP (classes) style code. Otherwise, If you are willing to take on the problems common of C also in your fbc code (like naked pointers and manual memory management, for example), then fbc and C work just fine together.

People going for performance in a sane way would use a 3D oriented library to begin with.

Yes, in general, the preference of the developers is that new functionality would be though libraries only and not mess with the compiler quirk stuff (anymore).

For example, to extend gfxlib, assuming user is satisfied with gfxlib:
- port the gfxlib library headers to fbc if you want to write your extension in fbc, or
- use the gfxlib library headers directly if you want to write your extension in C
- compile the new extension (modules / translation units) to a separate library, and
- create a fbc include (.bi file) with #inclib to pull it in to user source when they #include the header.

In other words, very much in favor extending fbc through an external, quite possibly separately maintained library, rather than including everything under fbc compiler itself.

[marcov]

Mixing of .o files from C and from Fbc.

fbc has a high degree of compatibility with C. Regardless of what anyone says [yes, I'm thinking of your opinions, marcov, :) ], it is a goal of the project. In theory, you should be able to freely mix C and fbc code.

And how can you do this more (with the gas backend) than FPC ? Maybe a few varargs related things (*), but that is not what we were talking about here.

My critique about fbc is not about mixing FB and C, but imitating(like preprocessor use, and import model). That is something else, and not a C compatibility feature. At best some easier conversion, but even that is doubtful for anything non trivial.

Keep in mind that FPC has had a textmode IDE with the whole of GDB and the compiler linked into it maintained for longer than FB exists, working and interfacing with eachother. This only ended last year.

Some limitations apply when you start getting in to OOP (classes) style code. Otherwise, If you are willing to take on the problems common of C also in your fbc code (like naked pointers and manual memory management, for example), then fbc and C work just fine together.

Which FPC all does too, so that is not the point.

People going for performance in a sane way would use a 3D oriented library to begin with.

Yes, in general, the preference of the developers is that new functionality would be though libraries only and not mess with the compiler quirk stuff (anymore).

[/quote]

and besides the 3D argument, I also think the speed increase due to C should not be overrated. Most highspeed C comes through careful crafting, strategic assembler and exploiting other platform specific optimizations and most of all, a lot of of continuous maintenance and work.

Usually the abstractions are the problem (like single pixel setting functions), rather than the actual code.

[coderJeff]

Hi marcov, I meant the poke with the utmost respect. I think we have had some friendly worthwhile discussions from time to time about FPC's development and fbc's short-comings. I feel you have me at disadvantage as I think I am certain you likely know a lot more about fbc than I will know about FPC.

I am honest with myself: FreeBASIC is not taking over the world. Based on downloads and some other metrics I have a feeling that the project is important to some people. And that makes it interesting and fun for me to work on. And if there are some others that want to do the same, that's all good with me.

You bring up valid points about import model, improvements, etc. You are not wrong.

I think probably more precise is to say gcc compatibility and following same ABI on whichever target it runs. Maybe some users are trying other compilers? We are not making a generic optimizing assembler + linker. I'm not sure we will ever have enough resources for such work. But some users are willing to let fbc target gcc to get their fbc program to run on their system. So gcc is the fallback and the way forward on to other targets. Maybe if the LLVM backend gets going again it could be that instead /or also.

True, fbc's preprocessor is C-like. However, I would say "preprocessor" is a misnomer. It's actually part of the lexer/parser, so variable types, sizes and constant values are evaluated as the source compiles. But also true, like C, can get really hairy if used for large template/generic code as there is no type safety in macros.

---
I had a quick look around fp.org and looks great. I think it's been a while since I gave it a serious look. Well organized, looks like lots of contributors. You have many same packages that we translate headers for plus others. It was interesting to read through FPC's planned features and bug reports and see the some similarities with fbc. Overall looks like a great project to be involved with.

[marcov]

Hi marcov, I meant the poke with the utmost respect. I think we have had some friendly worthwhile discussions from time to time about FPC's development and fbc's short-comings. I feel you have me at disadvantage as I think I am certain you likely know a lot more about fbc than I will know about FPC.

I get a bit of allergic when people put C on a pedestal. Usually the statements are very broad and suggests so much more (like seamlessly working while in reality you usually have to have create external headers) than it actually implements/does. Left unchecked people get the idea that a language is optimal for working with C, while they have to translate headers like everybody else.

At the very least it should be made clear what aspect of C compatibility is meant, if it is about backend, external linking, source and principles similarities etc. And if it is generic C or some specific implementation. In the case of GCC it is worse since it still has the tendency to emulate *nix on Windows, requiring import libraries generated with some external tool to simply link.

I am honest with myself: FreeBASIC is not taking over the world. Based on downloads and some other metrics I have a feeling that the project is important to some people. And that makes it interesting and fun for me to work on. And if there are some others that want to do the same, that's all good with me.

Neither is FPC. At best we are one of the larger ones in the pond with small fish.

I think probably more precise is to say gcc compatibility and following same ABI on whichever target it runs.

Doesn't gcc C++ have multiple ABIs on Windows (SJ and SEH etc?) :-)

Anyway we roughly do the same, but we diverge occasionally for targets where gcc is not the dominant compiler. That mostly are retro targets though, and to a lesser degree, Windows.

Maybe some users are trying other compilers? We are not making a generic optimizing assembler + linker. I'm not sure we will ever have enough resources for such work.

The internal assembler (based on NASM tables) and linker were for a couple of reasons mostly to do with compilation speed and smartlinking (removing procedures that are in a .o/.a but not used) on Windows, and minor reasons in general going beyond what binutils supports.

The fact that binutils for windows 64-bit took so long also didn't help.

But some users are willing to let fbc target gcc to get their fbc program to run on their system. So gcc is the fallback and the way forward on to other targets. Maybe if the LLVM backend gets going again it could be that instead /or also.

LLVM or Clang? Iow linking to LLVM to create a backend, or generating C customized for CLang? As for the former, I'm not hearing happy sounds in that camp. Anyway, long term you will have to, since BSDs and Apple targets are moving away from GCC>

True, fbc's preprocessor is C-like. However, I would say "preprocessor" is a misnomer. It's actually part of the lexer/parser, so variable types, sizes and constant values are evaluated as the source compiles. But also true, like C, can get really hairy if used for large template/generic code as there is no type safety in macros.

Yes. But as said my main gripe is that the preprocessor state of a module, say A bleeds into the declaration part (header/.h/.bi) of module B that A #includes. That is really bad behaviour because the same (.bi/.h) file can mean different things when two modules #include it. As said, C++20 contains a new modular approach to mitigate that and increase compilation speed and reduce such unexpected side effects.

I had a quick look around fp.org and looks great. I think it's been a while since I gave it a serious look. Well organized, looks like lots of contributors. You have many same packages that we translate headers for plus others. It was interesting to read through FPC's planned features and bug reports and see the some similarities with fbc. Overall looks like a great project to be involved with.

To really get a feel for it, you need to check lazarus.freepascal.org and the galleries in the wiki.

One of the great advantages is having a nearly complete toolchain in-house, at least on some targets. In the next release we debut an own resource compiler, but the big one (debugger) will take a bit longer and is already 10 years in on/off development.

As always, in projects like FB/FPC, the limitation is manpower.

[TeeEmCee]

LLVM or Clang? Iow linking to LLVM to create a backend, or generating C customized for CLang? As for the former, I'm not hearing happy sounds in that camp. Anyway, long term you will have to, since BSDs and Apple targets are moving away from GCC

Some already don't support GCC. For example Android has also dropped GCC and Nintendo Switch and PS4 have always used clang exclusively. (The fact that llvm/clang allows closed source forks is no doubt a reason for its adoption by the likes of Nintendo and nVidia. In fact even telling you that Switch uses clang is in violation of the incredibly severe NDA, but it's common knowledge.)

There is a third option, isn't there? Emit LLVM bitcode, without linking to the LLVM libraries, and without using Clang. I thought that was the original intent behind LLVM.

[marcov]

There is a third option, isn't there? Emit LLVM bitcode, without linking to the LLVM libraries, and without using Clang. I thought that was the original intent behind LLVM.

That's also fairly close to LLVM, so I lumped that in the same category. I added the "link to the llvm libraries" later.

Anyway afaik that's what we (FPC) do. From what I (indirectly) hear, the bitcode format is largely governed by CLang needs too, and not really stable out of principle.

[erik]

Yes, I also want FreeBASIC to use the LLVM code emitter. At the moment, FreeBASIC is able to generate LLVM assembly language format (.ll), but I do not know what to do with it next.

[TeeEmCee]

Yes, I also want FreeBASIC to use the LLVM code emitter. At the moment, FreeBASIC is able to generate LLVM assembly language format (.ll), but I do not know what to do with it next.

Huh? You don't need to do anything with the .ll files, "fbc -gen llvm" will generate .ll (LLVM assembly language) files, compile them to .asm (IIUC this step is unnecessary and done only in case the user wants .asm files), assemble them to .o/.obj, and link them into a library or executable. Run "fbc -gen llvm -v test.bas" and you will see the individual steps. (Edit: oh, you probably don't have LLVM installed, so fbc doesn't get any further than .ll files. Install LLVM and ensure the executables are in your PATH)

I was wrong to speak of targeting LLVM bitcode. fbc doesn't produce bit LLVM .bc bitstream files (which contain binary-encoded IR), it produces human-readable .ll assembly. I could understand the IR not being totally stable but hopefully the .ll format is a more stable target.

[erik]

Install LLVM and ensure the executables are in your PATH

I downloaded "LLVM for Win64" from their official website and installed it in %ProgramFiles%. What program do I need to run to get the assembly code from the ll file? (file llc.exe missing)

[TeeEmCee]

Well, I'm not a Windows user, but did some googling, and can see a number of people online mentioning that (for years already) there is no llc.exe provided (although you can compile it yourself).
fbc wants llc.exe, but it turns out that clang can compile .ll files too. It defaults to producing an executable/library, so you would need to use "clang -c file.ll" to produce an .o/.obj file, or "clang -S file.ll -o file.asm" for asm.

I'm writing a patch to make fbc invoke clang instead of llc on Windows.

[TeeEmCee]

See https://github.com/freebasic/fbc/pull/279
Has anyone ever actually used -gen llvm on Windows before? Was it intended that FB would provide a llc.exe build on Windows? Since clang.exe will (eventually) also be usable for -gen gcc, it seems preferably to just provide clang.exe and not need llc.exe.

BTW the LLVM backend is pretty broken at the moment, when using LLVM 10 anyway. I tried various examples and all of them longer than two lines failed to compile. Probably it requires an earlier LLVM, and marcov was right.

[dkl]

Last time I worked on this was, I think, around LLVM 3.2... the older release packages from llvm.org back then still included llc.exe. Nowadays it's not included for whatever reason, but it looks like you can get it from the llvm package in MSYS2 (might be using that because of MinGW-w64 anyways) or from other pre-built llvm toolchains, or build from sources. On Linux I haven't yet seen an llvm package that didn't include llc...

It's not in FreeBASIC, because (at least the way I remember it) -gen llvm is experimental and not usable yet. But the idea definitely was to include and use llc.exe. I mean, if clang.exe works as alternative, that's nice as backup, but probably llc.exe only would be preferable to reduce distribution size (unless maybe gcc is dropped too and clang is used instead for -gen gcc).

[TeeEmCee]

On Linux I haven't yet seen an llvm package that didn't include llc...

If you include crosscompiling toolchains, the Android NDK for Linux doesn't include it and apparently never has. I just downloaded the latest NDK to check it doesn't have it either. In fact the changelog says that "as" will be removed in a future release, and clang should be used to assemble asm instead! (But presumably llvm-as can be used.)

I don't like that my patch unconditionally uses clang instead of llc, it would be best to support either regardless of platform. For that, I suppose I should either do a test invoke of "llc --version" or recover if the exec/shell fails.

[dkl]

Checking whether llc --version works sounds interesting. Maybe it could even be used to decide which LLVM version to target, to adjust the code generation accordingly. If that doesn't work out for some reason, then yea, using clang only/always would certainly be easiest.