DOS FAQ

Forum for discussion about the documentation project.
Post Reply
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

DOS FAQ

Post by angros47 »

I noticed that the FAQ for the DOS version of FreeBasic, https://www.freebasic.net/wiki/FaqDOS, still have some outdated information.

In particular: it states that the DOS version doesn't support multithreading, nor dynamic libraries. Actually, since version 1.06, they are both supported: multithreading is possible thanks to pthreads, that is integrated in the FreeBasic runtime: since it relies on signals, it has some limitations: it doesn't work under DosBox emulator (since it doesn't emulate signals), and crashes, although it works under DosEMU and under the dos box of Windows XP. Also, if another program is called (for example using SHELL) all the threads are paused, so it cannot be used to magically make DOS multitasking, but it allows to compile under DOS programs using multithreading.
Dynamic libraries are also supported: differently from Windows (that has the .DLL format) and Linux (that has the .so format), DOS doesn't have a standard format for dynamic libraries, but the DJGPP environment has the format .DXE (originally used to add the FPU emulation on processors not supporting it, but versatile enough to provide most of the functions offered by dynamic libraries). FreeBasic does support both creation and loading of those libraries (the only difference is that the command DyLibLoad requires to specify the filename with extension, while in Linux and Windows extension is not needed)

These improvements were developed years ago by Monochromator, and included in the official version 1.06 by me some years ago, but I noticed that documentation has never been updated. Even the pages about ThreadCreate, DyLibLoad and so on still state that these features are not available on DOS

Also, the DOS faq mentions that sound is not supported, while my sfx library provides some support for DOS, as well (Sound Blaster and WSS for PCM sound, MPU-401 and OPL2 for MIDI)

I wonder if the documentation for the DOS FAQ and for the commands relative to threads and to dynamic libraries should be updated accordingly
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: DOS FAQ

Post by fxm »

I do not think I am the right person to do these DOS platform specific updates (I only use Windows).
But yet I am surprised to see no trace of such improvements in the 'changelog.txt' file.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: DOS FAQ

Post by coderJeff »

Old Posts:
- Dynamic link libraries in DOS
- Multi-threading in DOS

Was added in fbc 1.06.0 but no changelog.txt message was made. This was around the time I got involved with fb (again) in 2017. I think the commits / merge were made by dkl? But I don't remember.

I think I must have tested the DLL on winxp - with a full djgpp install nearby and on the path, with DJDIR configured.

Obviously, no one in 5 years has stepped forward to take this to the finish line. Let's see if we can make some progress...

I am trying to create a windows to dos cross compiler, so that might be some of the issues, not sure. I can at least cross compile fbc itself and run it in dosbox - which allows me to copy over files to the emulator without having to involve other PC's, funky network set-ups, or file transfer schemes.

I'm using the tools from: https://github.com/andrewwutw/build-djg ... s/tag/v3.0
- specifically the mingw gcc 9.3 tools for cross compile.
- I tried building from sources, but it didn't work out for me (yet, but I only tried for 2 weekends). For now I'm using the binary release generously provided by Andrew Wu.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: DOS FAQ

Post by coderJeff »

DOS DLL:
Cross compiling from windows to dos needs some corrections in src/compiler/fbc.bas - the section of code that runs dxe3gen is gated by host __FB_DOS__ instead of the '-target dos' option FB_COMPTARGET_DOS. With that fixed, let's see where we are at:

Code: Select all

#cmdline "-target dos -arch 486 -dll"
function add2( byval x as integer, byval y as integer ) as integer export
    return x + y
end function
How about the cross compiler from windows:
fbc-win32 func.bas
ld options in 'ldopt.tmp': -o "func.dxe" -I "func_il.a" -U "func.o"
making DXE: d:\fb.git\\bin\dos\dxe3gen.exe @ldopt.tmp
d:\fb.git\\bin\dos\dxe3gen.exe: no output file name given (-h for help)
Seems that cross compiler version of dxe3gen.exe won't take an options file @ldopt.tmp
How about the dos version of dxe3gen.exe in dosbox:
ld options in 'ldopt.tmp': -o "func.dxe" -I "func_il.a" -U "func.o"
making DXE: d:\fb\fb1100\\bin\dos\dxe3gen.exe @ldopt.tmp
Error: neither DXE_LD_LIBRARY_PATH nor DJDIR are set in environment
Seems some work to do.
Also just noticed: the exe path has double '\\' in the path. I don't think that's causing an issue, but it probably means some poor handling of paths in fbc (as a separate issue).

OK will try setting the DXE_LD_LIBRARY_PATH manually next.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: DOS FAQ

Post by coderJeff »

DOS DLL's "works" but I don't know how useful it is in current state:

I'm thinking that Monochromator must have made a customized dxe3gen (maybe ld too?) because there are a few issues I found with available builds of the djgpp tools:
- dxe3gen needs DXE_LD_LIBRARY_PATH to be set to find dxe.ld linker script
- ld is looking for libraries prefixed with "lib"
- on cross compile version, dxe3gen wants to call i586-pc-msdosdjgpp-ld for linking

I've fixed these by prefixing "lib" to the import library (should you want to link to the import library instead of dylibload at run time). On 8.3 file systems this is a severe limitation since the import library is going to be named "LIB" + base + "_IL.A" and just gets truncated.

Fixed: if DXE_LD_LIBRARY_PATH is not already set, fbc will set it before calling dxe3gen. So user doesn't have to set manually.

At least one call to DyLibLoad("") is needed to load / initialize the fb runtime. This part I don't quite understand because when making an executable we are still statically linked to libfb. Having DyLibLoad() in the program will link in every function from libfb making the executables at least the size of the entire runtime lib. So maybe there is some other file naming manipulation / import library usage I don't know about yet.

But if libfb.dxe does actually load dynamically then all the dependent libraries need to be dynamic too? Not sure.

I am tempted to drop the libfb.dxe altogether and see if user made DLL's can work by themselves. Honestly, it doesn't make sense that libfb.dxe be loaded dynamically, unless I misunderstand and user made DLL's need libfb.dxe too. In which case it seems every library needs to be dynamic. So I am a little confused as to how this is supposed to work overall, or if it is badly implemented so far.
angros47
Posts: 2321
Joined: Jun 21, 2005 19:04

Re: DOS FAQ

Post by angros47 »

Import libraries are not needed for DOS DLL's.

Actually, under Windows or Linux dynamic linking can happen in two different ways: load-time, or run-time:

https://stackoverflow.com/questions/205 ... ic-linking
https://www.tutorialspoint.com/dll/dll_quick_guide.htm

Basically, load-time is when you use #inclib to specify the library name, and "declare sub/function" to specify the symbols you want to use. If the library is not present, the program will immediately return an error message, and won't work

Run-time is when you use dylibload to specify the library name, and "dim as sub/function" followed by "dylibsymbol" to specify the symbols.

Import libraries are useful only when you want to use load-time dynamic linking, they are not used for run-time dynamic linking

Under DOS, load-time linking is not supported at all (it would require a different loader for executables, and the loader is part of the OS, so it cannot be replaced); run-time linking, on the other hand, can be managed entirely by the application, so it is possible, and that's what happens in dxe dynamic libraries.

For that reason, the freebasic library itself must be statically linked, otherwise there would be no way to load it (DOS won't load it, since its loader is much simpler than the ones of Linux or Windows, and the executable could not load anything, since the code to load dynamic libraries is in the freebasic library).
About the fact that DyLibLoad will include the entire fb runtime, there is a reason for that: the dynamic libraries (the dxe files) will likely be created with FreeBasic, and they will have to be able to use the FB runtime (otherwise, the library wouldn't be able to use commands like PRINT, OPEN and so on); yet, the FB runtime is not included in the .dxe file, instead the version included in the main executable is used (the main executable must have the FB runtime included into it, for what I said before, so it would be a waste of memory to include it in each library again).

Still, the compiler cannot know in advance which functions will be needed by the dynamically linked module, so it has no choice, it must include all of them, to be safe.

Since the command DyLibLoad needs to pass to the dynamic library a list of ALL the symbols of the freebasic runtime, and since that list changes each time the Freebasic runtime library is modified, such a list is stored in the file symb_reg.txt, that is generated automatically by using maksymbr.bas

The procedure is described here: viewtopic.php?p=186593#p186593

The list of symbols is acquired from the file libfb.dxe, so it is necessary to create that file to have an up-to-date symbol list. After that, it is possible to update the file symb_reg.txt, and then recompile the runtime library with it (complicated, I know, but I wasn't the one who invented dxe3gen). So, the file libfb.dxe is just a temporary intermediate file needed to produce the updated symbol list, and can be removed with no issues.

I hope this answers your questions.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: DOS FAQ

Post by coderJeff »

Thanks, most helpful was:
angros47 wrote: Jun 05, 2022 17:55 So, the file libfb.dxe is just a temporary intermediate file needed to produce the updated symbol list, and can be removed with no issues.
I was really hung up on libfb.dxe. When I wrote src/rtlib/dos/libfbexp.mk to help automate building symb_reg.txt, I didn't delete libfb.dxe, otherwise the build process would constantly want to make it and delete it. I had it in my mind that it was needed - which is incorrect.
angros47 wrote: Jun 05, 2022 17:55 Import libraries are useful only when you want to use load-time dynamic linking, they are not used for run-time dynamic linking

Under DOS, load-time linking is not supported at all (it would require a different loader for executables, and the loader is part of the OS, so it cannot be replaced);
I am probably missing the nuances of definitions, but it *probably doesn't matter*. I did some reading on delorie's site and think I have a handle on DXE's. Import libraries can be used with DOS, just not with fbc - which is what I think you were trying to say.

So for now, it doesn't seem like there is any point to generate the import library and should just be removed from fbc's build steps.

I am continuing to try out this feature and find it's limitations - which are several: So far,

Limitations / Cautions:
- DXE must be loaded through dylibload()
- procedures must be accessed through dylibsymbol()
- no global variables shared between DXE's (extern / common shared, etc)
- no gfx functions allowed in DXE's (dylibload() will fail)
- no static libraries from the DXE at all (dylibload() will fail)
- no complex TYPEs or EXTENDed types (types having member procedures) can be exported
- DXE must only use procedures available in libfb or itself or dylibload() will fail (even though compilation of the DXE will have succeeded)
- EXPORT is ignored, all public procedures are exported
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: DOS FAQ

Post by coderJeff »

Started to add some updates for DOS dynamic link libraries (DXE's):

Export - Update platform differences for DOS, exported is ignored
DylibSymbol - Update platform differences for DOS, ordinals are not supported
DylibFree - Remove Platform differences section
DylibLoad - libname must have the .dxe extension on DOS
-dll - Add platform difference notes
-dylib - Add link to -dll, DXE files on DOS

Import libraries can work with dos DXE's with some limitations, but improves usability. Mainly need to call DylibOpen("") before anything else. Can kind of do it with a module constructor, but constructor ordering becomes important, and reliable constructor execution order is not guaranteed. I'll post an example soon. (Sorry, haven't even looked at dos threading yet. Hopefully not too many more weekends).
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: DOS FAQ

Post by D.J.Peters »

ProPgSharedLibraries
Using Shared Libraries on Windows
Using Shared Libraries on Linux
Using Shared Libraries on DOS <-- is missing

Joshy
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: DOS FAQ

Post by coderJeff »

Added: Shared Libraries - DOS

- with examples as a starting point for interested users.
- DOS is main subject
- examples will also work on Win/Lin because of the added '#fidef __FB_DOS__' sections.
- I feel like it should have it's own page.

@fxm, I didn't add any links to the page, but was thinking it could just link from Shared Libraries

EDIT: still to do:
- document DXE_LD_LIBRARY_PATH environment variable (set when dxe3gen is invoked, if not already set)
- document usage of 'dxe.ld' linker script for dxe3gen
- add the (or add to the) version sections - dxe.ld was not included in any fbc release yet, but 'dxe.ld' is needed for dxe3gen
- backlinks to Shared Libraries - DOS from platform differences sections in Dylib* pages, etc
- no exports are generated (by build or in release) for DOS multithreaded libs, so combination of DXE + mt probably won't work, but haven't tested
- I still didn't start on testing the multithread stuff yet.
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: DOS FAQ

Post by fxm »

coderJeff wrote: Jun 25, 2022 15:26 Added: Shared Libraries - DOS

@fxm, I didn't add any links to the page, but was thinking it could just link from Shared Libraries
Okay, I'll do it.

[edit]
Done.
Last edited by fxm on Jun 25, 2022 15:55, edited 1 time in total.
Reason: Update.
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: DOS FAQ

Post by fxm »

coderJeff wrote: Jun 25, 2022 15:26 Added: Shared Libraries - DOS

- backlinks to Shared Libraries - DOS from platform differences sections in Dylib* pages
Done.
Post Reply