Making the libtinfo/ncurses dependency optional

Linux specific questions.
Post Reply
dkl
Site Admin
Posts: 3235
Joined: Jul 28, 2005 14:45
Location: Germany

Making the libtinfo/ncurses dependency optional

Post by dkl »

Hi,

today I was (once again) thinking about how odd it is that fbc links against libtinfo even though it does nothing more than PRINT to stdout. How about getting rid of that dependency, not for all FB programs, but for those that do not require it, such as fbc itself? As far as I know, this is still one of the major issues with installing fbc on Linux.

It would probably be ideal if the rtlib could be adjusted to only require linking against libtinfo if certain functionality such as LOCATE, COLOR, INKEY etc. is used, and if so, pull in libtinfo automatically. However, I also noticed that due to various reasons there are more references to libtinfo from other functions, such as EXEC(), which are also used by fbc. Plus there is some initialization done at start-up, for which I don't know how it could be made optional at link-time.

So how about having two rtlib variants - "normal console I/O" and "minimal/PRINT only"? We could #ifdef out all the termcap.h uses and disable functionality that requires it, then add a compiler option for selecting between the two, even showing compile-time errors if the minimal one was selected, but the normal variant would be required.

It would probably be more work to maintain two variants (even if it's just with #ifdefs, not actually duplicated sources), but I think I could give it a try.
adeyblue
Posts: 299
Joined: Nov 07, 2019 20:08

Re: Making the libtinfo/ncurses dependency optional

Post by adeyblue »

It looks like there aren't that many tinfo symbols used so could you just:
1. Implement stubs for them all in rtlib
2. Mark the stubs as weak symbols

profit? If the user has tinfo all the weak symbols will be replaced with the actual ones during linking, if they don't, make sure the stubs do sane things. I think it works like that anyway. You can do it in FB too with
Asm
.weak <symbolname>
End Asm

Or if it's that valuable and irreplaceable, the code is probably GPL compatable, just take its code and bung it into rtlib. I'm not Linux-y though so I don't know why it's such a headache to either install or to just plumb in a different library if this one's obsolete
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: Making the libtinfo/ncurses dependency optional

Post by caseih »

ncurses is pretty liberally-licensed. MIT licensed, and they use it on BSD, so it's definitely not as restrictive of GPL, but compatible with GPL'd and LGPL code. So probably would be just fine to copy the relevant parts of libtinfo right into the runtime library (with the right attribution of course). That would pretty handily resolve the problem, although I like the weak reference idea too.
TJF
Posts: 3809
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: Making the libtinfo/ncurses dependency optional

Post by TJF »

caseih wrote: Jun 05, 2022 3:23 ... to copy the relevant parts of libtinfo right into the runtime library ...
... would cause doubled code when the program requires to link against libtinfo. I'd prefer the
dkl wrote: Jun 04, 2022 23:32... two rtlib variants - "normal console I/O" and "minimal/PRINT only"...
solution.
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Making the libtinfo/ncurses dependency optional

Post by marcov »

Some terminfo stuff is just a bunch of ioctls that you can easily move to the core lib.
dkl
Site Admin
Posts: 3235
Joined: Jul 28, 2005 14:45
Location: Germany

Re: Making the libtinfo/ncurses dependency optional

Post by dkl »

Adding libtinfo to libfb is an interesting idea that I hadn't considered, I'll look into that to see whether licensing would allow it and whether it seems doable. I wouldn't want to copy the database but maybe the functions.

However, it's not only about libtinfo. The FB runtime is doing some things for console I/O that are not directly related to libtinfo but may also be problematic:
- running a background thread which reads from /dev/tty for INKEY & co
- disabling echoing in the input terminal at startup and restoring original state on exit. This part even is currently buggy, as was noticed for example here, but I hope that that can be fixed separately from the libtinfo issue.
- switch stdin to non-blocking (if it's a TTY)
- handles SIGWINCH
- and more, also for stdout (if it's a TTY)...
And it could be useful to be able to disable that, for programs that don't need/want it.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: Making the libtinfo/ncurses dependency optional

Post by caseih »

TJF wrote: Jun 05, 2022 6:08... would cause doubled code when the program requires to link against libtinfo.
No it wouldn't. Not unless you created a statically-linked binary, but that would bloat it anyway with a copy of glibc as well, dwarfing the little bits of libtinfo. As for RAM usage while loading and running the binary, there's also no penalty either, since libtinfo is certainly already in memory, shared and already used by many other running binaries on an Linux system.

I think the advantages to including the relevant parts of libtinfo (with or without the weak reference) are way more than having two runtime paths. Simpler, easier to maintain for the developers. I don't see a huge burden maintaining a a tiny fork of a few bits of libtinfo either. Having two runtime versions seems overly complicated to me.
dkl
Site Admin
Posts: 3235
Joined: Jul 28, 2005 14:45
Location: Germany

Re: Making the libtinfo/ncurses dependency optional

Post by dkl »

Hey again, I've experimented a bit, here are the results...

1. libfb without terminal support (DISABLE_TERM): PRINT and perhaps INKEY still work, but with reduced feature set, and basically nothing else related to console works (no LOCATE, COLOR, VIEW, CLS, etc.). This allows disabling more than just libtinfo, for example it disables all of FB's tty configuration such as disabling echoing. This would even allow FB programs to use ncurses/termcap themselves, without any interference by libfb. Problem: It would require separate libfb.a binaries, i.e. instead of the current 4 variants (libfb{,mt,pic,mtpic}.a) there would have to be an additional "noterm" variant for each, so 8 in total. That would be a rather ugly/annoying change to the FB makefile.

2. libfb containing parts of libtinfo. Even with the minimal required code copied from ncurses, it still was ~17k LOC. The required symbols are PC, UP, BC, ospeed, tgetstr, tgoto, tgetent, tgetflag, tgetnum, tputs, and all their dependencies, which are a lot of libtinfo/ncurses internals. I think this is not an option, it's just too much. Besides, this is basically the same as linking against libtinfo/libncurses statically - no need to copy the code into libfb, when we can just compile ncurses additionally. That's certainly an option, at least for deployment of fbc itself, and it doesn't even require any changes to FB.

3. Weak symbols in libfb (dummy implementations of tgetstr & co which return errors only): It works - if omitting libtinfo, libfb will use its weak symbols instead, and if linking libtinfo before libfb, libfb will use the strong symbols from libtinfo (link order matters for this, weak/strong alone does not do the trick...). This is nicer than the DISABLE_TERM variant, because it allows eliminating the libtinfo dependency with a link-time option, instead of requiring an rtlib compile-time option. Also there are no additional code paths in the rtlib - The code for handling the errors from the dummy tgetstr & co implementations is the same as when using proper libtinfo. Interestingly, this even causes libfb to skip the additional tty configuration (such as disabling echoing). The only problem I'm still wondering about here is whether this would interfere with FB programs trying to use ncurses/libtinfo directly (in comparison, DISABLE_TERM would allow doing that in an FB program, without having the FB runtime do it too), but this isn't working now either and could be allowed by introducing a new overridable global var, like the existing __fb_enable_vt100_escapes.
TJF
Posts: 3809
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: Making the libtinfo/ncurses dependency optional

Post by TJF »

Thanks for the info.

Now I'd favor the weak symbols (over the eight rtlib variants).
TeeEmCee
Posts: 375
Joined: Jul 22, 2006 0:54
Location: Auckland

Re: Making the libtinfo/ncurses dependency optional

Post by TeeEmCee »

Being able to compile rtlib without curses/terminfo is needed for platforms that don't have it, that's why I added a DISABLE_NCURSES build option in this commit on my android branch. However I intended to rename it to DISABLE_TERMINFO. And I made no attempt to behave well if any console functions are called. Of course there are also other platforms without termcap.h.

However I think doubling the number of different libfb configurations that are needed should be avoided if at all possible.

To create portable Unix builds of the OHRRPGCE I use your 3rd option, linking to these stub functions. I didn't bother making them weak because I don't link to tinfo/ncurses (I don't use fbc to link when creating portable binaries -- a -nodeflibs argument would be very useful. Without it, I use gcc to invoke the linker instead. But I'd use gcc anyway, for things such as ASan). INKEY works.
The only problem I'm still wondering about here is whether this would interfere with FB programs trying to use ncurses/libtinfo directly (in comparison, DISABLE_TERM would allow doing that in an FB program, without having the FB runtime do it too), but this isn't working now either
Doesn't ncurses in a FB program work? It works for me, although I haven't tried using it much. Provided the libraries are linked in the correct order I would think the weakly linked symbols shouldn't be a problem.

It's not possible to detect with certainty whether a FB program uses console functions such as LOCATE, because even if it contains them it may call SCREEN or SCREENRES (possibly conditionally).

Even if we can fix fbc not being portable, it would still be a shame if FB programs which do use console functions aren't portable. The easiest solution to that would seem to be to statically link to libtinfo, except AFAIK it isn't normally provided as an .a on GNU/Linux, so it would require shipping a copy of it which is an extra nuisance.
dkl
Site Admin
Posts: 3235
Joined: Jul 28, 2005 14:45
Location: Germany

Re: Making the libtinfo/ncurses dependency optional

Post by dkl »

Interesting, that's the same as my DISABLE_TERM attempt and the termcap function stubs.

Good to know that an rtlib compile-time option like DISABLE_TERM/TERMINFO/NCURSES is needed anyways for portability. I see no problem having it, as long as it's not necessary to build & deploy every possible variant everytime. The FB runtime has other options like DISABLE_GPM or DISABLE_X11 already. So maybe both ideas (DISABLE_* option and weak symbols) can be combined.
TeeEmCee
Posts: 375
Joined: Jul 22, 2006 0:54
Location: Auckland

Re: Making the libtinfo/ncurses dependency optional

Post by TeeEmCee »

Yes, that's what I was recommending.

Well, our patches aren't the same: I wrote the termcap stubs to keep as much working as possible by returning success codes so rtlib's hInit() continues through the normal setup. getkey, multikey, inkey, fb_keyhit, getmouse, color, width (and the SIGWINCH and other signal handlers) and disabling of key echoing all work. cls, locate and line input don't. But I could extend it slightly, assuming a VT100-compatible console, to make those work too. (Doesn't FB assume VT100 compatible anyway)?

Also, even on Android where there's no termcap/terminfo it's still useful to emit vt100 codes in case of producing an executable that prints to a terminal emulator. Such as when when using the adb utility (part of the Android SDK) to get a remote shell on an Android phone, or running an terminal app.
dkl
Site Admin
Posts: 3235
Joined: Jul 28, 2005 14:45
Location: Germany

Re: Making the libtinfo/ncurses dependency optional

Post by dkl »

TeeEmCee wrote: Jun 22, 2022 3:23 Well, our patches aren't the same: I wrote the termcap stubs to keep as much working as possible by returning success codes so rtlib's hInit() continues through the normal setup. getkey, multikey, inkey, fb_keyhit, getmouse, color, width (and the SIGWINCH and other signal handlers) and disabling of key echoing all work. cls, locate and line input don't. But I could extend it slightly, assuming a VT100-compatible console, to make those work too. (Doesn't FB assume VT100 compatible anyway)?
That's very interesting, I'm wondering about what's best if running without termcap support (no matter whether it was disabled at compile-time, or initialization fails at runtime).
a) Disable all (99%) of console functionality (by setting __fb_con.inited = FALSE), which already is a somewhat well supported error code path that triggers some simple fallbacks for Print, Input, Inkey and basically nothing else really works.
b) Keep as much functionality working/enabled as possible, since there is a lot that doesn't actually require termcap (by setting __fb_con.inited = INIT_* as usual and perhaps adding a termcap_inited boolean, or such). This would probably be more useful, but it would also effectively become a new "in between" rtlib mode. At least if it's not just an rtlib compile-time opt-in, but can be triggered at runtime because termcap initialization failed, then this might require additional work to be implemented sanely (thinking of additional error checking, such as handling fb_hTermOut() failure, possibly returning "illegal function call" errors where needed, similar to what's done in __fb_con.inited == FALSE mode).

Regarding the termcap function stubs, I've encountered an issue with that. It seems that weak symbols don't work as I would have expected. According to my tests:

Code: Select all

stubs in libfb.a, libtinfo.so, -lfb -ltinfo => stubs used
stubs in libfb.a, libtinfo.a , -lfb -ltinfo => stubs used
stubs in libfb.a, libtinfo.so, -ltinfo -lfb => stubs not used
stubs in libfb.a, libtinfo.a , -ltinfo -lfb => stubs used
and this was the same no matter whether the stubs were weak or not. So weak/strong doesn't seem to matter in this scenario, but linking order does (libs on command line, and object files in libs). If I didn't miss anything, the stubs don't seem to be a working solution either way. (But I still think the rtlib compile-time option is good to have.)
dkl
Site Admin
Posts: 3235
Joined: Jul 28, 2005 14:45
Location: Germany

Re: Making the libtinfo/ncurses dependency optional

Post by dkl »

After talking to Jeff I noticed that DISABLE_NCURSES from the Android branch has been merged now, which is good since it covers what I had in mind with DISABLE_TERMCAP. Plus, the DISABLE_NCURSES patch made be notice that I disabled more parts of fb_hInit() than necessary in my DISABLE_TERMCAP patch.

Anyways, @TeeEmCee, I agree that it makes sense to keep as much of libfb as possible working even with DISABLE_NCURSES. Based on that it does indeed seem useful to allow using the hard-coded terminal escape sequences even with DISABLE_NCURSES, especially since you mentioned a use case for this for Android (having to build without ncurses, but still running in a terminal that supports escapes).

Additionally, if you're still interested in adding the missing hard-coded VT100-compatible escape sequences, I think that would be a good idea. I don't know much about VT100 escape codes or whether this was ever intended for the Linux rtlib, but since it already contains some, I see no problems adding some more. Should it be necessary to disallow all escape sequences for some reason, that could still be a separate DISABLE_* option in the rtlib (which would be a superset of DISABLE_NCURSES).
Post Reply