Runtime location of a Function by name in a string

External libraries (GTK, GSL, SDL, Allegro, OpenGL, etc) questions.
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Runtime location of a Function by name in a string

Post by fxm »

@coderJeff,

Have you seen the posts above that conclude that 'Common' and 'Extern Import' both work to share variables with DLLs (even using 'DyLibLoad') on Linux, but neither work on Windows ?
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Runtime location of a Function by name in a string

Post by coderJeff »

It's been years since I looked at this kind of stuff. Shared library loading on Windows versus Linux/Unix is architecturally different. I remember investigating the differences in fbc's run time library, for example with graphics functions and file handling functions.

On Unix/Linux, loading a shared library works very similar to linking a static library and symbols declared and allocated in the either the main program or the shared library can be linked to and accessed by the other.

On Windows, a shared library must have all of it's symbols fully resolved except for those imported from another DLL. As far as I know the DLL can't automatically access symbols in the main program. The main program can only access automatically what is specified in the DLL's export table.

Currently on windows, there is no valid syntax to specify that variables should be placed in the DLL's export table. This would be the counterpart to EXTERN IMPORT.

We could try adding EXTERN EXPORT or SHARED EXPORT. It would be nice if there was someone willing take it on and take it to the finish line write all the tests and so on.
wallyg
Posts: 267
Joined: May 08, 2009 7:08
Location: Tucson Arizona

Re: Runtime location of a Function by name in a string

Post by wallyg »

Thank you all for all of your sage advice and for taking your time to try and help me work this out. After reading all of your comments, I think I have a way that will work for me from your examples.

If someone could get this all documented as part of the official documentation, I am sure it would be a great help to anyone else looking to do something similar.

Again thank you all for all of your advice and comments. It is wonderful to see the community work together to help one another.

I have been using Windows since 1.0 and have never used Linux. Do you think I am too long in the tooth to try using Linux on an old Windows laptop I used to use at work before I sold my business ( 7 years now ) and retired? It is currently running Windows 10. If so any suggestions for which Linux I would find to be the easiest for a novice to load / learn / use? Could it be loaded in parallel to Windows 10?

Wally
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Runtime location of a Function by name in a string

Post by fxm »

Such a synthetic sentence is already in the documentation (Static Libraries / Shared Libraries, in paragraph 'Sharing variables with static/shared library'):
... passing a parameter (by value or by reference) to a library procedure or returning a variable (by value or by reference) from a library function allows to indirectly exchange data (by value) or share data (by reference) with a shared library.
The datatype of this variable can also be a procedure pointer.
Perhaps an illustrative example (simple) is missing in documentation.
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: Runtime location of a Function by name in a string

Post by caseih »

wallyg wrote:Could it be loaded in parallel to Windows 10?
I think you could find Linux fun and there are a couple of ways of running it on top of Linux. First is to use VirtualBox to create a virtual computer that you can install any Linux distribution from an ISO file. The second option is to install the Microsoft Windows Subsystem for Linux 2. WSL2 is aimed more at developers, though.

For distro I might recommend Ubuntu or Linux Mint. Both contain easily-installable development tools and are probably familiar to most of the Linux users on this forum.
Imortis
Moderator
Posts: 1923
Joined: Jun 02, 2005 15:10
Location: USA
Contact:

Re: Runtime location of a Function by name in a string

Post by Imortis »

wallyg wrote:...If so any suggestions for which Linux I would find to be the easiest for a novice to load / learn / use? Could it be loaded in parallel to Windows 10?...
Zorin OS is a distro made for the purpose of making a transition from Windows to Linux as painless as possible.

https://zorin.com/os/
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Runtime location of a Function by name in a string

Post by fxm »

Loading Shared Libraries Dynamically

@coderJeff, @TJF and the others,

In the documentation, it is specified that 'DyLibSymbol' can also return a pointer to a variable.
I never managed to get that to work on Windows (works with pointers to procedures) !
How should the variable be declared in the dll ?
What with Linux ?
TJF
Posts: 3809
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: Runtime location of a Function by name in a string

Post by TJF »

@fxm

DYLIBSYMBOL is used in the executable loading the library, so you're asking for unidirectional transfer only.

On LINUX the variables get declared in the DLL by either COMMON or EXTERN IMPORT, see example in post viewtopic.php?f=14&t=29411&start=15#p285760
coderJeff wrote:We could try adding EXTERN EXPORT or SHARED EXPORT. It would be nice if there was someone willing take it on and take it to the finish line write all the tests and so on.
Why adding new features? Isn't it senseful to make the existing work, first?

From my point of view the compiler could implement a hidden dll_init() function, transfering/connecting the variables (PTRs) declered as COMMON in an (also hidden) UDT to either side, see examples in viewtopic.php?f=14&t=29411&start=15#p285797 and viewtopic.php?f=14&t=29411&start=15#p285810

This can get implemented for both, wodniws and LINUX (and DOS?), in order to avoid OS differences.
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Runtime location of a Function by name in a string

Post by fxm »

@TJF,

But my specific question for Linux is:
Can we retrieve in the main code the address of a variable, declared only in the DLL, using 'DyLibSymbol' ?
TJF
Posts: 3809
Joined: Dec 06, 2009 22:27
Location: N47°, E15°
Contact:

Re: Runtime location of a Function by name in a string

Post by TJF »

fxm wrote:Can we retrieve in the main code the address of a variable, declared only in the DLL, using 'DyLibSymbol' ?
That should be possible. I tried to figure it out, but failed using that code (test_dll.bas)

Code: Select all

EXTERN IMPORT TestInDll ALIAS "TestInDll" AS LONG

TestInDll = 987
which compiles by fbc -dll dll_test.bas to a library binary (libtest_dll.so), but the executable fails to load that binary (DYLIBLOAD("test_dll") returns zero).

I don't know what happens here, and I've no time to figure it out ATM, sorry.
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Runtime location of a Function by name in a string

Post by fxm »

I thought we should also define the variable in the DLL (but without 'Import'):

Code: Select all

EXTERN TestInDll ALIAS "TestInDll" AS LONG

Dim As Long TestInDll = 987
But when I do this on Windows, that does not work at run time (null pointer returned by 'DyLibSymbol').
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Runtime location of a Function by name in a string

Post by coderJeff »

fxm wrote:But when I do this on Windows, that does not work at run time (null pointer).
IMPORT keyword has no meaning in linux. It is silently ignored.

On windows, fbc currently has no syntax to put a variable symbol in the export table. Which is why the only example on Import wiki page is in C.

The variable must be shared (global) otherwise it's scoped in the implicit (dll) main.

Linux:

Code: Select all

'' fbc -dll test_dll.bas
extern TestInDLL alias "TestInDLL" as long
dim shared TestInDLL as long = 123

Code: Select all

'' test
var libhandle = dylibload( "test_dll")
dim TestInDLL as long ptr = dylibsymbol( libhandle, "TestInDLL" )
print *TestInDLL
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Runtime location of a Function by name in a string

Post by fxm »

fxm wrote:Such a synthetic sentence is already in the documentation (Static Libraries / Shared Libraries, in paragraph 'Sharing variables with static/shared library'):
... passing a parameter (by value or by reference) to a library procedure or returning a variable (by value or by reference) from a library function allows to indirectly exchange data (by value) or share data (by reference) with a shared library.
The datatype of this variable can also be a procedure pointer.
Perhaps an illustrative example (simple) is missing in documentation.
Workaround principle on Windows for exchanging data (by value) between main and dll code:
(dll loadable statically or dynamically as desired)

Code: Select all

' dllExchangeData.bas to be compile with -dll
' Exchanging data between main and dll code

' 'Alias' clause (in addition to 'Export') allows compatibility with dll loaded statically or dynamically

' exchange main variable
Sub passIntByVal Alias"passIntByVal"(Byval i As Integer) Export
    Print "  dll code receives by value main integer"
    Dim As Integer Idll = i
    Print "  dll code prints main integer"
    Print "  " & Idll
End Sub

' exchange dll variable
Dim Shared As Integer J = 5678
Function returnIntByVal Alias"returnIntByVal"() As Integer Export
    Print "  dll code returns by value dll integer"
    Return J
End Function

Code: Select all

' mainExchangeData.bas
' Exchanging data between main and dll code

' 'Alias' clause allows compatibility with dll loaded statically or dynamically

' dll loaded statically
    #inclib "dllExchangeData"
    Declare Sub passIntByVal Alias"passIntByVal"(Byval i As Integer)
    Declare Function returnIntByVal Alias"returnIntByVal"() As Integer
' or dll loaded dynamically
    'Dim As Any Ptr libhandle = DyLibLoad("dllExchangeData")
    'Dim As Sub(Byval i As Integer) passIntByVal = DyLibSymbol(libhandle, "passIntByVal")
    'Dim As Function() As Integer returnIntByVal = DyLibSymbol(libhandle, "returnIntByVal")


' exchange main variable
Dim Shared As Integer I = 1234
Print "main code passes by value main integer to dll code"
passIntByVal(I)

Print

' exchange dll variable
Print "main code requests by value dll integer from dll code"
Dim As Integer J = returnIntByVal()
Print "main code prints dll integer"
Print "" & J

Print

' dll loaded dynamically
    'DyLibFree(libhandle)

Sleep

Code: Select all

' Output:

main code passes by value main integer to dll code
  dll code receives by value main integer
  dll code prints main integer
  1234

main code requests by value dll integer from dll code
  dll code returns by value dll integer
main code prints dll integer
5678
Workaround principle on Windows for sharing data (by reference) between main and dll code:
(dll loadable statically or dynamically as desired)

Code: Select all

' dllShareData.bas to be compile with -dll
' Sharing data between main and dll code

' 'Alias' clause (in addition to 'Export') allows compatibility with dll loaded statically or dynamically

' share main variable
Dim Shared Byref As Integer Idll = *Cptr(Integer Ptr, 0)
Sub passIntByRef Alias"passIntByRef"(Byref i As Integer) Export
    Print "   dll code receives by reference main integer"
    @Idll = @i
End Sub

Sub printIdll Alias"printIdll"() Export
    Print "   dll code prints its own reference"
    Print "   " & Idll
End Sub

Sub incrementIdll Alias"incrementIdll"() Export
    Idll += 1
End Sub

' share dll variable
Dim Shared As Integer Jdll = 5
Function returnIntByRef Alias"returnIntByRef"() Byref As Integer Export
    Print "   dll code returns by reference dll integer"
    Return Jdll
End Function

Sub printJdll Alias"printJdll"() Export
    Print "   dll code prints its dll integer"
    Print "   " & Jdll
End Sub

Sub incrementJdll Alias"incrementJdll"() Export
    Jdll +=1
End Sub

Code: Select all

' mainShareData.bas
' Sharing data between main and dll code

' 'Alias' clause allows compatibility with dll loaded statically or dynamically

' dll loaded statically
    #inclib "dllShareData"
    Declare Sub passIntByRef Alias"passIntByRef"(Byref i As Integer)
    Declare Function returnIntByRef Alias"returnIntByRef"() Byref  As Integer
    Declare Sub printIdll Alias"printIdll"()
    Declare Sub printJdll Alias"printJdll"()
    Declare Sub incrementIdll Alias"incrementIdll"()
    Declare Sub incrementJdll Alias"incrementJdll"()
 ' or dll loaded dynamically
    'Dim As Any Ptr libhandle = DyLibLoad("dllShareData")
    'Dim As Sub(Byref i As Integer) passIntByRef = DyLibSymbol(libhandle, "passIntByRef")
    'Dim As Function() Byref  As Integer returnIntByRef = DyLibSymbol(libhandle, "returnIntByRef")
    'Dim As Sub() printIdll = DyLibSymbol(libhandle, "printIdll")
    'Dim As Sub() printJdll = DyLibSymbol(libhandle, "printJdll")
    'Dim As Sub() incrementIdll = DyLibSymbol(libhandle, "incrementIdll")
    'Dim As Sub() incrementJdll = DyLibSymbol(libhandle, "incrementJdll")

' share main variable
Dim Shared As Integer Imain = 1
Print "main code passes by reference main integer to dll code"
passIntByref(Imain)
Print "main code requests dll code to print its own reference"
printIdll()
Print "main code increments its main integer value"
Imain += 1
Print "main code requests dll code to print its own reference"
printIdll()
Print "main code requests dll to increments its own reference"
incrementIdll()
Print "main code prints its main integer"
Print "" & Imain

Print

' share dll variable
Dim Shared Byref As Integer Jdll = *Cptr(Integer Ptr, 0)
Print "main code requests by reference dll integer from dll code"
Dim As Integer Ptr pJdll = @(returnIntByRef())
Print "main code receives by reference dll integer"
@Jdll = pJdll
Print "main code prints its own reference"
Print "" & Jdll
Print "main code requests dll to increment its dll integer value"
incrementJdll()
Print "main code prints its own reference"
Print "" & Jdll
Print "main code increments its own reference"
Jdll += 1
Print "main code requests dll code to print its dll integer"
printJdll()
Print

' dll loaded dynamically
    'DyLibFree(libhandle)

Sleep

Code: Select all

' Output:

main code passes by reference main integer to dll code
   dll code receives by reference main integer
main code requests dll code to print its own reference
   dll code prints its own reference
   1
main code increments its main integer value
main code requests dll code to print its own reference
   dll code prints its own reference
   2
main code requests dll to increments its own reference
main code prints its main integer
3

main code requests by reference dll integer from dll code
   dll code returns by reference dll integer
main code receives by reference dll integer
main code prints its own reference
5
main code requests dll to increment its dll integer value
main code prints its own reference
6
main code increments its own reference
main code requests dll code to print its dll integer
   dll code prints its dll integer
   7
Workaround principle on Windows for call-backing procedures between main and dll code:
(dll loadable statically or dynamically as desired)

Code: Select all

' dllCallBack.bas to be compile with -dll
' Call-Backing procedures between main and dll code

' 'Alias' clause (in addition to 'Export') allows compatibility with dll loaded statically or dynamically

' call-back main procedure
Sub passSubPtr Alias"passSubPtr"(Byval p As Sub()) Export
    Print "  dll code receives main sub ptr"
    Dim As Sub() ps = p
    Print "  dll code calls main sub"
    ps()
End Sub

' call-back dll procedure
Declare Sub printFromDll()

Function returnSubPtr Alias"returnSubPtr"() As Sub() Export
    Print "  dll code returns dll sub ptr"
    Return @printFromDll
End Function

Sub printFromDll()
    Print "  Hello from dll sub body!"
End Sub

Code: Select all

' mainCallBack.bas
' Call-Backing procedures between main and dll code

' 'Alias' clause allows compatibility with dll loaded statically or dynamically

' dll loaded statically
    #inclib "dllCallBack"
    Declare Sub passSubPtr Alias"passSubPtr"(Byval ps As Sub())
    Declare Function returnSubPtr Alias"returnSubPtr"() As Sub()
 ' or dll loaded dynamically
    'Dim As Any Ptr libhandle = DyLibLoad("dllCallBack")
    'Dim As Sub(Byval ps As Sub()) passSubPtr = DyLibSymbol(libhandle, "passSubPtr")
    'Dim As Function() As Sub() returnSubPtr = DyLibSymbol(libhandle, "returnSubPtr")

Sub printFromMain()
    Print "Hello from main sub body!"
End Sub

' call-back main procedure
Print "main code passes main sub ptr to dll code"
passSubPtr(@printFromMain)

Print

' call-back dll procedure
Print "main code requests dll sub ptr from dll code"
Dim As Sub() ps = returnSubPtr()
Print "main code calls dll sub"
ps()

Print

' dll loaded dynamically
    'DyLibFree(libhandle)

Sleep

Code: Select all

' Output:

main code passes main sub ptr to dll code
  dll code receives main sub ptr
  dll code calls main sub
Hello from main sub body!

main code requests dll sub ptr from dll code
  dll code returns dll sub ptr
main code calls dll sub
  Hello from dll sub body!
[edit]
Second example (sharing by reference) added in documentation (Shared Libraries)
Last edited by fxm on Oct 12, 2021 20:32, edited 7 times in total.
Xusinboy Bekchanov
Posts: 783
Joined: Jul 26, 2018 18:28

Re: Runtime location of a Function by name in a string

Post by Xusinboy Bekchanov »

coderJeff wrote:It's been years since I looked at this kind of stuff. Shared library loading on Windows versus Linux/Unix is architecturally different. I remember investigating the differences in fbc's run time library, for example with graphics functions and file handling functions.

On Unix/Linux, loading a shared library works very similar to linking a static library and symbols declared and allocated in the either the main program or the shared library can be linked to and accessed by the other.

On Windows, a shared library must have all of it's symbols fully resolved except for those imported from another DLL. As far as I know the DLL can't automatically access symbols in the main program. The main program can only access automatically what is specified in the DLL's export table.
How can we get dynamic libraries to work in Linux like in Windows?
j8w344c6
Posts: 184
Joined: Oct 25, 2021 10:18

Re: Runtime location of a Function by name in a string

Post by j8w344c6 »

Xusinboy Bekchanov wrote:How can we get dynamic libraries to work in Linux like in Windows?
I think you can't since they are fundamentally different.
Post Reply