Runtime location of a Function by name in a string

[caseih]

Here's an example of an FB dll that calls a subroutine that is defined elsewhere (in the main program in this case):

Code: Select all

declare sub callback alias "callback" 'this subroutine is outside of the dll

sub testit alias "testit"
	print "in testit"

	callback 'call the external routine that is in the main program or another dll
end sub

If you compile that into a dll, then here's a main program that uses it:

Code: Select all

declare sub testit alias "testit" 'this routine is in the dll
declare sub callback alias "callback" 'this is my routine locally

sub callback export
	print "callback!"
end sub

testit 'call the dll routine

Using "alias" helps make sure that the subroutine names are always consistent. FB normally mangles the name, so if you wanted to call it from C, you'd have a hard time. alias lets me "canonize" the name as simple, unadorned lowercase name, which other languages are happy with.

[TJF]

To muddy the waters even more. What about variables/arrays that are declared as:

Static Shared as Integer MaximumEntities
...

Here's an example including both variables and function calls between main and dll. Declare COMMON variables in both code (finally included from a header file), like DLL code named wallyg_dll.bas

Code: Select all

TYPE UDT
  AS LONG dummy
END TYPE

COMMON SHARED AS INTEGER MaximumEntities
COMMON SHARED AS UDT PTR LocationOfABigTable
COMMON SHARED AS UDT TheWorldAsISeeIt()

COMMON SHARED main_func AS FUNCTION() AS STRING

'' the stuff above goes in a .bi header
'' compile by fbc -dll wallyg_dll.bas

SUB vars_in_dll ALIAS "vars_in_dll"() EXPORT
  ?__FUNCTION__
  ?"  ";MaximumEntities
  ?"  ";LocationOfABigTable
  ?"  ";TheWorldAsISeeIt(50, 500).dummy
END SUB

SUB main_sub_from_dll ALIAS "main_sub_from_dll"() EXPORT
  ?__FUNCTION__
  ?"  ";main_func()
END SUB

and executable code

Code: Select all

TYPE UDT
  AS LONG dummy
END TYPE

COMMON SHARED AS INTEGER MaximumEntities
COMMON SHARED AS UDT PTR LocationOfABigTable
COMMON SHARED AS UDT TheWorldAsISeeIt()

COMMON SHARED main_func AS FUNCTION() AS STRING

'' the stuff above goes in a .bi header
'' compile by fbc -export wallyg_main.bas

FUNCTION main_helper_func() AS STRING
  RETURN "The main_func output"
END FUNCTION

' initialize values for testing
main_func = @main_helper_func()
MaximumEntities = 111
LocationOfABigTable = NEW UDT
REDIM TheWorldAsISeeIt(1 TO 100,1 TO 1000)
TheWorldAsISeeIt(50, 500).dummy = 555

' test from main module
?__FUNCTION__
?"  ";main_func()
?__FUNCTION__
?"  ";TheWorldAsISeeIt(50, 500).dummy

?"** test from dylib:"
VAR dll = DYLIBLOAD("wallyg_dll")
IF dll THEN
  DIM vars_in_dll AS SUB()
  vars_in_dll = DYLIBSYMBOL(dll, "vars_in_dll")
  IF vars_in_dll _
    THEN vars_in_dll() _
    ELSE ?"symbol 'vars_in_dll' not found"

  DIM main_sub_from_dll AS SUB()
  main_sub_from_dll = DYLIBSYMBOL(dll, "main_sub_from_dll")
  IF main_sub_from_dll _
    THEN main_sub_from_dll() _
    ELSE ?"symbol 'main_sub_from_dll' not found"

  DYLIBFREE(dll)
END IF

DELETE(LocationOfABigTable)

In contrast to the caseih solution here I use a function pointer to make main_func() available in the DLL.

Note: The main code needs to get compiled with option -export.

[dodicat]

I cannot get TJF or caseih's code to work.
I get a crash with TJF and a non compile with caseih.
Win 10, tried in 32 bit fbc.
This works (everything a byref function)
The dll, passthings.dll

Code: Select all

'passthings.bas  compile -dll
Type udt
    As Long x
    As zstring * 90 bigtable(1 To 5)
End Type

Static As udt u
u.x=Cint(@u)


For n As Long=1 To 5
    If n=1 Then Print "Initialising big table"
    u.bigtable(n)="BigTable entry "+Str(n)+" = "+Str(Rnd*1000)
Next n

Function MaximumEntities() Byref As Integer Export
    Static As Integer i
    Return i
End Function

Function LocationOfABigTable() Byref As udt Ptr Export
    Static As udt Ptr i
    Return i
End Function

Function TheWorldAsISeeIt(n As Integer,m As Integer) Byref As udt Export
    Static As udt i(1 To 100,1 To 100)
    Return i(n,m)
End Function

function outside() byref as function() as long Export
    static as function() as long f
    return f
End function


For n As Long=1 To 100
    For m As Long=1 To 100
        If n=1 And m=1then Print "Initialising TheWorldAsISeeIt"
        TheWorldAsISeeIt(n,m)=Type(n+m)
    Next
Next

LocationOfABigTable()=@u
MaximumEntities()=78

'======================================================
 

Test code using dylibload (#inclib would be much cleaner)

Code: Select all

Type udt
    As Long x
    As zstring * 90 bigtable(1 To 20)
End Type

Extern fn Alias "dothis"As Function() As Long

Function dothis() As Long
    Print __function__
    Return 2021
End Function

Function dothat() As Long
    Print __function__
    Return -2021
End Function

Var L=Dylibload("passthings")
Dim MaximumEntities As Function() Byref As Integer
Dim LocationOfABigTable As Function() Byref As udt Ptr 
Dim TheWorldAsISeeIt As Function( As Integer,As Integer) Byref As udt 
Dim outside As Function() Byref As Function() As Long
If L Then
    MaximumEntities=Dylibsymbol(L,"MAXIMUMENTITIES")
    If MaximumEntities=0 Then Print "MaximumEntities not found":Sleep:End
    LocationOfABigTable=Dylibsymbol(L,"LOCATIONOFABIGTABLE")
    If LocationOfABigTable=0 Then  Print "LocationOfABigTable not found":Sleep:End
    TheWorldAsISeeIt=Dylibsymbol(L,"THEWORLDASISEEIT")
    If TheWorldAsISeeIt=0 Then Print "TheWorldAsISeeIt not found":Sleep:End
    outside=Dylibsymbol(L,"OUTSIDE")
    If outside=0 Then Print "outside not found":Sleep:End
Else
    Print "dll not found"
End If

Print "-----------------------------------"



Print  " MaximumEntities = ";MaximumEntities()
Print " LocationOfABigTable =  ";LocationOfABigTable()->x

Var v= Cast(udt Ptr,LocationOfABigTable()->x)

For n As Long=1 To 5
    Print v->bigtable(n)
Next
Print

Print "TheWorldAsISeeIt(6,8).x = ";TheWorldAsISeeIt(6,8).x

For n As Long=1 To 5
    LocationOfABigTable()->bigtable(n)+= " Altered in main prog"
Next
Print

For n As Long=1 To 5
    Print v->bigtable(n)
Next

outside()=@dothis
Print outside()()


outside()=@dothat
Print outside()()


Sleep
Dylibfree(L)
 

[fxm]

From FXM post, Aug 12, 2020 (see viewtopic.php?p=275000#p275000):

@users,

Would this type of information deserve to be included in the documentation (the missing part) (Common, Extern, Import, Static Libraries, Shared Libraries (DLLs))?

General remark about sharing variables with a library:

• A static library (compiled with -lib) allows the direct sharing of variables by using the COMMON [SHARED] keyword, both in library code and module code.
• A dynamic library (compiled with -dll or -dylib) does not support the COMMON keyword.
• Otherwise, for any library type (including a dynamic library), passing a parameter (by value or by reference) to a library procedure or returning a variable (by value or by reference) from a library function allows to indirectly exchange data (by value) or share data (by reference) with a shared library.

EXTERN was added in order to support the C libraries.
EXTERN offers a similar function, but while COMMON declares the variable in each module and reserves memory, EXTERN only declares variables and does not define them.
Like COMMON, EXTERN cannot be used in a FreeBASIC code module to be compiled into a dynamic library (this only works for a static library or another external module).

EXTERN IMPORT must be only used in external modules to access global variables from Win32 DLLs: the variable names will be added to the dynamic library import list so that their addresses can be fixed at run-time.

[edit]
- Added also:

A static or dynamic library can optionally have module constructors, a main code, and module destructors. The module constructors, then the main code are executed at library load. The module destructors are executed at library unload.

Done:
- KeyPgCommon → fxm [wording]
- KeyPgExtern → fxm [added information on its use]
- KeyPgImport → fxm [added information on its use]
- ProPgStaticLibraries → fxm [added information on sharing variables with static library]
- ProPgSharedLibraries → fxm [added information on sharing variables with shared library]

[TJF]

I cannot get TJF or caseih's code to work.
I get a crash with TJF ...

What kind of crash and where?

Here it compiles and runs under

Code: Select all

FreeBASIC Compiler - Version 1.07.1 (2019-09-27), built for linux-x86_64 (64bit)

Try option -gen gcc

[TJF]

(#inclib would be much cleaner)

#INCLIB works at compile-time. wallyg is asking for a run-time solution.

General remark about sharing variables with a library:

...
A dynamic library (compiled with -dll or -dylib) does not support the COMMON keyword.
...

That's not correct. At least under 64 bit LINUX the keyword COMMON also works for dynamic loaded libs (by DYLIBLOAD). (Did you compile with option -export?)

[caseih]

In contrast to the caseih solution here I use a function pointer to make main_func() available in the DLL.

Definitely a function pointer is the most portable and preferred way to do it. In practice I don't know of very many DLLs that call directly like my example did.

I get a crash with TJF and a non compile with caseih.

I compiled the dll on 64-bit linux with:

Code: Select all

fbc -dll testdll.bas

Then I compiled the main file with:

Code: Select all

fbc mainfile.bas -l testdll

That was it. To run it I had to set LD_LIBRARY_PATH of course:

Code: Select all

LD_LIBRARY_PATH=. ./main

[dodicat]

Thanks caseih, I'll have another try.
wallyg
If you keep my passthings.bas from previously in the same folder as this;

Code: Select all

#include "file.bi"
Type udt
    As Long x
    As zstring * 90 bigtable(1 To 20)
End Type

print exec("C:\stubs\fbc.exe ",curdir+"\passthings.bas"+" -dll ")

sleep 500
print fileexists("passthings.dll")

Function dothis() As Long
    Print __function__
    Return 2021
End Function

Function dothat() As Long
    Print __function__
    Return -2021
End Function

Var L=Dylibload("passthings")
Dim MaximumEntities As Function() Byref As Integer
Dim LocationOfABigTable As Function() Byref As udt Ptr 
Dim TheWorldAsISeeIt As Function( As Integer,As Integer) Byref As udt 
Dim outside As Function() Byref As Function() As Long
If L Then
    MaximumEntities=Dylibsymbol(L,"MAXIMUMENTITIES")
    If MaximumEntities=0 Then Print "MaximumEntities not found":Sleep:End
    LocationOfABigTable=Dylibsymbol(L,"LOCATIONOFABIGTABLE")
    If LocationOfABigTable=0 Then  Print "LocationOfABigTable not found":Sleep:End
    TheWorldAsISeeIt=Dylibsymbol(L,"THEWORLDASISEEIT")
    If TheWorldAsISeeIt=0 Then Print "TheWorldAsISeeIt not found":Sleep:End
    outside=Dylibsymbol(L,"OUTSIDE")
    If outside=0 Then Print "outside not found":Sleep:End
Else
    Print "dll not found":sleep:end
End If

Print "-----------------------------------"



Print  " MaximumEntities = ";MaximumEntities()
Print " LocationOfABigTable =  ";LocationOfABigTable()->x

Var v= Cast(udt Ptr,@LocationOfABigTable()->x)

For n As Long=1 To 5
    Print v->bigtable(n)
Next
Print

Print "TheWorldAsISeeIt(6,8).x = ";TheWorldAsISeeIt(6,8).x

For n As Long=1 To 5
    LocationOfABigTable()->bigtable(n)+= " Altered in main prog"
Next
Print

For n As Long=1 To 5
    Print v->bigtable(n)
Next

outside()=@dothis
Print outside()()


outside()=@dothat
Print outside()()


Sleep
Dylibfree(L)
kill "passthings.dll"
 

Note that you don't need the dll in situ, the code should create it via exec (as you wanted)
The path to my 64 bit fbc is "C:\stubs\fbc.exe ".
I am 99.9999999% sure you will have a different path, so you must use your path.
It works fine here with 64 bit version 1.08.1.

The dll is killed at the end.

[fxm]

General remark about sharing variables with a library:

...
A dynamic library (compiled with -dll or -dylib) does not support the COMMON keyword.
...

That's not correct. At least under 64 bit LINUX the keyword COMMON also works for dynamic loaded libs (by DYLIBLOAD). (Did you compile with option -export?)

I just tested your above code under Windows (gas 32-bit, gcc 9.3.0 32-bit, gcc 9.3.0 64-bit, gas64 64-bit, all with '-export' compile option):
For all, this compiles, but the execution fails:

__FB_MAINPROC__
The main_func output
__FB_MAINPROC__
555
** test from dylib:
VARS_IN_DLL
0
0

Aborting due to runtime error 6 (out of bounds array access) at line 18 of C:\Users\fxmam\OneDrive\Documents\Mes Outils Personnels\FBIde0.4.6r4-FreeBASIC1.09.0.win64\wallyg_dll.bas::VARS_IN_DLL()

[TJF]

I just tested your above code under Windows (gas 32-bit, gcc 9.3.0 32-bit, gcc 9.3.0 64-bit, gas64 64-bit, all with '-export' compile option):
For all, this compiles, but the execution fails:

Code: Select all

__FB_MAINPROC__
  The main_func output
__FB_MAINPROC__
   555
** test from dylib:
VARS_IN_DLL
   0
  0

Aborting due to runtime error 6 (out of bounds array access) at line 18 of C:\Users\fxmam\OneDrive\Documents\Mes Outils Personnels\FBIde0.4.6r4-FreeBASIC1.09.0.win64\wallyg_dll.bas::VARS_IN_DLL()

The desired output is (pointer 20845640 between 111 and 555 lines may vary)

Code: Select all

__FB_MAINPROC__
  The main_func output
__FB_MAINPROC__
   555
** test from dylib:
VARS_IN_DLL
   111
  20845640
   555
MAIN_SUB_FROM_DLL
  The main_func output

which here I get from

FreeBASIC Compiler - Version 1.07.1 (2019-09-27), built for linux-x86_64 (64bit)

FreeBASIC Compiler - Version 1.01.0 (01-12-2015), built for linux-arm (32bit)

and (option -gen gas or -gen gcc)

FreeBASIC Compiler - Version 1.07.0 (05-20-2019), built for linux-x86 (32bit)

Obviously there's a problem in the wodniws compiler. (Can anybody test under DOS?)

[dodicat]

I have tried various manoeuvres with your code TJF, but the common shared does not seem to work with windows dll's (as per help file)
I get the same as fxm, 0,0, then stall with the array.
I tried 1.07.1 and back to .24.0 also.
Sorry I have no DOS (or Linux) just now.

[TJF]

I have tried various manoeuvres with your code TJF, but the common shared does not seem to work with windows dll's (as per help file)

As I wrote, COMMON works for DYLIBLOAD since years on LINUX. Your issue seems to be related to the run-time linker used for DYLIBLOAD; it binds functions, but obviously not COMMON variables on your OS.

What about the EXTERN IMPORT alternative, does this work for wodniws?

Code: Select all

TYPE UDT
  AS LONG dummy
END TYPE

#IF __FB_OUT_DLL__
 #DEFINE DLL_TRANSFER EXTERN IMPORT
#ELSE
 #DEFINE DLL_TRANSFER COMMON SHARED
#ENDIF

DLL_TRANSFER AS INTEGER MaximumEntities
DLL_TRANSFER AS UDT PTR LocationOfABigTable
DLL_TRANSFER AS UDT TheWorldAsISeeIt()

DLL_TRANSFER main_func AS FUNCTION() AS STRING

'' the stuff above goes in a .bi header
'' compile by fbc -dll wallyg_dll.bas

SUB vars_in_dll ALIAS "vars_in_dll"() EXPORT
  ?__FUNCTION__
  ?"  ";MaximumEntities
  ?"  ";LocationOfABigTable
  ?"  ";TheWorldAsISeeIt(50, 500).dummy
END SUB

SUB main_sub_from_dll ALIAS "main_sub_from_dll"() EXPORT
  ?__FUNCTION__
  ?"  ";main_func()
END SUB

[TJF]

What about the EXTERN IMPORT alternative, does this work for wodniws?

I found an old wine fbc

Code: Select all

FreeBASIC Compiler - Version 1.01.0 (12-28-2014), built for win32 (32bit)

and tried the EXTERN IMPORT solution (which is working on LINUX). It doesn't compile, thoughing errors like

Code: Select all

wallyg_dll.o:fake:(.text+0x61): undefined reference to `MAXIMUMENTITIES`

Note: A declaration (EXTERN IMPORT) rises an 'undefined reference' error?!


@Wally

You could work arround that compiler bug by packing all COMMON variables in to an UDT (named DLLclass here) and transfer a pointer to that UDT in an dll_init function, like wallyg_dll.bas

Code: Select all

TYPE UDT
  AS LONG dummy
END TYPE

TYPE STRIFU AS FUNCTION() AS STRING

TYPE DLLclass
  AS INTEGER MaximumEntities
  AS UDT PTR LocationOfABigTable
  AS UDT TheWorldAsISeeIt(1 TO 100,1 TO 1000)
  AS STRIFU main_func
END TYPE

COMMON SHARED AS DLLclass PTR DAT

'' the stuff above goes in a .bi header
'' compile by fbc -dll wallyg_dll.bas

SUB dll_init ALIAS "dll_init"(BYVAL P AS DLLclass PTR) EXPORT
  DAT = P
END SUB

SUB vars_in_dll ALIAS "vars_in_dll"() EXPORT
WITH *DAT
  ?__FUNCTION__
  ?"  ";.MaximumEntities
  ?"  ";.LocationOfABigTable
  ?"  ";.TheWorldAsISeeIt(50, 500).dummy
END WITH
END SUB

SUB main_sub_from_dll ALIAS "main_sub_from_dll"() EXPORT
WITH *DAT
  ?__FUNCTION__
  ?"  ";.main_func()
END WITH
END SUB

and wallyg_main.bas

Code: Select all

TYPE UDT
  AS LONG dummy
END TYPE

TYPE STRIFU AS FUNCTION() AS STRING

TYPE DLLclass
  AS INTEGER MaximumEntities
  AS UDT PTR LocationOfABigTable
  AS UDT TheWorldAsISeeIt(1 TO 100,1 TO 1000)
  AS STRIFU main_func
END TYPE

COMMON SHARED AS DLLclass PTR DAT

'' the stuff above goes in a .bi header
'' compile by fbc wallyg_main.bas

FUNCTION main_helper_func() AS STRING
  RETURN "The main_func output"
END FUNCTION

' initialize values for testing
DAT = NEW DLLclass
WITH *DAT
.MaximumEntities = 111
.LocationOfABigTable = NEW UDT
.TheWorldAsISeeIt(50, 500).dummy = 555
.main_func = @main_helper_func()

' test from main module
?__FUNCTION__
?"  ";.main_func()
?__FUNCTION__
?"  ";.TheWorldAsISeeIt(50, 500).dummy

?"** test from dylib:"
VAR dll = DYLIBLOAD("wallyg_dll")
IF dll THEN
  DIM dll_init AS SUB(BYVAL AS DLLclass PTR)
  dll_init = DYLIBSYMBOL(dll, "dll_init")
  IF dll_init _
    THEN dll_init(DAT) _
    ELSE ?"symbol 'dll_init' not found"

  DIM vars_in_dll AS SUB()
  vars_in_dll = DYLIBSYMBOL(dll, "vars_in_dll")
  IF vars_in_dll _
    THEN vars_in_dll() _
    ELSE ?"symbol 'vars_in_dll' not found"

  DIM main_sub_from_dll AS SUB()
  main_sub_from_dll = DYLIBSYMBOL(dll, "main_sub_from_dll")
  IF main_sub_from_dll _
    THEN main_sub_from_dll() _
    ELSE ?"symbol 'main_sub_from_dll' not found"

  DYLIBFREE(dll)
END IF

DELETE(.LocationOfABigTable)
END WITH
DELETE(DAT)

Here this works on both OSs.

[dodicat]

TJF
Works OK here, 32/64 bits
static will also do
static SHARED AS DLLclass PTR DAT
and
dim AS DLLclass PTR DAT
in the main program.
Which gets us away from common shared which is not really a windaes dll thing, as has been tested (and re-tested)

Looks as though arrays and variables must be wrapped up in udt's or byref functions or some other way to get access via .dlls in Win.

[TJF]

Which gets us away from common shared which is not really a windaes dll thing, as has been tested (and re-tested)

Perhaps the compiler bug gets fixed sometimes ...

Looks as though arrays and variables must be wrapped up in udt's or byref functions or some other way to get access via .dlls in Win.

BYREF gets complicated when you need bi-directional access (ie. DLL code should change array TheWorldAsISeeIt). And it's slow due to the function calling overhead.

The dll_init() function has a further advantage: you can get ride of some ALIAS/DYLIBSYMBOL magic by re-passing the function pointers from the DLL, like wallyg_dll.bas:

Code: Select all

TYPE UDT
  AS LONG dummy
END TYPE

TYPE DLLclass
  AS INTEGER MaximumEntities
  AS UDT PTR LocationOfABigTable
  AS UDT TheWorldAsISeeIt(1 TO 100,1 TO 1000)
  main_func AS FUNCTION() AS STRING

  vars_in_dll AS SUB()
  main_sub_from_dll AS SUB()
END TYPE

COMMON SHARED AS DLLclass PTR DAT

'' the stuff above goes in a .bi header
'' compile by fbc -dll wallyg_dll.bas

SUB vars_in_dll()
WITH *DAT
  ?__FUNCTION__
  ?"  ";.MaximumEntities
  ?"  ";.LocationOfABigTable
  ?"  ";.TheWorldAsISeeIt(50, 500).dummy
END WITH
END SUB

SUB main_sub_from_dll()
WITH *DAT
  ?__FUNCTION__
  ?"  ";.main_func()
END WITH
END SUB

SUB dll_init ALIAS "dll_init"(BYVAL P AS DLLclass PTR) EXPORT
  DAT = P
WITH *DAT
  .vars_in_dll = @vars_in_dll()
  .main_sub_from_dll = @main_sub_from_dll()
END WITH
END SUB

and wallyg_main.bas

Code: Select all

TYPE UDT
  AS LONG dummy
END TYPE

TYPE DLLclass
  AS INTEGER MaximumEntities
  AS UDT PTR LocationOfABigTable
  AS UDT TheWorldAsISeeIt(1 TO 100,1 TO 1000)
  main_func AS FUNCTION() AS STRING

  vars_in_dll AS SUB()
  main_sub_from_dll AS SUB()
END TYPE

COMMON SHARED AS DLLclass PTR DAT

'' the stuff above goes in a .bi header
'' compile by fbc -export wallyg_main.bas

FUNCTION main_helper_func() AS STRING EXPORT
  RETURN "The main_func output"
END FUNCTION

' initialize values for testing
DAT = NEW DLLclass
WITH *DAT
.MaximumEntities = 111
.LocationOfABigTable = NEW UDT
.TheWorldAsISeeIt(50, 500).dummy = 555
.main_func = @main_helper_func()

' test from main module
?__FUNCTION__
?"  ";.main_func()
?__FUNCTION__
?"  ";.TheWorldAsISeeIt(50, 500).dummy

?"** test from dylib:"
VAR dll = DYLIBLOAD("wallyg_dll")
IF dll THEN
  DIM dll_init AS SUB(BYVAL AS DLLclass PTR)
  dll_init = DYLIBSYMBOL(dll, "dll_init")
  IF dll_init _
    THEN dll_init(DAT) _
    ELSE ?"symbol 'dll_init' not found"

  .vars_in_dll()
  .main_sub_from_dll()

  DYLIBFREE(dll)
END IF

DELETE(.LocationOfABigTable)
END WITH
DELETE(DAT)