variadic functions and argument lists in fbc

[coderJeff]

fbc has a few macros for working with variadic functions: va_first, va_next(), va_arg(), that worked using a pointer to the argument stack. These macros don't work on all targets (like 64-bit) because arguments can be passed to procedures in cpu registers.

sf.net bug https://sourceforge.net/p/fbc/bugs/881/

Working on a replacement though, it's not quite ready to merge in to master:
github https://github.com/freebasic/fbc/pull/115

In the update, to work with variadic functions and argument lists, here's what we've come up with:

cva_list is added as a default type by fbc to expose the argument list. It works kind of like an object, in that usually we don't need to know what's doing, only how to use it. There are a few methods in it's API for accessing a variable length list of arguments passed in to a procedure..

Code: Select all

dim args as cva_list = any
cva_start( args, param )  '' constructor
cva_copy( dest, args )    '' copy constructor
cva_end( args )           '' destructor
result = cva_arg( args, datatype ) '' method returns data and advances to next argument

Note the constructor/destructor pairs:
- cva_start() needs exactly one matching cva_end()
- cva_copy() needs exactly one matching cva_end()

If it looks familiar, it is because it's just about the same in C. We could have come up with our own api, but then it wouldn't be compatible (link-able) with anything but our own code.

If you are currently using any of the new names being added: cva_list, cva_start, cva_end, cva_copy, cva_arg and depending on target __va_list_tag, __va_list, you will need to use #undef to get it out of your way, or rename the symbol in your code.

- No documentation written yet. This post is the first info.
- No guarantees that it is bug free. Please submit bug reports.
- Need help testing on ARM targets. (Or I have to create a VM for it)

Examples:

Iterate through a list of arguments

Code: Select all

sub proc cdecl( n as integer, ... )
	dim args as cva_list
	cva_start( args, n )
	for i as integer = 1 to n
		print "arg " & i & " = " & cva_arg( args, integer )
	next i
	cva_end( args )
end sub

proc( 3, 11, 22, 33 )

Pass argument list to another procedure

Code: Select all

sub proc2 cdecl( n as integer, byval args as cva_list )
	dim x as cva_list = any
	cva_copy( x, args )
	for i as integer = 1 to n
		print "arg " & i & " = " & cva_arg( x, integer )
	next i
	cva_end( x )
end sub

sub proc1 cdecl( n as integer, ... )
	dim x as cva_list = any
	cva_start( x, n )
	proc2( n, x )
	cva_end( x )
end sub

proc1( 3, 11, 22, 33 )

----

Internal stuff:

The implementation and code emitted for cva_list, cva_start, cva_copy, cva_end, cva_arg varies depending on target, backend, and architecture.

In -gen gas backend, these macros translate to expressions because we are providing our own implementation. With the cva_* macros being similar to the implementation of the fbc's existing va_* macros but having different behaviours.

In -gen gcc backend, the cva_* macros will map to gcc's builtin macros:
cva_list => __builtin_va_list
cva_start => __builtin_va_start
cva_arg => __builtin_va_arg
cva_end => __builtin_va_end
cva_copy => __builtin_va_copy

The cva_list data type is a hairy one, because it's type varies depending on the target:

Code: Select all

'' dos/win/linux x86 - gas backend
type cva_list as any alias "char" ptr

'' dos/win/linux/arm x86 - gcc backend
type cva_list as any alias "__builtin_va_list" ptr

'' win32 x86_64 - gcc backend
type cva_list as any alias "__builtin_va_list" ptr

'' linux x86_64 - gcc backend
type __va_list_tag alias "__va_list_tag"
	as ulong gp_offset
	as ulong fp_offset
	as any ptr overflow_arg_area
	as any ptr reg_save_area
end type  
type cva_list as __va_list_tag alias "__builtin_va_list"

'' arm64/aarch64 - gcc backend
type __va_list alias "__va_list"
	as any ptr __stack
	as any ptr __gr_top
	as any ptr __vr_top
	as long __gr_offs
	as long __vr_offs
end type
type cva_list as __va_list alias "__builtin_va_list"

The intent is that cva_list and cva_* macros will work on all the targets, though I have only tested on x86 & x86_64 for win/lin. I don't have a set-up for testing LLVM backend, or ARM targets.

Also, you should notice some new syntax with the AS datatype ALIAS "modifier". This is valid syntax. A side note here: a while ago added the following syntax: byval as long alias "long". This was specifically added to allow fbc programs to link with win32's 32 bit long int in gcc c++, which was previously impossible without this type modifier

What the ALIAS "modifier" allows, is using a datatype as you would expect in fbc, but then, in the backend emitter, do something special with it; mostly used for name mangling and linking. fbc has to do some other tricky stuff to be compatible across multiple targets because __builtin_va_list is typed differently on various targets.

The documenation for usage of ALIAS "modifier" should get its own new page, maybe KeyPgAliasTypeModifier

[coderJeff]

This update will deal with C like variadic paramaters only.

In this old thread:
viewtopic.php?p=242057#p242057

I would separate Basic level variadic parameters from C level variadic parameters, and treat that as separate constructs. And maybe not allow defining C level variadic functions in FB. (only FB level variadic parameters)

So, having a more "basic" level of variadic parameters is still possible. Maybe using a func(name...) like syntax or similar, that would allow inspecting the argument list for number of arguments, types, using other complex types like STRING or ARRAY(), etc. But it's not in this update.

For reference, other threads talking about variadic functions:
viewtopic.php?p=241348#p241348
viewtopic.php?f=2&t=22951
viewtopic.php?p=244495#p244495
viewtopic.php?t=21668

There's probably other threads as well.

[coderJeff]

varargs feature now merged into fbc 1.07.0 (for development info, see pull request #115)

New wiki pages:
cva_list | cva_arg | cva_start | cva_copy | cva_end

Also Updated:
Variable Arguments | Functional Keyword List | Alphabetical Keyword List | Procedures
... (Ellipsis) | va_first | va_next | va_arg

UPDATE:
New wiki page for ALIAS (Modifier) syntax, WIP.

Found first bug while writing the doc (dang it!).
dim x as any alias "char" ptr 'OK, is valid syntax
dim x as any alias "char" ' BUG, no error but should be "invalid type" error

[fxm]

Quickly a build from St_W so that we can try this!

[fxm]

@coderJeff

From documentation:

.....
Cva_Start is like a constructor for the variadic argument_list object and must eventually have a matching call to Cva_End, which is like a destructor.
.....

Does this mean that if not calling Cva_End, the "destructor" will be called automatically when the Cva_List variable goes out of scope (as for a classic Dim)?

[fxm]

@coderJeff

Can we use dynamic Cva_List variables (with New/Delete) as following? :

Code: Select all

Sub proc cdecl(byval count As Integer, ... )
    Dim pargs As Cva_List ptr = New Cva_List

    Cva_Start( *pargs, count )

    For i As Integer = 1 To count
        Print Cva_Arg( *pargs, Integer )
    Next
   
    Delete pargs
End Sub

proc( 4, 4000, 300, 20, 1 )

This works, but with warnings when using gcc 32/64-bit instead of gas 32-bit:

• Compiler output:
  C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.bas(11) warning 4(1): Suspicious pointer assignment
  C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.bas(11) warning 4(1): Suspicious pointer assignment
  C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.c: In function 'PROC':
  C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.c:39:14: warning: comparison of distinct pointer types lacks a cast
  if( TMP$3$1 == (void**)0u ) goto label$5;
  ^
  C:\Users\fxmam\Documents\Mes Outils Personnels\FBIde0.4.6r4_fbc1.07.0\FBIDETEMP.c:71:14: warning: comparison of distinct pointer types lacks a cast
  if( PARGS$1 == (void**)0u ) goto label$10;

[coderJeff]

.....
Cva_Start is like a constructor for the variadic argument_list object and must eventually have a matching call to Cva_End, which is like a destructor.
.....

Does this mean that if not calling Cva_End, the "destructor" will be called automatically when the Cva_List variable goes out of scope (as for a classic Dim)?

cva_end is never called automatically. cva_start & cva_end must always be used in pairs. I'm open to suggestions for better terminology.

Code: Select all

scope
	dim args as cva_list    '' allocation
	cva_start( args, count) '' initialization, construction, create
	cva_end( args )         '' de-initialization, destruction, destroy, clean-up, tear-down
end scope                  '' deallocation

[coderJeff]

Can we use dynamic Cva_List variables (with New/Delete)? :

Yes, it's supposed to work with pointer types. Thank-you for testing.

For example:

Code: Select all

Sub proc cdecl(byval count As Integer, ... )
    Dim pargs As Cva_List ptr = allocate( sizeof(Cva_List) )

    Cva_Start( *pargs, count )

    For i As Integer = 1 To count
        Print Cva_Arg( *pargs, Integer )
    Next
   
    if( cast( any ptr, pargs) ) then
		deallocate( pargs )
	end if
End Sub

proc( 4, 4000, 300, 20, 1 )

The issues (bugs) appear to be with null pointer checks, with comparisons (IF statements) or generated (-exx compiler option) or internal pointer checks (NEW/DELETE). Here's a variation of your original test case for new/delete that highlights each case:

Code: Select all

Sub proc cdecl(byval count As Integer, ... )

	'' BUG: warning: comparison of distinct pointer types lacks a cast
	'' due to ptr check in NEW
    Dim pargs As Cva_List ptr = New Cva_List

	'' BUG: suspicious pointer assignment 
	'' due to null pointer check with '-exx'
    Cva_Start( *pargs, count )

    For i As Integer = 1 To count
    	'' BUG: suspicious pointer assignment
    	'' due to null pointer check with '-exx'
        Print Cva_Arg( *pargs, Integer )
    Next

	'' BUG: warning: comparison of distinct pointer types lacks a cast
	'' due to comparison of cva_list ptr with NULL
	if( pargs ) then
		print "OK"
	end if
	
	'' BUG: warning: comparison of distinct pointer types lacks a cast
	'' due to ptr check in DELETE
	Delete pargs

End Sub

proc( 4, 4000, 300, 20, 1 )

I wrote a test for new/delete in tests/functions/va_cva_api.bas, it passes, but obviously, I missed the output being generated. Thanks.

[MrSwiss]

Took the example from Documentation, and run it OK (see comments in code):

Code: Select all

'
'   tested with St_W's build: DEV x64 WIN standalone --> OK!
'   whole crt dir, with all files, copied from FBC 1.06.0 x64 WIN
'
'' pass the args list to a function taking an cva_list type argument
#include "crt/stdio.bi"

sub myprintf cdecl(fmt as zstring ptr, ...)
    Dim As cva_list args
    cva_start(args, fmt)
    vprintf(fmt, args)
    cva_end(args)
end Sub

dim as string s = "bar"

myprintf(!"integer = %i, longint = %lli, float = %f\n", _
	1, 1ll shl 32, 3.3)
myprintf(!"string = %s, string = %s\n", "foo", s)

Sleep

[fxm]

.....
Cva_Start is like a constructor for the variadic argument_list object and must eventually have a matching call to Cva_End, which is like a destructor.
.....

Does this mean that if not calling Cva_End, the "destructor" will be called automatically when the Cva_List variable goes out of scope (as for a classic Dim)?

cva_end is never called automatically. cva_start & cva_end must always be used in pairs. I'm open to suggestions for better terminology.

I did not pay enough attention because "eventually" is for me French a "false friend" of "éventuellement" (possibly).

[coderJeff]

When counting_pine, dkl, and I were discussing the design of the C va_list type, there were 2 ways we could go:
1) an abstract type, like what we are using now, where fbc knows some things about va_list usage, but not the full implementation in the backend.
2) a completely new and separate type (like STRING or BOOLEAN), which would require a full implementation for every platform.

I think doing the abstract type first was relatively easier and does not prevent a full type later if that is desired. The intent is that it solves all the most common use cases, even though, we may find fbc generating some code that the backend compiler might reject.

----

I almost forgot about the feature that counting_pine had worked on leading up to his initial contribution to the cva_list type:

#DUMP(expr) and #ODUMP(expr) preprocessor statements for debugging expressions. The exact format of the AST output might vary depending on if you are running a release version or a debug version of fbc. And only works for expressions, not statements.

Example:

Code: Select all

dim as integer x

#print NATURAL EXPRESSION
#DUMP( 2 * x * 4 * 5 )
 
#print OPTIMIZED EXPRESSION
#ODUMP( 2 * x * 4 * 5 )

Output:

Code: Select all

NATURAL EXPRESSION
0                      * =-=  [integer]
                                     / \
1                    * =-=  [integer]
                                   / \
2                  * =-=  [integer]
                                 / \
3    VAR( X ) (integer) [integer]

3                                   2 (integer) [integer]

2                                     4 (integer) [integer]

1                                       5 (integer) [integer]

OPTIMIZED EXPRESSION
0                      * =-=  [integer]
                                     / \
1        VAR( X ) (integer) [integer]

1                                       40 (integer) [integer]

[coderJeff]

The issues (bugs) appear to be with null pointer checks, with comparisons (IF statements) or generated (-exx compiler option) or internal pointer checks (NEW/DELETE). Here's a variation of your original test case for new/delete that highlights each case:

I created a pull request for the following:

- disable suspicious warnings on NULL pointer checks for pointers to cva_list types
- disable -gen gcc warning of comparison of distinct pointer types lacks a cast when comparing cva_list pointer types
- disallow ANY ALIAS "modifier" unless it is also a pointer type, like ANY PTR. Allocated data types must have non-zero size.

[TJF]

Hi all!

Unfortunately this implementation is not sufficient. CVA_ARG returns the parameter values (a *CPTR(dtype PTR, expr2)). But more often we need the pointer itself (= returning expr2). This is necessary to call functions like

Code: Select all

void
g_object_set_valist (GObject *object,
                     const gchar *first_property_name,
                     va_list var_args);

or

Code: Select all

void
gtk_style_get_valist (GtkStyle *style,
                      GType widget_type,
                      const gchar *first_property_name,
                      va_list var_args);

where first_property_name is a pointer to the parameter name, and var_args is a pointer to further parameters.

So what we really need is a further keywort like CVA_PTR(args, type) that returns a pointer to the value, instead of the value itself. (CVA_ARG could be a macro using CVA_PTR).

Regards

[coderJeff]

"args" is the "pointer" to the list. Actually, on some platforms it is a pointer to the stack. On other platforms it is a structure that tracks the current argument number & value. i.e. where arguments are passed in registers and don't have an address exactly.

A variadic procedure takes a variable number of arguments and is indicated by last parameter ellipsis (...)

The cva_list type is a data type that represents that list.

I'm not familiar with the framework but maybe this might help illustrate:

Code: Select all

void _some_action (SomeKindOfObj *obj,
	const gchar *first_property,
	...)
{
	va_list args;
	va_start (args, first_property);
	g_object_set_valist (G_OBJECT (obj), first_property, args);
	va_end (args);
}

The ellipsis allows passing mutiple arguments to _some_action( obj, "PROP1', "VALUE, "PROP2", 123 ).
The va_list data type abstracts the variable argument list and allows it to be passed to g_object_set_valist ();

Do you have a freebasic example we can work with? Or a more complete C example to translate?

[TJF]

Hi coderJeff, thanks for the anwser!

Do you have a freebasic example we can work with?

Sure, ie. look at line 253. You can find similar code in each of the projects objects.

Code: Select all

The ellipsis allows passing mutiple arguments to _some_action( obj, "PROP1', "VALUE, "PROP2", 123 ).

Not entirely correct. In order to mark the end of the list, the va_list always needs a terminating NULL.

I'm not familiar with the framework but maybe this might help illustrate:

Unfortunately your code is unacceptable. I need 1:1 compatibility to the API of the underlying GooCanvas library. but ...

"args" is the "pointer" to the list.

This is the information I couldn't find in the docs. It helps partialy, but obviously that info is not correct. I was able to compile (and run) the following code on a 32-bit system using -gen gcc

Code: Select all

  'VAR va = VA_FIRST(), arg = VA_ARG(va, ZSTRING PTR)
  'IF arg THEN g_object_set_valist(G_OBJECT(polax), arg, VA_NEXT(va, ANY PTR))
  DIM AS CVA_LIST args : CVA_START(args, Text)
  VAR arg = CVA_ARG(args, gchar PTR)
  IF arg THEN g_object_set_valist(G_OBJECT(polax), arg, args)
  CVA_END(args)

But on a 64-bit system I get the error message

Code: Select all

error 58: Type mismatch, at parameter 3 of G_OBJECT_GET_VALIST()

The compiler doesn't accept args as va_list (previously defined as TYPE AS ANY PTR va_list).

Regards