Compiler tests

New to FreeBASIC? Post your questions here.
Juergen Kuehlwein
Posts: 284
Joined: Mar 07, 2018 13:59
Location: Germany

Compiler tests

Post by Juergen Kuehlwein »

I want to run the compiler test suite in the "\tests" folder of the source code package under Windows (Win 7 home 64 bit, to be specific) I downloaded GNU make and put the binaries (make.exe, lidiconv2.dll and libintl3.dll) into the \tests folder. Now i open a command prompt in this folder and run: "make -f cuint-tests.mk fail=1 basic=1 -d"

This is what i keep getting:
...
Finished prerequisites of target file `dirlist.mk'.
No need to remake target `dirlist.mk'.
Finished prerequisites of target file `cunit-tests.inc'.
Must remake target `cunit-tests.inc'.
Creating temporary batch file D:\Users\ADMINI~1\AppData\Local\Temp\make3276-1.ba
t
CreateProcess(D:\Users\ADMINI~1\AppData\Local\Temp\make3276-1.bat,D:\Users\ADMIN
I~1\AppData\Local\Temp\make3276-1.bat,...)
Putting child 0x0046f7f0 (cunit-tests.inc) PID 4730992 on the chain.
Live child 0x0046f7f0 (cunit-tests.inc) PID 4730992
Main thread handle = 0x00000098
Der Befehl "printf" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.
Reaping losing child 0x0046f7f0 PID 4730992
Cleaning up temp batch file D:\Users\ADMINI~1\AppData\Local\Temp\make3276-1.bat
make[1]: *** [cunit-tests.inc] Error 1
Removing child 0x0046f7f0 PID 4730992 from chain.
make[1]: Leaving directory `D:/Users/administrator/Desktop/FreeBASIC/FreeBASIC-1
.05.0-source/tests'
Reaping losing child 0x0046f730 PID 4698760
make: *** [cunit-tests] Error 2
Removing child 0x0046f730 PID 4698760 from chain.
Translation:
Der Befehl "printf" ist entweder falsch geschrieben oder
konnte nicht gefunden werden.
The command "printf" was misspelled or could not be found


What is wrong?


JK
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Compiler tests

Post by marcov »

Dat is duidelijk, printf kan niet gevonden worden of is fout geschreven.

translation: that is clear, printf cannot be found or is wrongly spelled.

Usually this means either the "C" runtime is not linked, or the symbol name mismatches. If it is the latter, since printf is standard and should be there, that probably means one side spells "printf" with a leading underscore as per COFF ABI, and one not.
Juergen Kuehlwein
Posts: 284
Joined: Mar 07, 2018 13:59
Location: Germany

Re: Compiler tests

Post by Juergen Kuehlwein »

marcov,

you as a Dutch shouldn´t have much of a problem translating the German sentence, maybe others might have had.

The "C" runtime is present and according to dependency walker nothing "mission critical" is missing. What to try next ?


Thanks


JK
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Compiler tests

Post by marcov »

Juergen Kuehlwein wrote:
The "C" runtime is present and according to dependency walker nothing "mission critical" is missing. What to try next ?
I provided a "latter" part of my answer. IOW printf doesn't match _printf.
Juergen Kuehlwein
Posts: 284
Joined: Mar 07, 2018 13:59
Location: Germany

Re: Compiler tests

Post by Juergen Kuehlwein »

I provided a "latter" part of my answer. IOW printf doesn't match _printf.
forgive me my ignorance - i have absolutely no experience with make or makefiles. Printf is a C function for formatted output. I really don´t get the point. Please explain more or tell me where to get useful info. I was "googling" around the last hours - to no avail.

JK
St_W
Posts: 1619
Joined: Feb 11, 2009 14:24
Location: Austria
Contact:

Re: Compiler tests

Post by St_W »

instead of setting up everything for a working environment yourself i recommend to use msys2 https://www.msys2.org/ or similar preconfigured environments.
lizard
Posts: 440
Joined: Oct 17, 2017 11:35
Location: Germany

Re: Compiler tests

Post by lizard »

Juergen Kuehlwein wrote: The command "printf" was misspelled or could not be found
What is wrong?
Can it be you don't have gcc on the path?
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Compiler tests

Post by marcov »

Juergen Kuehlwein wrote:
I provided a "latter" part of my answer. IOW printf doesn't match _printf.
forgive me my ignorance - i have absolutely no experience with make or makefiles.
Then maybe, avoid GCC on Windows? It is a bucket full of pain.

That said, passing verbosity options to gcc and looking over its output is often key in solving gcc's mysteries.
Printf is a C function for formatted output. I really don´t get the point. Please explain more or tell me where to get useful info. I was "googling" around the last hours - to no avail.
I avoid gcc on windows out of principle, but the bit is that a function in C (e.g. printf), sometimes gets an underscore (_) prefixed on the linker level.

If it happens depends on calling conventions and exact windows port (mingw, msys) and how they match the headers.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Compiler tests

Post by coderJeff »

Juergen Kuehlwein wrote: The command "printf" was misspelled or could not be found
What is wrong?
"printf" in this case is a shell command, like from bash shell.
St_W wrote:instead of setting up everything for a working environment yourself i recommend to use msys2 https://www.msys2.org/ or similar preconfigured environments.
@Juergen, yes, do this. A unix-y type shell and commands are needed. find, xargs, sed, and a few other commands are used to implement the framework (find files, build a list of tests, etc), independantly of creating executables. Depending on the shell (e.g. cmd.exe) there are limitations on what may still work. Setting up the environment yourself can to lead incompatible versions of tools in the tool chain or shell. I see you are running the test suite for fbc 1.05.0. I haven't run that version in a long time. Eventually, you will find the CUnit library dependence. This dependency has been removed in fbc 1.06 The framework and format of the tests are different in fbc-1.06.

For fbc-1.06:

1) See ./tests/readme.txt

2) $ make
- this will print a list of make options

3) $ make check
- this will test some basic set-up, you should get something like:

Code: Select all

D:\fb.git\tests>make check
Checking that fb compiler fbc.exe exists: OK
Checking that fbc.exe working: OK
Checking that compiled program executes: OK
4) Do a small test first: $ make unit-tests DIRLIST=pretest

Code: Select all

D:\fb.git\tests>make unit-tests DIRLIST=pretest
make -f unit-tests.mk
make[1]: Entering directory 'D:/fb.git/tests'
Generating unit-tests.inc : Done
cd fbcunit && make FPU= ARCH= TARGET=
make[2]: Entering directory 'D:/fb.git/tests/fbcunit'
fbc -mt -g -exx -i ./inc -lib src/fbcunit.bas src/fbcunit_qb.bas src/fbcunit_console.bas src/fbcunit_report.bas -x lib/libfbcunit.a
make[2]: Leaving directory 'D:/fb.git/tests/fbcunit'
fbc.exe -c -w 3 -i fbcunit/inc -m fbc-tests -mt pretest/compile_with_fbcunit.bas
fbc.exe -c -w 3 -i fbcunit/inc -m fbc-tests -mt pretest/style_direct.bas
fbc.exe -c -w 3 -i fbcunit/inc -m fbc-tests -mt pretest/style_module.bas
fbc.exe -c -w 3 -i fbcunit/inc -m fbc-tests -mt pretest/style_namespace.bas
fbc.exe -c -w 3 -i fbcunit/inc -m fbc-tests -mt pretest/style_simple.bas
fbc.exe -c -w 3 -i fbcunit/inc -m fbc-tests -mt fbc-tests.bas
fbc.exe -l fbcunit -l user32 -p fbcunit/lib -x fbc-tests.exe ./pretest/compile_with_fbcunit.o ./pretest/style_direct.o ./pretest/style_module.o ./pretest/style_namespace.o ./pretest/style_simple.o ./fbc-tests.o
./pretest/compile_with_fbcunit.o() warning 20(2): Object files or libraries with mixed multithreading (-mt) options
./fbc-tests.exe

SUMMARY

 Asserts    Passed    Failed  Suite                                      Tests
--------  --------  --------  --------------------------------------  --------
       3         3         0  fbc_tests.pretest.style_simple                 3
       2         2         0  fbc_tests.pretest.style_namespace              2
       1         1         0  fbc_tests.pretest.style_module                 1
       2         2         0  fbc_tests.pretest.style_direct                 2
       1         1         0  fbc_tests.pretest.compile_with_fbcunit         1
--------  --------  --------  --------------------------------------  --------
       9         9         0  Total                                          9

make[1]: Leaving directory 'D:/fb.git/tests'

D:\fb.git\tests>
Juergen Kuehlwein
Posts: 284
Joined: Mar 07, 2018 13:59
Location: Germany

Re: Compiler tests

Post by Juergen Kuehlwein »

Thanks Jeff,

so the point is, it cannot be run from a Windows command prompt at all. I must use a different shell, which has to be installed first on Windows.

for "make unit-tests DIRLIST=pretest" i get:
make -f unit-tests.mk
make[1]: Verzeichnis „/d/freebasic/jk-fork/tests“ wird betreten
fbc.exe -l fbcunit -l user32 -p fbcunit/lib -x fbc-tests.exe ./pretest/compile_with_fbcunit.o ./pretest/style_direct.o ./pretest/style_module.o ./pretest/style_namespace.o ./pretest/style_simple.o ./fbc-tests.o
./pretest/compile_with_fbcunit.o() warning 20(2): Object files or libraries with mixed multithreading (-mt) options
./fbc-tests.exe

SUMMARY

Asserts Passed Failed Suite Tests
-------- -------- -------- -------------------------------------- --------
3 3 0 fbc_tests.pretest.style_simple 3
2 2 0 fbc_tests.pretest.style_namespace 2
1 1 0 fbc_tests.pretest.style_module 1
2 2 0 fbc_tests.pretest.style_direct 2
1 1 0 fbc_tests.pretest.compile_with_fbcunit 1
-------- -------- -------- -------------------------------------- --------
9 9 0 Total 9

make[1]: Verzeichnis „/d/freebasic/jk-fork/tests“ wird verlassen
which is similar but not equal to what you posted.

Running "make log-tests" results in this in "failed-fb.log":
FAILED LOG - for log-tests -lang fb
./cpp/mangle.log:cpp/mangle.bmk : RESULT=FAILED
./functions/stack-align/stack-alignment.log:functions/stack-align/stack-alignment.bmk : RESULT=FAILED
./namespace/cpp/cpp.log:namespace/cpp/cpp.bmk : RESULT=FAILED
Is this expected ?


JK


(Btw, a short hint in the readme.txt stating this for Unix and Linux dummies like me, would have saved me a lot of time)
Juergen Kuehlwein
Posts: 284
Joined: Mar 07, 2018 13:59
Location: Germany

Re: Compiler tests

Post by Juergen Kuehlwein »

Jeff,

Running "make unit-tests" i see 10 fails in wstring\utf_conv.bas - is this correct ?

JK
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Compiler tests

Post by coderJeff »

Juergen Kuehlwein wrote:
FAILED LOG - for log-tests -lang fb
./cpp/mangle.log:cpp/mangle.bmk : RESULT=FAILED
./functions/stack-align/stack-alignment.log:functions/stack-align/stack-alignment.bmk : RESULT=FAILED
./namespace/cpp/cpp.log:namespace/cpp/cpp.bmk : RESULT=FAILED
Is this expected ?
This is not expected, though it looks familiar like I've seen it happen before. Those specific tests require linking an fbc .BAS module to gcc .C/.CPP module. It could be either gcc not found (not on PATH), or wrong version of gcc. If checking PATH does not fix, can also set environment variable "set GCC=path-to-gcc/gcc.exe" to force a specific version of gcc.
Juergen Kuehlwein wrote: (Btw, a short hint in the readme.txt stating this for Unix and Linux dummies like me, would have saved me a lot of time)
Thanks. Very few people try the test suite, so we don't get much feed back.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Compiler tests

Post by coderJeff »

Juergen Kuehlwein wrote: Running "make unit-tests" i see 10 fails in wstring\utf_conv.bas - is this correct ?
These tests are not expected to fail.

Maybe results depend on language code page?
- wstring\utf_conv.bas source file has extended ASCII characters ( >= chr(128) )
- I am running on Win 7, code page 850

So, it could be a bad test.
Juergen Kuehlwein
Posts: 284
Joined: Mar 07, 2018 13:59
Location: Germany

Re: Compiler tests

Post by Juergen Kuehlwein »

Jeff,


thanks for your help! As per your recommendation i use MSYS2 for a command shell, which comes with it´s own version of gcc. So a different gcc and different a codepage make the difference :-)


JK
St_W
Posts: 1619
Joined: Feb 11, 2009 14:24
Location: Austria
Contact:

Re: Compiler tests

Post by St_W »

fbc_tests.string.format.numberFormatTest is the only test known to fail on win x64, on other platforms all tests succeed.

You can find test results (including console output) for windows & linux, x86 & x64 on my CI server's (=Jenkins) website, see https://freebasic.net/forum/viewtopic.php?f=17&t=22903 (but please avoid downloads or other bandwith-intensive tasks, because it's a normal PC that runs the CI server with a typical home-user internet connection, and not a real server in a datacenter; that's also why you'll find the direct link only on the linked page and I avoid using it anywhere else - sorry for the inconvenience).
hint: you can distinguish commands from output in the "console output" by looking at the beginning of the lines: commands are prefixed with one (or more) "+" signs, while regular console output is not (but it could theoretically also contain a "+" sign on the beginning of a line; in that rare case you can't distinguish easily).
Post Reply