gcc 5.2 vs gcc 8.1

[deltarho[1859]]

I didn't have a problem using gcc 8.1 with dodicat's code except I had to use a full path with FileExists.

I had a problem with a Win API not being found some time ago and if I remember correctly it was because of cc1.exe not being in the right location in St_W's 32-bit 8.1 build.

With 32-bit we should have ...\bin\libexec\gcc\i686-w64-mingw32\8.1.0\cc1.exe. ... [1]

With 64-bit we should have '\bin\libexec\gcc\cc1.exe" ... [2]

With St_W's 8.1 builds the latter path was used for both 32-bit and 64-bit.

Added: With Paul Squires' WinFBE_Suite only one libexec folder is used and we have the 64-bit cc1.exe in 'libexec\gcc\'. In 'libexec\gcc\' we have the folder 'i686-w64-mingw32'. FBC 32 digs into that for the 32-bit cc1.exe.

[deltarho[1859]]

Apologies, jj, [1] and [2] above need reversing. ie the short path is for 32-bit and the long path should be for 64-bit.

I spotted this when I downloaded the standalone FBC 1.05 64-bit which I had 'dumped' some time ago. I won't tell you why I resurrected it again but I had a good reason.

[deltarho[1859]]

@St_W

Re my last post to you the 7.3 64-bit build, although slightly different to the 8.1 build, is OK. I put cc1.exe into the wrong location of my WinFBE_Suite.

However, according to the tables in my second post above 7.3 64-bit is coming in a little behind the 5.2 64-bit which is bad news. It may be worthwhile building 7.3 64-bit again.

I definitely cannot test 7.3 32-bit because of the missing libexec folder.

If it transpires that 7.3 is slower than 5.2 then nobody, including me, will want it and I will stay with 8.1, even though it is a 0.1 revision because that is leaving 5.2 standing.

Added: According to benchmark tests, 7.3 is sometimes faster than 5.2 and sometimes slower. It may be that my random number test is one occasion where 5.2 has the edge. However, 8.1 is beating both of them, with 64-bit.

[jj2007]

The good news is that I got dodicat's code running on 8.1. Following your suggestions, I managed with trial and error to find out that ..\FreeBasic\bin\libexec\gcc\i686-w64-mingw32\8.1.0\cc1.exe is needed to make gcc happy. That is the only file in the FreeBasic\bin\libexec\gcc\i686-w64-mingw32\8.1.0 folder. What an ugly mess, this "toolchain" - did they ever hear about so-called "installers"?

The bad news is that it's not faster at all. Exactly the same time as for 4.9.2 code.

[deltarho[1859]]

I spotted the fact that cc1.exe should use the shorter path for 32-bit and the longer path for 64-bit when St_W introduced his gcc 8.1 builds. This situation is true for the FBC 1.05 32-bit installation and the FBC 1.05 64-bit standalone. I joined FB at 1.05 but I suspect this debacle has been the case for years. Of course, for those that do not 'dabble' with their setups, it makes no odds.

The bad news is that it's not faster at all. Exactly the same time as for 4.9.2 code.

We need to be careful with generalized statements. There are quite a few very extensive comparisons of the different versions of the gcc compiler. In my opening post I wrote, "Obviously, we cannot generalize from a specific test like this so I cannot say if your kind of work will get much of a boost." With my 'real life' random number test gcc 8.1 is streets ahead of gcc 5.2 with both 32-bit and 64-bit.

GCC 8.1 vs. GCC 7.3 Compiler Benchmarks On Five AMD/Intel Linux Systems had this to say: "Overall, the GCC 8.1 performance does appear to be in good shape and in a number of tests did show measurable performance improvements over GCC 7.3. Long story short, of the Intel and AMD systems tested with Ubuntu 18.04 x86_64, the biggest winner was Intel Skylake hardware seeming to show performance improvements most often but also the AMD Zen performance did improve too in a number of the tests carried out for this article today." Note that is on Linux.

The jury is out with regard gcc 7.3 on Windows for the moment but I do not think that I am sticky my neck out by saying that gcc 5.2 should be wrapped up and sent to the nearest museum.

With regard FBC dkl released 8 versions between Jun 24, 2013 and Jan 31, 2016, that is 2½ years. 1.05 has been stuck in the doldrums for 2½ years.

Imagine someone checking out FreeBASIC and looked at its history and then realized gcc 5.2 has moved on to 6, 7 and 8. They will think that FreeBASIC's development came to a grinding halt in Jan 2016 and move onto another BASIC dialect. Could you blame them?

Food for thought admin.

[deltarho[1859]]

Here is a simple example where there isn't a blind bit of difference between 5.2 and 8.1. Take a string, hash it with SHA512 one million times.

5.2: 19.64 seconds (exe 57856 bytes)
8.1: 19.53 seconds (exe 54784 bytes)

8.1 binaries tend to be smaller than 5.2 binaries.

What would be good is a comparison with a 'beafy' project but I don't have any. <smile>

[St_W]

I definitely cannot test 7.3 32-bit because of the missing libexec folder.

Copy & paste issue in the build script. Fixed it and uploaded a new build fbc_win32_gcc730_0005_2018-10-04.zip
http://users.freebasic-portal.de/stw/bu ... om/gcc730/

cc1.exe paths relative to gcc.exe:
32-bit: ..\i686-w64-mingw32\bin\cc1.exe
64-bit: ..\x86_64-w64-mingw32\bin\cc1.exe
I used different paths than in the other packages to allow simpler combination of 32/64-bit builds.

"gcc.exe: error: CreateProcess: No such file or directory".

That means that gcc.exe can't find the C compiler cc1.exe. I'd advise to not mix binaries of different builds (that doesn't include header files and other files that are not built when compiling the FreeBasic compiler).

[deltarho[1859]]

@St_W

Thank you.

Here is a table comparing gccs 5.2, 7.3 and 8.1. The percentages are relevant to 5.2.

Code: Select all

                         32-bit
                     
               5.2       7.3          8.1
               
RndMT        368.54  529.96  43%  605.82  64%
3            379.64  400.66   5%  441.05  16%
1            385.14  376.97  -2%  425.61  11%
Knuth64      395.02  740.05  87%  876.88 121%
CryptoRndII  405.46  426.92   5%  472.95  16%
4            405.49  427.72   5%  473.10  17%
CMWC4096     410.16  646.30  57%  463.32  13%
PCG32II      412.33  739.42  79%  876.88 112%
2            412.67  434.99   5%  487.03  18%
MsWs         423.38  670.49  58%  785.64  85%
xorshf96     436.18  691.75  58%  821.13  88%
Num Recipes  448.17  714.83  59%  851.20  90%
Infinity     872.87  739.29 -15%  876.71   0%

                           64-bit

                5.2       7.3           8.1
               
1             480.27  465.26  -3%   527.90  10%
RndMT         549.73  653.71  19%   718.88  31%
3             683.06  745.95   9%   836.17  22%
CryptoRndII   696.89  894.05  28%   967.30  39%
CMWC4096      697.87  724.28   4%   807.60  16%
Knuth64       786.34 1185.85  50%  1379.30  75%
4             816.74  883.04   8%   975.25  19%
PCG32II       855.29  930.69   9%  1381.46  62%
2             866.25  951.14  10%  1056.88  22%
xorshf96      903.23  986.03   9%  1206.84  34%
MsWs          912.36 1083.54  19%  1306.19  43%
Num Recipes   924.99 1061.13  15%  1318.81  43%
Infinity     1342.10 1158.24 -14%  1380.77   3%

For some strange reason, the other day, I had "7.3 64-bit is coming in a little behind the 5.2 64-bit". I noticed in your last link the 64-bit was modified not longer after the 32-bit so, perhaps, we have a new 64-bit build.

Anyway, we now have the 7.3 results sitting between 5.2 and 8.1 which we should expect. I have removed FB's generator #5 because no one would use that except for a small number of requests. That leaves us with twelve generators. Using my old maxim of '>= 7%' is significant then 7 out of 12 7.3 32-bit results are significantly faster than 5.2 and by a good margin. With 64-bit we have 10 and 5 with a good margin.

Working on the principle that a revision .3 should be more robust than a revision 0.1 and the fact that 7.3 is outperforming 5.2, with this test, then it would seem that 7.3 would be a good replacement for 5.2.

Unfortunately, I cannot recommend 7.3 because it is not robust. The 64-bit test would not compile with either the -O3 or -O2 optimization level. I had to drop down to -O1. I have another little exe (144KB) which would not compile with -O3 but did with -O2. On the other hand, 8.1 compiled both this test and the other exe at -O3.

What a bummer! Had it not been for this 'wrinkle' I would have been happy to have 7.3 as my default setup. It is worth noting that had -O3 been OK then the 7.3 64-bit would have much better.

@srvaldez

Hi. If I remember correctly you are using gcc 7.3. Have you had any issues compiling 64-bit with -O3? I am thinking that we could have a 'dodgy' 64-bit build.

[srvaldez]

@srvaldez

Hi. If I remember correctly you are using gcc 7.3. Have you had any issues compiling 64-bit with -O3? I am thinking that we could have a 'dodgy' 64-bit build.

I have msys2 with gcc 7.3 and FB in the toolchain but I almost always use the standalone version with gcc 5.2, sometimes I use the version that is packaged by Paul Squires in his WinFBE, especially when testing your code, because you seem to be using it.
there are issues with FB if used with gcc versions higher than 6.4 when compiling to static lib, other than that, there don't appear to be major problems.
I am on vacation from recreational programming, so I won't be doing much if anything programming wise.

[deltarho[1859]]

@srvaldez

Thanks for getting back to me so quickly.

@St_W

Is it possible to have a 'dodgy' 64-bit build? We cannot expect -O3 to always work but the first other source code balking cannot be right. I have -O3 working on 'tiddler' pieces of code.

[St_W]

Is it possible to have a 'dodgy' 64-bit build? We cannot expect -O3 to always work but the first other source code balking cannot be right. I have -O3 working on 'tiddler' pieces of code.

The build configuration is analogous to the other win64 builds and I didn't spot any issues after quickly reviewing it, but still it's certainly possible that something's wrong. I just don't know how I could find out :-|
Do you get any error messages when compiling with -O3 fails? Or could you provide a code example that triggers that issue (the shorter the better)?

(fbc_win64_gcc730_0004_2018-10-03.zip just got re-uploaded; it's exactly the same file as previously)

[deltarho[1859]]

I think that we have something wrong because in a nutshell.
Two pieces of code compile with gcc 5.2 and -O3.
Two pieces of code compile with gcc 8.1 and -O3.
Two pieces of code don't compile with gcc 7.3, one failing with -O3 and the other failing with -O2.

Or could you provide a code example that triggers that issue (the shorter the better)?

The above test includes a bi file which includes four bas files and then we have another bas file included so that will not be "the shorter the better". The other failing code is over a thousand lines and whilst not large is not short.

Do you get any error messages when compiling with -O3 fails?

Here is the output from poseidonFB, which is better at reporting errors than WinFBE.

Code: Select all

---------------------------------------
-O3 Output
 
"f:/winfbe_suite178x/freebasic-1.06.0/fbc64.exe" -b "F:/FreeBASIC/drRand/MonteCarlo1.bas" -s console -gen gcc -Wc -O3
 
F:/FreeBASIC/drRand/MonteCarlo1.c: In function 'main':
F:/FreeBASIC/drRand/MonteCarlo1.c:522:26: warning: array subscript is below array bounds [-Warray-bounds]
  *(double**)&tmp$313$0 = (double*)((uint8*)A$0 + -8ll);
                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
F:/FreeBASIC/drRand/MonteCarlo1.c:533:28: warning: array subscript is below array bounds [-Warray-bounds]
  *(FBSTRING**)&tmp$314$0 = (FBSTRING*)((uint8*)B$0 + -24ll);
                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
F:/FreeBASIC/drRand/MonteCarlo1.c:836:1: internal compiler error: in print_reg, at config/i386/i386.c:18023
 }
 ^
libbacktrace could not find executable to open
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://sourceforge.net/projects/mingw-w64> for instructions.
 
 
Compile Error!
---------------------------------------
 
---------------------------------------
-O2 Output
 
 
Compile File: F:/FreeBASIC/drRand/MonteCarlo1.bas......
 
"f:/winfbe_suite178x/freebasic-1.06.0/fbc64.exe" -b "F:/FreeBASIC/drRand/MonteCarlo1.bas" -s console -gen gcc -Wc -O2
 
F:/FreeBASIC/drRand/MonteCarlo1.c: In function 'main':
F:/FreeBASIC/drRand/MonteCarlo1.c:522:26: warning: array subscript is below array bounds [-Warray-bounds]
  *(double**)&tmp$313$0 = (double*)((uint8*)A$0 + -8ll);
                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
F:/FreeBASIC/drRand/MonteCarlo1.c:533:28: warning: array subscript is below array bounds [-Warray-bounds]
  *(FBSTRING**)&tmp$314$0 = (FBSTRING*)((uint8*)B$0 + -24ll);
                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 
Compile Error!
---------------------------------------

Of course, this is not FBC talking to us but gcc.

I did some Googling yesterday and found that gcc 7 has had some optimization issues but I could not tie them in with what we are getting; in particular revision .3. If there were any outstanding bugs gcc 7.4 would have been released by now with gcc 7.3 being released 25 Jan 2018.

What I will do is break the above test code down. I am testing 13 random number generators. I will try six. If that fails I will try three of them. If the six are successful then I will try three of the others. That is a binary search so I will not have to do 13 trials. It will take a while all the same.

[deltarho[1859]]

Ok, one of my generators is causing a problem.

The following fails to compile:

Code: Select all

#Include Once "CMWC4096.bas"
#Include Once "PCG32II.bas"

Dim As pcg32 pcg
Dim As Double t, x, y
Dim As Long i

' Section 1
t = Timer + 2
Do
  x = CMWCse
  y = CMWCse
  x = Sqr( x*x + y*y)
Loop Until Timer >= t

' Section 2
t = Timer + 2
Do
  x = pcg.randse
  y = pcg.randse
  x = Sqr( x*x + y*y)
Loop Until Timer >= t 

Print "Done"

Sleep

with

Code: Select all

Compile File: F:/FreeBASIC/drRand/MonteCarlo3.bas......

"f:/winfbe_suite178x/freebasic-1.06.0/fbc64.exe" -b "F:/FreeBASIC/drRand/MonteCarlo3.bas" -s console -gen gcc -Wc -O3

F:/FreeBASIC/drRand/MonteCarlo3.c: In function 'main':
F:/FreeBASIC/drRand/MonteCarlo3.c:122:1: internal compiler error: in print_reg, at config/i386/i386.c:18023
 }
 ^
libbacktrace could not find executable to open
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://sourceforge.net/projects/mingw-w64> for instructions.


Compile Error!

Now the error is different from the above errors but it is a failed -O3 nonetheless.

If we comment #Include Once "CMWC4096.bas" and comment Section 1 then the compilation is successful.

However, and this is crazy, "CMWC4096.bas" compiles successfully on its own.

I wonder what "libbacktrace could not find executable to open" means. We got that in my previous post.

The other code which was failing with -O3 does not use "CMWC4096.bas" and that also mentions "libbacktrace could not find executable to open"

I could remove CMWC4096 from the above test code to see how gcc 7.3 fairs with -O3 but with the other unrelated code failing there may be other failures on the horizon.

On the face of it, it looks like we have a bug in gcc 7.3.

Added: The other code which is failing is Encrypternet.bas and that is fine with 5.2 -O3 and 8.1 -O3.

[srvaldez]

@deltarho[1859]
have you considered using -Ofast instead of -O3 ?
I find that -Ofast is more stable than -O3
if you want small size then use -Os which is stable as well.
I find that -O2 is close to optimum and usually there are no problems, however -Ofast or -Os work when -O2 fails
PS
please provide a link to your include files so I can test.

[deltarho[1859]]

I am going to back away from this for a short while - it is doing my head in.

There is one aspect of WinFBE which infuriates me. I mentioned it to Paul over a year ago and several times since.

I have just compiled some code. I got a message box telling me the compilation time and the size of the exe and had no errors or warnings. The application started to run.

However, when I checked the Output Window I found that the compilation actually failed and the running code was the previous successful compilation.

The only way to be certain of a failure is to kill the previous exe. That way we get told the exe cannot be found because, obviously, the compilation failed.

With poseidonFB there is an option "before Compile, delete Existed Execute File"

I am at a total loss to understand why Paul has done nothing about this.

I have just pointed poseidonFB to gcc 8.1 and it failed. Why? I reckon it is because WinFBE is using i686-w64-ming32 for the 64-bit cc1.exe when it should use x86_64-w64-ming32. I will check that out.

You may remember my finding gcc 4.8.2 in the WinFBE_Suite 1.7.9 instead of gcc 8.1.0. Well, that is also true for 1.7.8 and 1.7.7. I didn't check every revision but had a look at 1.6.0 and gcc 4.8.2 is there as well. So we have been very lucky until 8.1.0 when I started to get the occasional internal compiler error. This seems to have been resolved by using the gcc 8.1.0. I should tell you that I have known about this for a few days but decided not to mention it because I mentioned what to do for 1.7.9.

I like WinFBE, on the whole, but there is a limit to my patience.

I will get to your comments, srvaldez, when I have had a 'rest'. <smile>