Will this array subscript warning bite me later?

General FreeBASIC programming questions.
Post Reply
cbruce
Posts: 163
Joined: Sep 12, 2007 19:13
Location: Dallas, Texas

Will this array subscript warning bite me later?

Post by cbruce »

.
Windows 10 64-bit program compiles and runs ok as far as I can tell, but gets an array subscript warning when compiled with gcc -O2, -O3, or -Ofast optimization options. Is this something to be worried about?

The code stub below compiles and can show the warning that I am asking about if you use one of the above optimization options..

Code: Select all

#include once "fbgfx.bi"
#if __FB_LANG__ = "fb"
  Using FB '' Keyboard scan code constants are stored in the FB namespace in lang FB
#endif
'
type TESTDATA__type
  A as ulongint
  B as ulongint
  C as ulongint
end type
dim shared as TESTDATA__type  gTestData
' ------------------------------------------------------------
sub Wait_For_ESC_or_CtrlC__Then_Return()
  ' Clear Inkey buffer
  While Inkey <> "": Wend
  Do
    Sleep 25
    If MultiKey(SC_ESCAPE) Then          ' [ESC] key?
      exit do
    end if
    If MultiKey(SC_CONTROL) And MultiKey(SC_C) Then       ' [CTRL+C] key combination?
      exit do
    end if
  Loop
end sub
' ------------------------------------------------------------
sub Wait_For_ESC__Then_Return()
  ' Clear Inkey buffer
  While Inkey <> "": Wend
  Do
    Sleep 25
  loop until inkey = chr(27)          ' [ESC] key?
end sub
' ------------------------------------------------------------
Function read_testdata_file() as integer
  dim as integer f
  f = freefile()
  open "TESTDATA.DAT" for input as #f
  If Err > 0 Then
    ' << You folks shouldn't have a TESTDATA.DAT file, so you should be in this error handler >>
    Print "Error opening TESTDATA.DAT"
    Wait_For_ESC_or_CtrlC__Then_Return()    '' << MultiKey() HANGS >>
    '''Wait_For_ESC__Then_Return()    " << InKey() WORKS >>
    print "inside - after the Open ERR wait()"
    close #f
    return 1
  end if
  print "inside - after good file open()"
  '
  dim as string keys(1 to 3)
  input #f, keys(1), _
            keys(2), _
            keys(3)
  If Err > 0 Then
    Print "Error reading TESTDATA.DAT"
    Wait_For_ESC_or_CtrlC__Then_Return()
    '''Wait_For_ESC__Then_Return()
    print "inside - after the Read ERR wait()"
    close #f
    return 2
  end if
  print "inside - after good file read()"
  '
  close #f
  '
  gTestData.A  = val(keys(1))
  gTestData.B  = val(keys(2))
  gTestData.C  = val(keys(3))
  '
  RETURN 0
end Function
' ============================================================
print "main - before the testdata file read()"
read_testdata_file()
print "main - after the testdata file read()"
'
end
The warning is bracketed below by the "<<< ===== >>>" lines...

Code: Select all

Successful Compile (Errors 0 Warnings 0)

Primary Source: D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.bas
Target Compilation: D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.exe (73 KB, 74240 bytes)
Compile Time: 0.2 seconds (2021-04-10 10:52:22)

Command Line: 
D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\fbc64.exe -m "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.bas" -v -s console -gen gcc -Wc -Ofast  -x "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.exe"

FreeBASIC Compiler - Version 1.07.1 (2020-05-19), built for win64 (64bit)
Copyright (C) 2004-2019 The FreeBASIC development team.
standalone
target:       win64, x86-64, 64bit
compiling:    D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.bas -o D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.c (main module)
compiling C:  D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\bin\win64\gcc.exe -m64 -march=x86-64 -S -nostdlib -nostdinc -Wall -Wno-unused-label -Wno-unused-function -Wno-unused-variable -Wno-unused-but-set-variable -Wno-main -Werror-implicit-function-declaration -O0 -fno-strict-aliasing -frounding-math -fno-math-errno -fwrapv -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -Wno-format -masm=intel "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.c" -o "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.asm" -Ofast 
D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.c: In function 'READ_TESTDATA_FILE':
                  <<< ===== >>>
D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.c:152:27: warning: array subscript -1 is below array bounds of 'FBSTRING[3]' {aka 'struct <anonymous>[3]'} [-Warray-bounds]
  *(FBSTRING**)&tmp$13$1 = (FBSTRING*)((uint8*)KEYS$1 + -24ll);
                           ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                  <<< ===== >>>
assembling:   D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\bin\win64\as.exe --64 --strip-local-absolute "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.asm" -o "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.o"
linking:      D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\bin\win64\ld.exe -m i386pep -o "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.exe" -subsystem console "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\lib\win64\fbextra.x" --stack 1048576,1048576 -s -L "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\lib\win64" -L "." "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\lib\win64\crt2.o" "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\lib\win64\crtbegin.o" "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\lib\win64\fbrt0.o" "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\Untitled1.o" "-(" -lfbgfx -lgdi32 -lwinmm -luser32 -lfb -lgcc -lmsvcrt -lkernel32 -lmingw32 -lmingwex -lmoldname -lgcc_eh "-)" "D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\lib\win64\crtend.o" 
Thanks,
CBruce
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: Will this array subscript warning bite me later?

Post by MrSwiss »

Best bet is: get the latest version of WinFBE (GCC 5.2) release ...
Later GCC versions are not yet 'officially released' .
cbruce
Posts: 163
Joined: Sep 12, 2007 19:13
Location: Dallas, Texas

Re: Will this array subscript warning bite me later?

Post by cbruce »

Thanks @MrSwiss. gcc 5.2 shows no warnings with this code!
deltarho[1859]
Posts: 4292
Joined: Jan 02, 2017 0:34
Location: UK
Contact:

Re: Will this array subscript warning bite me later?

Post by deltarho[1859] »

Replace

Code: Select all

dim as string keys(1 to 3)
with

Code: Select all

dim as string keys(3)
Obviously, keys(0) is redundant, but your subscript warning may not show now.

I may be wrong but what we have here, I reckon, is C/C++ evolving and so does gcc but fbc remains in the dark ages and why gcc 5.2 throws no error warning because gcc 5.2 is in the dark ages.

Fortunately, such cases are rare. Considering the gap between 5.2 and 8.4 matters could be a lot worse. This problem starts with gcc 6.4 ( released 4 July 2017 ) and above. Whenever I see 'warning: array subscript -1 is below array bounds' I just ignore it if I get a successful compile and an apparently successful execution. The only time that I will use 5.2 is if someone put a shotgun to my head. I now use gcc 10.2 as my default. Mind you I have a lot more toolchains that I can fall back onto before landing on 5.2.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Will this array subscript warning bite me later?

Post by coderJeff »

deltarho[1859] wrote:I may be wrong but what we have here, I reckon, is C/C++ evolving and so does gcc but fbc remains in the dark ages and why gcc 5.2 throws no error warning because gcc 5.2 is in the dark ages.
I don't know, this seems like a false positive on gcc's side. The emitted C code is same for in any case and only shows on optimizations.
My opinion is that the warning is meaningless.

Short story long ... Couple things happening here:
- fbc is calling gcc with '-Wall' so that all warnings are enabled
- you could disable the warning by passing '-Wc -Wno-array-bounds'
- fbc emits the fixed length array as a C array to have C automatically allocate the array on the stack, alignment, etc.
- fbc optimizes the array indexing by calculating a pointer as if the array had a 0 element position (which it doesn't so the pointer is actually outside the C array). The intent is to save some cycles and make the array indexing simpler by eliminating an offset calculation on every array access.
- fbc also casts away (or tries to) the array type in C and just work with pointers and pointer indexing.
- But it seems that when gcc has optimizations enabled it can't forget that the memory location is a C array and gives the warning even though we cast to non-array pointer.

One fix is to have fbc add '–Wno-array-bounds' automatically. The other is to emit less optimized code to C, then let gcc handle the optimizing -- just to quiet the warning.
deltarho[1859]
Posts: 4292
Joined: Jan 02, 2017 0:34
Location: UK
Contact:

Re: Will this array subscript warning bite me later?

Post by deltarho[1859] »

coderJeff wrote:- you could disable the warning by passing '-Wc -Wno-array-bounds'
Yep, that worked.

However, it begs the question of why don't we have an issue with gcc 5.2.
One fix is to have fbc add '–Wno-array-bounds' automatically.
With WinFBE I could hard-wire that into SetCompilerSwithesII (SCSII). I hard-wire '-fpu sse' in the latest SCSII but Paul doesn't include the latest SCSII in his WinFBE builds. Either he missed it or decided against it.

I won't bother because ignoring that warning does not appear to be problematic, not everyone uses WinFBE, and you may do something where an updated SCSII is no longer needed.

Note: Some may think that I am being a bit fbc negative of late. I am not - I'm just trying to avoid folk from not migrating from gcc 5.2. Flaming heck, gcc are developing gcc 11. Image It saddens me to think that if you do not include 5.2 as an additional to 8.x in fbc 1.08 you may have a riot on your hands.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Will this array subscript warning bite me later?

Post by coderJeff »

deltarho[1859] wrote:However, it begs the question of why don't we have an issue with gcc 5.2.
I don't know, some of the *active* gcc bug reports are from 10 years ago and updates over years.

Here's what I see though:

Code: Select all

dim as string x(1 to 4)
x(1) = chr(5,5)
With -O 2 or -O 3 optimization this will give gcc's array bounds warning (on a newer gcc).

There's a specific gcc optimization -ftree-vrp that is enabled with -O2 or higher. Disabling this one optimization also disables the bounds warning.
fbc-win64 array.bas -O 3 -Wc -fno-tree-vrp
-- no warning
The emitted C code is same with or without the warning.

Only description I found: https://gcc.gnu.org/onlinedocs/gcc/Opti ... tions.html
-ftree-vrp
Perform Value Range Propagation on trees. This is similar to the constant propagation pass, but instead of values, ranges of values are propagated. This allows the optimizers to remove unnecessary range checks like array bound checks and null pointer checks. This is enabled by default at -O2 and higher. Null pointer check elimination is only done if -fdelete-null-pointer-checks is enabled.
deltarho[1859]
Posts: 4292
Joined: Jan 02, 2017 0:34
Location: UK
Contact:

Re: Will this array subscript warning bite me later?

Post by deltarho[1859] »

I have been to the last link before and forgotten what a beast the gcc compiler is. It is a bit like delving deeper and deeper into Windows and the deeper we go, the more we wonder how it manages to do anything right but, for the most part, it does; similarly with gcc. One thing is for sure when -O2 or higher works without issues the performance boost is extraordinary compared with no optimization.
cbruce
Posts: 163
Joined: Sep 12, 2007 19:13
Location: Dallas, Texas

Re: Will this array subscript warning bite me later?

Post by cbruce »

coderJeff wrote:There's a specific gcc optimization -ftree-vrp that is enabled with -O2 or higher. Disabling this one optimization also disables the bounds warning.
fbc-win64 array.bas -O 3 -Wc -fno-tree-vrp
I use the latest WinFBE, so i don't know if it is related to the WinFBE build of FreeBASIC or not... but when I try to add the -fno-tree-vrp optimization flag I get an error that it is an invalid option.:

Code: Select all

Failed Compile (Errors 2  Warnings 0  [2021-04-13  01:55:54])

Command Line: 
D:\FB_WinFBE\WinFBE_Suite\FreeBASIC-1.07.1-gcc-8.4\fbc64.exe -m "D:\FreeBASICx64\00_BRUCE\PCG32_FULL\TEST.bas" -v -s console -gen gcc -Wc -Ofast -fno-tree-vrp  -x "D:\FreeBASICx64\00_BRUCE\PCG32_FULL\TEST.exe"

error 81: Invalid command-line option, "-fno-tree-vrp"
deltarho[1859]
Posts: 4292
Joined: Jan 02, 2017 0:34
Location: UK
Contact:

Re: Will this array subscript warning bite me later?

Post by deltarho[1859] »

@cbruce

Try -Wc -Ofast -Wc -fno-tree-vrp

We should be able to use comma separation but that does not work for me.
deltarho[1859]
Posts: 4292
Joined: Jan 02, 2017 0:34
Location: UK
Contact:

Re: Will this array subscript warning bite me later?

Post by deltarho[1859] »

Actually, comma separation does work as in:

-Wc -Ofast,-fno-tree-vrp

but do not put a space after the comma.
cbruce
Posts: 163
Joined: Sep 12, 2007 19:13
Location: Dallas, Texas

Re: Will this array subscript warning bite me later?

Post by cbruce »

Thanks @deltarho and @coderJeff !!
The flag and the additional "-Wc" works great!
[edit] ... as does the comma separation!
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Will this array subscript warning bite me later?

Post by coderJeff »

This warning should be fixed now in fbc 1.08.0.

I made changes to C code generated in the gcc backend so there's no need to remove optimizations or disable warnings to suppress this particular warning.

In working on this one, I noticed:
- as the gcc optimization level goes up, so does the possibility for warnings
- some errors in the test-suite when compiled with -O 2 mostly having to do with floating point precision. I'll probably have to relax the tolerance on tests to allow the test suite to pass, maybe add a __FB_OPTIMIZE__ constant to indicate the optimization level (to help in writing the tests).
deltarho[1859]
Posts: 4292
Joined: Jan 02, 2017 0:34
Location: UK
Contact:

Re: Will this array subscript warning bite me later?

Post by deltarho[1859] »

coderJeff wrote:This warning should be fixed now in fbc 1.08.0.
It looks like it. Image

I have a 9.3 WL 1.08.0 built a little while back where I am having to suppress the warning. I have just received 12.0 ES 1.08.0 from my German friend with the latest 1.08.0 build, and I am not having to suppress the warning. I was going to hard wire the suppression into WinBFE's SetCompilerSwitchesII, but I don't have to now.

Thanks Jeff.

Whilst on the subject of getting fbc to use 'newer stuff' we should have a look at FB_RND_REAL which is still using CryptGenRandom with Windows. This was superseded with BCryptGenRandom in Windows Vista 14 years ago. We should have a Windows version check and use BCryptGenRandom for Windows Vista and later. BCryptGenRandom uses later recommendations from the NIST and is in bcrypt.bi and, undoubtedly, <wincrypt.h>.
Post Reply