GCC compile delay on large projects?

[mrToad]

Edit: I revised this after I realized it wasn't related to fbc version.

Compiling with fbc -gen gcc -Wc -O3 I get almost 10 seconds of delay before compile is successful, tested on two large projects. Using an IDE this delay starts after all the warnings are displayed, and just before Make Done.

This is for Windows. I get the same results whether Win 32 or 64 fbc. By removing -Wc -O3 perhaps four or five seconds of the delay is removed, but it's still considerably longer than compiling with GAS, which had maybe 1 second of delay before Make Done.

Smaller projects don't have the delay, so it is probably the size of the code.

Just curious if this long delay is normal?

[Provoni]

With compile delay do you mean compile time?

My project takes about 2 minutes to compile. Size matters allot especially with the optimization flags added in.

[jj2007]

My project takes about 2 minutes to compile

How many lines of code is that?

[Provoni]

My project takes about 2 minutes to compile

How many lines of code is that?

More than 50k: viewtopic.php?f=8&t=23188

[marcov]

Good chance that that is the linking step. gnu ld is not the fastest linker.

You could try to play with settings (e.g. turn on or off dead code elimination etc)

[jj2007]

My somewhat bigger projects assemble at more than 10k lines per second, on an old Core i5. You must be mad waiting 2 minutes for a compile...

[srvaldez]

Edit: I revised this after I realized it wasn't related to fbc version.

Compiling with fbc -gen gcc -Wc -O3 I get almost 10 seconds of delay before compile is successful, tested on two large projects. Using an IDE this delay starts after all the warnings are displayed, and just before Make Done.

This is for Windows. I get the same results whether Win 32 or 64 fbc. By removing -Wc -O3 perhaps four or five seconds of the delay is removed, but it's still considerably longer than compiling with GAS, which had maybe 1 second of delay before Make Done.

Smaller projects don't have the delay, so it is probably the size of the code.

Just curious if this long delay is normal?

when using gcc, some Basic code gets hugely expanded and can take a long time for the compiler to digest, I am curious about the executable sizes of the GAS and gcc compiles.
sorry I have no concrete example to illustrate this huge expansion, I once had one example borrowed from the QB64 forum but can't find it now, it involved printing a string construct.

[dodicat]

You really will notice a long compile time with gcc if you have a large data section.
That is
DATA 1,2,3,78 , ...
(maybe 10000 + elements, say loading a bitmap via it's colours).

If the data is in an include file it takes ages to load and compile.

Here is a tester.
Use a bitmap of about 800 X 600, 32 or 24 bits depth

Code: Select all

'Data maker
#include "crt.bi"
function ULongToBase(N as ulong,_base as byte) as string
    dim as zstring * 50 buffer
    _itoa(n,@buffer,_base)
    return ucase(buffer)
end function


Screen 20,32
' "colours.bas" wii be the output file

Type bitmap_size
    As Long across,down
End Type

'========================  YOUR BITMAP ===============
Dim  mybitmap As String ="azul.bmp" '<--- here
'==============================================

Redim shared As ULong pixel_colours(0,0) 'shared to increase potential size

Function size(bmp As String) As bitmap_size 'fetch bitmap width/height 
    Dim As Long w,h
    Open bmp For Binary As #1
    Get #1, 19, w
    Get #1, 23, h
    Close #1
    Return Type<bitmap_size>(w,h)
End Function

Sub load(bmp As String,pixels() As Long) 'load bitmap into array of point colours
    Bload bmp
    dim count as Long
    dim flag as long
    dim as string comma=","
    dim as string dash="_"
    Dim temp As bitmap_size=size(bmp)
    Redim pixels(temp.across,temp.down)
    if temp.across*temp.down=0 then print bmp;" Not found":sleep:end
    windowtitle str(temp.across) + ", " + str(temp.across)
    dim max as Long=(1+temp.across)*(1+temp.down)
     For x As Long=0 To temp.across
        For y As Long=0 To temp.down
            count+=1
     Next y
    Next x
    dim as long c=count- count mod 8
    count=0
    For x As Long=0 To temp.across
        For y As Long=0 To temp.down
            count=count+1
            if count>c-1 then flag=1:comma="":dash=""
            dim as ulong clr=Point(x,y) 
            if clr=0 then clr=rgb(0,0,0)
            pixels(x,y)=clr
            if count=max then
                comma=""
                dash=""
            end if
            'THIS IS THE CONVERSION
            if count mod 8=1 then print #2,"""";
            dim as string g= right( "0000000" + ulongToBase(pixels(x,y),36),7)
              print #2, g ; 
            if count mod 8=0 then print #2,"""" & comma + dash
            if flag then exit sub
        Next y
    Next x
    
End Sub

'Use mouse or keys to move bitmap in this run
Sub drawbitmap(mx As long,my As Long,mybitmap As bitmap_size,scale as single=1)
    #macro magnify(pivotx,pivoty,px,py,scale)
    var rotx=scale*(px-pivotx)+pivotx
    var roty=scale*(py-pivoty)+pivoty
    #endmacro
    For x As Long=mx To mx+mybitmap.across
        For y As Long=my To my+mybitmap.down
            magnify(mx,my,x,y,scale)
            line(rotx-scale/2,roty-scale/2)-(rotx+scale/2,roty+scale/2),pixel_colours(x-mx,y-my),BF
        Next y
    Next x
    
End Sub

Dim As bitmap_size bitmap_info=size(mybitmap) 'get height/width of bitmap

'initial running code for external file
open "colours.bas" for output as #2
print #2,"dim shared as long w,h"
print #2,"w=";;str(bitmap_info.across)
print #2,"h=";str(bitmap_info.down)
print #2,"redim shared as ulong a((w+1)*(h+1))"
print #2 ,"DATA _"

draw string(bitmap_info.across,bitmap_info.down), "PLEASE WAIT"

load mybitmap,pixel_colours() 'set bitmap colours into an array


'variables for the mouse and arrow keys
Dim As Long mx,my,mw,mb,copymx,copymy
dim as string i

'Set magnification
Do
    Getmouse(mx,my,mw,mb)
    if mb=2 then exit do
    if mw<0 then mw=10
    if mw=0 then mw=1
    If mb=1 Then
        copymx=mx:copymy=my
    End If
    i=inkey
    if i= chr(255) + "K"  then copymx=copymx-5
    if i= chr(255) + "M"  then copymx=copymx+5
    if i= chr(255) + "P"  then copymy=copymy+5
    if i= chr(255) + "H"  then copymy=copymy-5
    
    Screenlock
    Cls
    
    drawbitmap(copymx,copymy,bitmap_info,mw/10) 'SCALE 
    draw string(20,20),"TURN MOUSE WHEEL  --- MAGNIFICATION = "&str(mw/10)
    draw string(20,50),"USE ARROW KEYS OR"
    draw string(20,80),"LEFT MOUSE TO DRAG IMAGE"
    draw string(20,110),"ESC OR RIGHT CLICK TO END"
    
    
    Screenunlock
    Sleep 1,1
    
Loop Until i=Chr(27)

'More running code for external file:
print #2," #include ";chr(34);"crt.bi";chr(34)
print #2,"screen 20,32"
print #2,"dim shared as double magnification"
print #2,"dim shared as any pointer image"
print #2,"image=imagecreate(w,h)"
print #2,"magnification= "; mw/10

print #2,"function ULongFromBase(N as string,_base as byte) as ulong"
 print #2,"   return strtoul(n,0,_base)"
print #2,"end function"

print #2,"function map(a as single,b as single,x as single,c as single,d as single) as single"
 print #2,"   return ((d)-(c))*((x)-(a))/((b)-(a))+(c)"
print #2,"end function"

print #2,"function scaler(im as any ptr) as any ptr"
print #2,"    dim as integer w,h"
print #2,"    dim as long x,y,xpos,ypos"
print #2,"    imageinfo im,w,h"
print #2,"    dim as long nx=w*magnification,ny=h*magnification"
print #2,"    dim as any ptr im2=imagecreate(nx,ny)"
print #2,"    for x=0 to nx"
print #2,"        for y =0 to ny"
print #2,"            xpos=map(0,nx,x,0,w)"
print #2,"            ypos=map(0,ny,y,0,h)"
print #2,"            pset im2,(x,y),point(xpos,ypos,im)" 
print #2,"        next y"
print #2,"    next x"
print #2,"    return im2"
print #2,"end function"

print #2,"sub read_data()"
print #2,"    dim as long ctr"
print #2,"  dim as single c=ubound(a)\8"
print #2,"redim as string s(1 to c)"
print #2,"for n as long=1 to c"
 print #2,"   read s(n)"
print #2,"    for m as long=1 to len(s(n)) step 7"
 print #2,"       ctr+=1"
print #2,"        dim as ulong x= ULongFromBase( mid(s(n),m,7) ,36)"
print #2,"        a(ctr)=x"
print #2,"        next m"
print #2,"    next n"
print #2,"end sub"

print #2,"Sub drawbitmap_to_image(scale as single=1,mx As long=0,my As long=0)"
print #2,"dim as long ctr"
print #2,"For x As long=0 To w"
print #2,"For y As long=0 To h"
print #2,"pset image,(x,y),a(ctr)"
print #2,"ctr+=1"
print #2,"Next y"
print #2,"Next x"
print #2,"End Sub"

print #2,"read_data()"

print #2,"drawbitmap_to_image(magnification)"

print #2,"var image2=scaler(image)"

'BITMAP image is now in image
'This just displays the image

print #2,"locate 2,2"
print #2,"windowtitle " ;chr(34)+ " magnification = " + str(mw/10);chr(34)
print #2,"put( 0, 0),image2,pset"

print #2,"Sleep"

print #2,"imagedestroy image"
print #2,"imagedestroy image2"

close #2
locate 10,10
print "SAVED -- colours.bas is your file"
print "Press a key to end"
sleep


  

Use your mouse wheel to re-size the bitmap, and right click to save colours.bas.

Running colours.bas, -gen gas is almost immediate, -gen gcc takes some minutes to compile.

[mrToad]

@srvaldez,
GCC : 522kb (~6 seconds wait to finish compile to exe)
GCC -Wc -O3 : 489kb (~10 seconds wait to finish compile to exe)
GAS: 423kb (~1 second wait to finish compile to exe)

@dodicat, I tested that code, it took a couple minutes for a 640x480 bitmap. Wow, interesting, very slow. I found ~100 DATA statements of some old (unused) code in one of my projects and removed them. It did not help compile time. I don't normally use DATA anyway. But maybe some old lib that is included is a problem, I don't know.

Something is making it oddly slow for a pretty fast processor, it seems to me. Just curiosity, I can live with the wait. :)

[jj2007]

Here is a tester.
Use a bitmap of about 800 X 600, 32 or 24 bits depth

With a 800x600 full colour bitmap, it takes a few seconds with Gas but ages with Gcc. The resulting exe is 7.46 MB for Gas but only 1.37MB for Gcc. Can it be that Gcc optimises for size?

[dodicat]

Hi jj2007.

I tested with a bitmap 800 X 600.

-gen gcc -Wc -O3 64 bits
Compile time 135 seconds
Size of .exe 4.2 Mb.

-gen gcc -Wc -O3 32 bits
Compile time 130 seconds
Size of .exe 3.8 Mb.

-gen gas
Compile time 1.8 seconds
Size of .exe 3.8 Mb.

This file (windows only) can process .bmp, and several other common formats. jpeg, .jpg . . .

Code: Select all

 

'Data maker
'Thank to member UEZ for the gdi procedures in Pload/getsize.
#include "crt.bi"

#if sizeof(integer)=8
#include "windows.bi"
#endif
#Include  "win/gdiplus.bi"
function ULongToBase(N as ulong,_base as byte) as string
    dim as zstring * 50 buffer
    _itoa(n,@buffer,_base)
    return ucase(buffer)
end function


Screen 21,32
' "colours.bas" wii be the output file

Type bitmap_size
    As Long across,down
End Type

'========================  YOUR BITMAP ===============
Dim  mybitmap As String ="AZUL.bmp" '<--- here
'==============================================

Redim shared As ULong pixel_colours(0,0) 'shared to increase potential size

function getsize(picture as string) as bitmap_size
    Dim As uinteger TMP
    dim as single w,h
   GDIPLUS.GdiplusStartup(@TMP,@type<GDIPLUS.GDIPLUSSTARTUPINPUT>(1),0)
   Dim as any Ptr Img
   if GDIPLUS.GdipLoadImageFromFile(Picture,@Img)>0 then exit function
   GDIPLUS.GdipGetImageDimension(Img,@w,@h)
   return type(w,h)
end function

Function Pload(Picture as String,byref i as any ptr=0) as long
   Dim As uinteger TMP 
   GDIPLUS.GdiplusStartup(@TMP,@type<GDIPLUS.GDIPLUSSTARTUPINPUT>(1),0)
   Dim as any Ptr Img
   if GDIPLUS.GdipLoadImageFromFile(Picture,@Img)>0 then return 0
   Dim As Single w,h
   GDIPLUS.GdipGetImageDimension(Img,@w,@h)
   if w*h=0 then return 0
   Dim As GDIPLUS.BitmapData Pdata
   Dim As Rect R=Type(0,0,w-1,h-1)
   GDIPLUS.GdipBitmapLockBits(Img,Cast(Any Ptr,@R),GDIPLUS.ImageLockModeRead,PixelFormat32bppARGB,@Pdata)
   For y as long = 0 To h-1
      For x as long = 0 To w-1 
           pset i,(x,y),Cast(ulong Ptr,Pdata.Scan0)[y*w+x]
      Next
   Next
return w*h
End Function

Sub load(bmp As String,pixels() As Long) 'load bitmap into array of point colours
    'Bload bmp
    pload bmp
    dim count as Long
    dim flag as long
    dim as string comma=","
    dim as string dash="_"
    Dim temp As bitmap_size=getsize(bmp)
    Redim pixels(temp.across,temp.down)
    if temp.across*temp.down=0 then print bmp;" Not found":sleep:end
    windowtitle str(temp.across) + ", " + str(temp.down) + " (original size)"
    dim _max as Long=(1+temp.across)*(1+temp.down)
     For x As Long=0 To temp.across
        For y As Long=0 To temp.down
            count+=1
     Next y
    Next x
    dim as long c=count- count mod 8
    count=0
    For x As Long=0 To temp.across
        For y As Long=0 To temp.down
            count=count+1
            if count>c-1 then flag=1:comma="":dash=""
            dim as ulong clr=Point(x,y) 
            if clr=0 then clr=rgb(0,0,0)
            pixels(x,y)=clr
            if count=_max then
                comma=""
                dash=""
            end if
            'THIS IS THE CONVERSION
            if count mod 8=1 then print #2,"""";
            dim as string g= right( "0000000" + ulongToBase(pixels(x,y),36),7)
              print #2, g ; 
            if count mod 8=0 then print #2,"""" & comma + dash
            if flag then exit sub
        Next y
    Next x
    
End Sub

'Use mouse or keys to move bitmap in this run
Sub drawbitmap(mx As long,my As Long,mybitmap As bitmap_size,scale as single=1)
    #macro magnify(pivotx,pivoty,px,py,scale)
    var rotx=scale*(px-pivotx)+pivotx
    var roty=scale*(py-pivoty)+pivoty
    #endmacro
    For x As Long=mx To mx+mybitmap.across
        For y As Long=my To my+mybitmap.down
            magnify(mx,my,x,y,scale)
            line(rotx-scale/2,roty-scale/2)-(rotx+scale/2,roty+scale/2),pixel_colours(x-mx,y-my),BF
        Next y
    Next x
    
End Sub

Dim As bitmap_size bitmap_info=getsize(mybitmap) 'get height/width of bitmap

'initial running code for external file
open "colours.bas" for output as #2
print #2,"dim shared as long w,h"
print #2,"w=";;str(bitmap_info.across)
print #2,"h=";str(bitmap_info.down)
print #2,"redim shared as ulong a((w+1)*(h+1))"
print #2 ,"DATA _"

draw string(bitmap_info.across,bitmap_info.down), "PLEASE WAIT"

load mybitmap,pixel_colours() 'set bitmap colours into an array


'variables for the mouse and arrow keys
Dim As Long mx,my,mw,mb,copymx,copymy
dim as string i
'Set magnification
Do
    Getmouse(mx,my,mw,mb)
    if mb=2 then exit do
    if mw<0 then mw=10
    if mw=0 then mw=1
    If mb=1 Then
        copymx=mx:copymy=my
    End If
    i=inkey
    if i= chr(255) + "K"  then copymx=copymx-5
    if i= chr(255) + "M"  then copymx=copymx+5
    if i= chr(255) + "P"  then copymy=copymy+5
    if i= chr(255) + "H"  then copymy=copymy-5
    
    Screenlock
    Cls
    
    drawbitmap(copymx,copymy,bitmap_info,mw/10) 'SCALE 
    draw string(20,20),"TURN MOUSE WHEEL  --- MAGNIFICATION = "&str(mw/10)
    draw string(20,50),"USE ARROW KEYS OR"
    draw string(20,80),"LEFT MOUSE TO DRAG IMAGE"
    draw string(20,110),"ESC OR RIGHT CLICK TO END"
    
    
    Screenunlock
    Sleep 1,1
    
Loop Until i=Chr(27)

'More running code for external file:
print #2," #include ";chr(34);"crt.bi";chr(34)
print #2,"dim shared as double magnification"
print #2,"magnification= "; mw/10
print #2,"dim as long w2,h2"
print #2,"w2 = w*magnification"
print #2,"h2 = h*magnification"
print #2,"screenres w2,h2,32"
print #2,"dim shared as any pointer image"
print #2,"image=imagecreate(w,h)"
print #2,"function ULongFromBase(N as string,_base as byte) as ulong"
 print #2,"   return strtoul(n,0,_base)"
print #2,"end function"

print #2,"function map(a as single,b as single,x as single,c as single,d as single) as single"
 print #2,"   return ((d)-(c))*((x)-(a))/((b)-(a))+(c)"
print #2,"end function"

print #2,"function scaler(im as any ptr) as any ptr"
print #2,"    dim as integer w,h"
print #2,"    dim as long x,y,xpos,ypos"
print #2,"    imageinfo im,w,h"
print #2,"    dim as long nx=w*magnification,ny=h*magnification"
print #2,"    static as any ptr im2:im2=imagecreate(nx,ny)"
print #2,"    for x=0 to nx"
print #2,"        for y =0 to ny"
print #2,"            xpos=map(0,nx,x,0,w)"
print #2,"            ypos=map(0,ny,y,0,h)"
print #2,"            pset im2,(x,y),point(xpos,ypos,im)" 
print #2,"        next y"
print #2,"    next x"
print #2,"    return im2"
print #2,"end function"

print #2,"sub read_data()"
print #2,"    dim as long ctr"
print #2,"  dim as single c=ubound(a)\8"
print #2,"redim as string s(1 to c)"
print #2,"for n as long=1 to c"
 print #2,"   read s(n)"
print #2,"    for m as long=1 to len(s(n)) step 7"
 print #2,"       ctr+=1"
print #2,"        dim as ulong x= ULongFromBase( mid(s(n),m,7) ,36)"
print #2,"        a(ctr)=x"
print #2,"        next m"
print #2,"    next n"
print #2,"end sub"

print #2,"Sub drawbitmap_to_image(scale as single=1,mx As long=0,my As long=0)"
print #2,"dim as long ctr"
print #2,"For x As long=0 To w"
print #2,"For y As long=0 To h"
print #2,"pset image,(x,y),a(ctr)"
print #2,"ctr+=1"
print #2,"Next y"
print #2,"Next x"
print #2,"End Sub"

print #2,"read_data()"

print #2,"drawbitmap_to_image(magnification)"

print #2,"var image2=scaler(image)"

'BITMAP image is now in image
'This just displays the image

print #2,"locate 2,2"
print #2,"windowtitle " ;chr(34)+ " magnification = " + str(mw/10);chr(34)
print #2,"put( 0, 0),image2,pset"

print #2,"Sleep"

print #2,"imagedestroy image"
print #2,"imagedestroy image2"

close #2
locate 10,10
print "SAVED -- colours.bas is your file"
print "Press a key to end"
sleep


 

[marcov]

Here is a tester.
Use a bitmap of about 800 X 600, 32 or 24 bits depth

With a 800x600 full colour bitmap, it takes a few seconds with Gas but ages with Gcc. The resulting exe is 7.46 MB for Gas but only 1.37MB for Gcc. Can it be that Gcc optimises for size?

Not likely. Differences like that are either dead code elimination or debug info.

[coderJeff]

The long compile time for gcc is due a bug in string building found within the gcc emitter itself:
Speed issue with string concatenation and a solution
Memory allocation study for character data of var-len string

I don't think there is a bug report on sf.net.

gas emitter can emit data statements to ASM as they are encountered.
gcc emitter must build a list of all the data statements before emit, and due the bug mentioned above, we get really long compile times with gcc emitter and data statements.

Workaround would be to modify the string builder in fbc's gcc emitter and avoid the bug.
Solution would be to fix the bug in compiler/run-time.

[mrToad]

@coderJeff, that insight really does help explain things.

I'm not sure where in my project there are heavy Data statements though, maybe tucked away in an old attached lib, I will look. I don't care for them anyway, but it's nice to understand the problem. Thanks for your reply.

[coderJeff]

So we don't lose this: #917 Long complile times for gcc backend