Len of udt

General FreeBASIC programming questions.
Post Reply
Dinosaur
Posts: 1478
Joined: Jul 24, 2005 1:13
Location: Hervey Bay (.au)

Len of udt

Post by Dinosaur »

Hi All

Basically I want to write this udt to a file, and it does not matter which way I approach it, the numbers don't stack up.
The UDT

Code: Select all

Type Smarties
    PortNbr     As  Integer = 0      'as edited in Settings2
    Emergency   As  Long    = 0
    AutoMan     As  Long    = 0       '0 or 1
    Ignore      As  Long    = 0       '0 or 1
    DayNbr(1 to 7) As  Long
    On1Serial   As  Integer = 0
    Off1Serial  As  Integer = 0
    On2Serial   As  Integer = 0
    Off2Serial  As  Integer = 0
    OnOff       As  Integer = 0
    RelayState  As  Integer = 0
    Done1       As  Integer = 0
    Done2       As  Integer = 0
    OnHr1       As  Integer = 0        'Say 9 am
    OnMin1      As  Integer = 0
    OffHr1      As  Integer = 0        'Say 9 am
    OffMin1     As  Integer = 0

    OnHr2       As  Integer = 0        'Say 9 am
    OnMin2      As  Integer = 0
    OffHr2      As  Integer = 0        'Say 9 am
    OffMin2     As  Integer = 0

    Status      as  Integer = 0        'indicates if all schedules are done
    Led         as  Long    = 0
    Delay       as  Long    = 0        'Transition Delay
    Hue         as  Long    = 0        '0 to 360
    Sat         as  Long    = 0        'How much of that color
    Temp        as  Long    = 0        '0 for Color, 2500 to 9000 for White
    Bright      as  Long    = 0        '0 to 100
    OldHue      as  Integer = 0
    OldSat      as  Integer = 0
    OldTemp     as  Integer = 0
    OldBright   as  Integer = 0
    OldDelay    as  Integer = 0
    Jaar        as  Integer = 2019
    Maand       as  Integer = 11
    Dag         as  Integer = 5
    Uur         as  Integer = 9
    Mins        as  Integer = 0
    Sec         as  Integer = 0
    PwrSave     as  Long    = 0
    ShutDown    as  Long    = 0
    DayName     as  String * 9
End Type
Dim Shared Smarty(0 to 25) as Smarties
From my calculation there are 29 Integers , 18 Long & 9 bytes of String = 269 bytes
Multiply that x 26 records in the file should be 6994 bytes.
BUT, when I "PUT" this into a file the print statement Sizeof(Smarty) shows 320 and for 26 shows 8320. ??

Code: Select all

    If fileexists(ErrControl.ErrFile) Then
        Open ErrControl.ErrFile For Binary As #rF Len = SizeOf(Smarty)
        Print "Len to write = ";SizeOf(Smarty) * 26
        Print "LOF before   = ";Lof(rF)
        For Xq as Integer = 0 to 25
            Put #rF,Xq,Smarty(Xq)
        Next
        Print "LOF after  = ";Lof(rF)
    EndIf
Len to write = 8320
LOF before = 8000
LOF after = 8000
I have tried Binary, Random,opening by Len(Smarty), opening by SizeOf(Smarty) and each time the numbers appear wrong.
The filesize Lof(rF) after the file is opened (after previously being written to) reports 8000.Equally the file is actually 8000 bytes.
8000 / 26 = 307.69 ??
So obviously I have something fundamentally wrong.
Then just to verify my sanity, I did "Print SizeOf(Smarty(1).Ignore)" which is a long. It reports 4 and "SizeOf(Smarty(1).PortNbr)"
which is an Integer reports 8.
Now I am totally confused.
I am used to being able to calculate the length and ending up with a file that is perhaps 1 byte larger for eof, but all these numbers
don't add up.

Would love someone to point me to the error of my ways,it's got to be something really stupid and simple.

Regards
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: Len of udt

Post by caseih »

Hmm. By my calculations 29*8 + 18*4 + 9 is 313 bytes. Obviously I'm missing some padding that's going on. No clue what's going on, though. You should look at your file in a hex editor.

As for your estimate, INTEGER is 8 bytes, and LONG is 4 bytes. This is unlike C where int is usually 32 bits no matter what the platform, and long int is usually 64-bits. I guess this insanity is why the C standard library defines types like int32_t and int64_t to make it explicit.

EDIT: So looking at the code you posted, you are opening for binary, which means put will place records at the byte position you specify, not the record position (1-based counting, see below). I think you need to open your file as RANDOM, not BINARY. RANDOM allows you to select by record, which seems to be what you're wanting to do.

So it turns out that 320*25 is 8000. So your program is writing 25 records to the file, not 26. I read the docs on PUT and it says "position
Is the position where Put must start in the file. If the file was opened For Random, the position is in records, else it is given in bytes. If omitted, writing starts at the present file pointer position. The position is 1-based: i.e. the first record or byte of a file is at position 1.
If position is omitted or zero (0), file writing will start from the current file position." So your loop starts at 0, which puts that first record probably at the start of the file if it's empty, then it goes to do record 1, which is also at the start.
fxm
Moderator
Posts: 12085
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Len of udt

Post by fxm »

314 bytes for all fields because FreeBASIC adds a terminal character (value = 0) to the fix-len string ('DayName as String * 9').
Total (in 64-bit) = 29*8 + 18*4 + 9+1 = 314

By using 'Field = 1', to cause the Type to be packed as tightly as possible without any padding bytes being added between the fields or at the end of the Type ('Type Smarties Field = 1'), 'Len()' also returns 314.
Otherwise, 6 padding bytes are added at the Type end, after 'DayName' to get a block of length multiple of the length of an 'Integer'. (9+1 + 6 = 16). This is not necessary elsewhere because all blocks of 'Long' fields have lengths in multiples of 2. Therefore, 314 + 6 = 320.

[edit]
For more information, see at:
Structure packing/field alignment
FIELD
Last edited by fxm on Nov 09, 2019 9:46, edited 2 times in total.
RockTheSchock
Posts: 252
Joined: Mar 12, 2006 16:25

Re: Len of udt

Post by RockTheSchock »

If you work with RANDOM files and Type records it's important not to use "As Integer". Just "as Integer" could be 4 byte or 8 byte weather you compile for a 32 or 64 bit platform. . So if at some point you are changing platform, it would break file format compatibility and you could accidently corrupt your record files.

And you must explicitly set Field=x to disable the plattform dependant padding at the end of a record. For best compatibilty with future platforms (not x86 / x64 based) use Field=8. For smallest filesize use Field=1
https://www.freebasic.net/wiki/wikka.ph ... ructLayout

Instead use :
As Integer<32> / As Long
As Integer<64> / As LongInt

https://www.freebasic.net/wiki/wikka.ph ... yPgInteger

If you wonder why there is a padding of 4 or 8 bytes as default, when you are not using explicit FIELD. It's because of speed. Access to unaligned RAM memory can be up to 50% slower.
https://developer.ibm.com/articles/pa-dalign/

Another observation: You can use an Long for the date and a Long for the time or longInt for a datetime.
dodicat
Posts: 7978
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Len of udt

Post by dodicat »

Because of datatype integer then either 32 bit write/read or 64 bit write/read, but cannot cross write/read.

Using sizeof to to load the udt array from file:

Code: Select all

 

#include "file.bi"

Type Smarties
    PortNbr     As  Integer = 0      'as edited in Settings2
    Emergency   As  Long    = 0
    AutoMan     As  Long    = 0       '0 or 1
    Ignore      As  Long    = 0       '0 or 1
    DayNbr(1 to 7) As  Long
    On1Serial   As  Integer = 0
    Off1Serial  As  Integer = 0
    On2Serial   As  Integer = 0
    Off2Serial  As  Integer = 0
    OnOff       As  Integer = 0
    RelayState  As  Integer = 0
    Done1       As  Integer = 0
    Done2       As  Integer = 0
    OnHr1       As  Integer = 0        'Say 9 am
    OnMin1      As  Integer = 0
    OffHr1      As  Integer = 0        'Say 9 am
    OffMin1     As  Integer = 0

    OnHr2       As  Integer = 0        'Say 9 am
    OnMin2      As  Integer = 0
    OffHr2      As  Integer = 0        'Say 9 am
    OffMin2     As  Integer = 0

    Status      as  Integer = 0        'indicates if all schedules are done
    Led         as  Long    = 0
    Delay       as  Long    = 0        'Transition Delay
    Hue         as  Long    = 0        '0 to 360
    Sat         as  Long    = 0        'How much of that color
    Temp        as  Long    = 0        '0 for Color, 2500 to 9000 for White
    Bright      as  Long    = 0        '0 to 100
    OldHue      as  Integer = 0
    OldSat      as  Integer = 0
    OldTemp     as  Integer = 0
    OldBright   as  Integer = 0
    OldDelay    as  Integer = 0
    Jaar        as  Integer = 2019
    Maand       as  Integer = 11
    Dag         as  Integer = 5
    Uur         as  Integer = 9
    Mins        as  Integer = 0
    Sec         as  Integer = 0
    PwrSave     as  Long    = 0
    ShutDown    as  Long    = 0
    DayName     as  String * 9
End Type

Dim  Smarty(0 to 25) as Smarties

print "creating file"
for n as long=0 to 25
    smarty(n).OffMin1=n*3
    smarty(n).OffMin2=25-n
    smarty(n).DayName= str(timer)
    print str(n)+":",smarty(n).OffMin1,smarty(n).OffMin2,smarty(n).DayName
    do
    if mid(str(timer),1,9)<> smarty(n).dayname then exit do
    loop
next


sub loadfile(file as string,b() as Smarties)
   If FileExists(file)=0 Then Print file;" not found":Sleep:end
   var  f=freefile
    Open file For Binary Access Read As #f
    If Lof(f) > 0 Then
      Get #f, , b()
    End If
    Close #f
end sub

Sub savefile(filename As String,p() As Smarties)
    var n=freefile
    If Open (filename For Binary Access Write As #n)=0 Then
        Put #n,,p()
        Close
    Else
        Print "Unable to save " + filename
    End If
End Sub
'===============

savefile("smarty.doc",smarty())
print "saved as smarty.doc"
print "-------------------"


var lngth=filelen("smarty.doc")\sizeof(smarties)
redim as smarties x(0 to lngth-1)
loadfile("smarty.doc",x())

for n as long=lbound(x) to ubound(x)
   print str(n)+":", x(n).OffMin1,x(n).OffMin2,x(n).DayName
next
print "retrieved"
print "-------------------"

kill "smarty.doc"
if fileexists("smarty.doc") then print "Delete smarty.doc manually" else print "OK"
sleep

 
Dinosaur
Posts: 1478
Joined: Jul 24, 2005 1:13
Location: Hervey Bay (.au)

Re: Len of udt

Post by Dinosaur »

Hi All

Many thanks for all the detailed replies and the effort put into it.

This started because of corrupted data in the file that stored the Smarty udt.
In the end I filled all the fields with zeroes except for the string which I filled with
"ABCDEFGHI". In a Hex editor I could see the 7 bytes padded at the end of each
record. But then I also saw the fields corrupting in the file.
For example I changed the string for the second write to "A1CDEFGHI" and saw the "1"
slowly progressing through "AA1CDEFGHI" to "GHI A1CDEFGHI" as if my data was slowly being shifted..

The Long & Integer really stumped me. CGUI demands Long pointers and that is the only reason I use Long.
But I now understand that for C it is really a 32 bit pointer.
But thanks to RockTheShock I will now change my ways on As Integer.

Will work at it today to resolve this, so what I write is what I read.

Regards
caseih
Posts: 2157
Joined: Feb 26, 2007 5:32

Re: Len of udt

Post by caseih »

Dinosaur wrote:For example I changed the string for the second write to "A1CDEFGHI" and saw the "1"
slowly progressing through "AA1CDEFGHI" to "GHI A1CDEFGHI" as if my data was slowly being shifted.
And indeed it was, as you were using PUT with a position index that you intended to be a record number but was in fact a byte number when you opened in BINARY format.
Dinosaur
Posts: 1478
Joined: Jul 24, 2005 1:13
Location: Hervey Bay (.au)

Re: Len of udt

Post by Dinosaur »

Hi All
And indeed it was, as you were using PUT with a position index that you intended to be a record number but was in fact a byte number when you opened in BINARY format.
And I woke up to that the first test after all the replies.

Thanks to all once again.
Oh for the simple QB days.

Regards
Post Reply