Feature request: strings in UDTs

General FreeBASIC programming questions.
dodicat
Posts: 7518
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Feature request: strings in UDTs

Post by dodicat »

I did this a few years ago, with small changes.
You have a udt of fixed length strings of different sizes, or just simply strings.
You have a text file of arbitrary length, you want to load the text file into an array of these string udt's.
Because of these extra terminating chr(0), you must load via ubyte arrays, and transfer the ubtye arrays to the strings.

Code: Select all

#include "file.bi"

type arraytype 
    as ubyte a(1 to 50)
    as ubyte b(1 to 50)
    as ubyte c(1 to 10)
end type

type stringtype
    as string * 50 a,b
    as string * 10 c
    'or
    'as string a,b,c
end type

sub loadfile overload(file as string,b() as arraytype)
	If FileExists(file)=0 Then Print file;" not found":Sleep:end
   var  f=freefile
    Open file For Binary Access Read As #f
    If Lof(f) > 0 Then
      Get #f, , b()
    End If
    Close #f
end sub

Function loadfile(file as string) as String
	If FileExists(file)=0 Then Print file;" not found":Sleep:end
   var  f=freefile
    Open file For Binary Access Read As #f
    Dim As String text
    If Lof(f) > 0 Then
      text = String(Lof(f), 0)
      Get #f, , text
    End If
    Close #f
    return text
end Function

Sub savefile(filename As String,p As String)
    Dim As Integer n
    n=Freefile
    If Open (filename For Binary Access Write As #n)=0 Then
        Put #n,,p
        Close
    Else
        Print "Unable to load " + filename
    End If
End Sub



'create a random string of characters for example
#define irange(f,l) Int(Rnd*(((l)+1)-(f))+(f))
dim as string g=string(1000,0)
for n as long=0 to len(g)-1  '65 90   97 122
    g[n]=iif(rnd>.5,irange(65,90),irange(97,122))'48+rnd*9
next


'save string to text file
savefile("text.txt",g)
print "The original random file"
print loadfile("text.txt")
print


'======================== work with text file ===========


var lngth=filelen("text.txt")\sizeof(arraytype) 'get load dimension

dim as arraytype a(1 to lngth)
dim as stringtype s(1 to lngth)

'must load into ubyte arraytype
loadfile("text.txt",a())

'transfer arraytype to stringtype
for n as long=1 to ubound(a)
   for m as long=1 to 50: s(n).a+= chr(a(n).a(m)):next
   for m as long=1 to 50: s(n).b+= chr(a(n).b(m)):next
   for m as long=1 to 10: s(n).c+= chr(a(n).c(m)):next
next

'now work with stringtype
print "string udt elements .a, .b, .c  1 to ";ubound(s)
for n as long=1 to ubound(s)
    print ".a ="; s(n).a
    print ".b =";s(n).b
    print ".c =";s(n).c
       print 
next

print


'check
print "Check"
'gather up all the string udt characters
dim as string acc
for n as long=1 to ubound(s)
    acc+=s(n).a+s(n).b+s(n).c
next
print
dim as long stringudtsize=len(acc)
print "part of original text file used"
dim as string L= loadfile("text.txt")
print mid(L,1,stringudtsize)
print "All the characters from the array of string udt"
print acc
print iif(acc=mid(L,1,stringudtsize),"OK","ERROR")
    
sleep
kill "text.txt"


  
The only inconvenience is the flipping between ubyte arrays and strings.
speedfixer
Posts: 593
Joined: Nov 28, 2012 1:27
Location: California
Contact:

Re: Feature request: strings in UDTs

Post by speedfixer »

What I do:

1 - a UDT with string pointer as the string place holder
2 - a function that holds a redimmable array of strings that manages adding, removing strings
-- this passes back a pointer of the created string to the caller of the 'add' option of function
-- increases array size when needed
-- removes the string when 'remove' option of function called
-- remove does NOT reduce array size, merely set empty 'flag' (like a nonsense char string or unprintables that can be tested)
-- (above helps with error tracking)
-- subsequent add replaces previous removed string slot in array - usually oldest removed == next in
-- easy to make thread safe
3 - accessory functions like the ubyte manipulation when needed

The string - referenced by the pointer - is usable like any other variable: shared, not shared - depends on who/what holds the pointer, UDT or not.
Making a string = "" does NOT remove the placeholder in the array; pointer is still valid.
Not fast at beginning, but not slow after in use for a short bit.

Simple to manage saving the UDTs if that is desired - though not as a simple flat file.
Any file read/write is custom, anyway.
Save as in random access: not so simple. But not hard.

david
fxm
Moderator
Posts: 11200
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Feature request: strings in UDTs

Post by fxm »

fxm wrote: Jun 24, 2022 12:33 My proposal:
An advanced UDT of size = 5 bytes, and allowing up to 5 useful zstring characters

Code: Select all

Type myZstring5 Extends Zstring
        Declare Constructor(Byref z As Const Zstring = "")
        Declare Operator Let(Byref z As Const Zstring)
        Declare Operator Cast() Byref As Const Zstring
        Declare Operator [](Byval i As Integer) Byref As Ubyte
    Private:
        Const N = 5
        Dim As Ubyte u(0 To myZstring5.N - 1)
End Type

Constructor myZstring5(Byref z As Const Zstring = "")
    This = z
End Constructor

Operator myZstring5.Let(Byref z As Const Zstring)
    FB_memcopyclear(This.u(0), myZstring5.N, z[0], Len(z))
End Operator

Operator myZstring5.Cast() Byref As Const Zstring
    Static As Zstring * (myZstring5.N + 1) z
    FB_memcopyclear(z[0], myZstring5.N + 1, This.u(0), myZstring5.N)
    Operator = z
End Operator

Operator myZstring5.[](Byval i As Integer) Byref As Ubyte
    Operator = This.u(i)
End Operator

Operator Len(Byref mz5 As myZstring5) As Uinteger
    Operator = Len(Str(mz5))
End Operator


Dim As myZstring5 mz5 = "12345"
Print mz5, "Len() = " & Len(mz5), "Sizeof() = " & Sizeof(mz5)
mz5 = "123"
Print mz5, "Len() = " & Len(mz5), "Sizeof() = " & Sizeof(mz5)
mz5 = "abcdefg"
Print mz5, "Len() = " & Len(mz5), "Sizeof() = " & Sizeof(mz5)
mz5[4] = Asc("!")
Print mz5, "Len() = " & Len(mz5), "Sizeof() = " & Sizeof(mz5)
Print Left(mz5, 3)
Print "12345" & mz5

Sleep

The documentation example (from type page) adapted to use my advanced macro-UDT (generated for 6 characters) as member of its type (hdr2) instead of the ubyte array and instead of the static conversion function:

Code: Select all

'' Example showing the problems with fixed length string fields in UDTs
'' Suppose we have read a GIF header from a file
''                        signature         width        height
Dim As ZString*(10+1) z => "GIF89a" + MKShort(10) + MKShort(11)

'---------------------------------------------------------------------

#macro myZstring(size)
    
    Type myZstring##size Extends Zstring
            Declare Constructor(Byref z As Const Zstring = "")
            Declare Operator Let(Byref z As Const Zstring)
            Declare Operator Cast() Byref As Const Zstring
            Declare Operator [](Byval i As Integer) Byref As Ubyte
        Private:
            Dim As Ubyte u(0 To size - 1)
    End Type
    
    Constructor myZstring##size(Byref z As Const Zstring = "")
        This = z
    End Constructor
    
    Operator myZstring##size.Let(Byref z As Const Zstring)
        FB_memcopyclear(This.u(0), size, z[0], Len(z))
    End Operator
    
    Operator myZstring##size.Cast() Byref As Const Zstring
        Static As Zstring * (size + 1) z
        FB_memcopyclear(z[0], size + 1, This.u(0), size)
        Operator = z
    End Operator
    
    Operator myZstring##size.[](Byval i As Integer) Byref As Ubyte
        Operator = This.u(i)
    End Operator
    
    Operator Len(Byref mz As myZstring##size) As Uinteger
        Operator = Len(Str(mz))
    End Operator

#endmacro

'---------------------------------------------------------------------

Print "Using an advanced UDT"

myZstring(6)

Type hdr2 Field = 1
   AS myZstring6 myz6 '' Dimension 6
   As UShort wid, hei
End Type

Dim As hdr2 Ptr h2 = CPtr(hdr2 Ptr, @z)
'' Viewing and comparing is correct

Print h2->myz6, h2->wid, h2->hei '' Prints GIF89a  10  11 (ok)
If h2->myz6 = "GIF89a" Then Print "ok" Else Print "error" '' Prints ok

Sleep
Last edited by fxm on Jun 30, 2022 5:48, edited 1 time in total.
Reason: UDT updated to a macro-generable UDT for any character size.
dodicat
Posts: 7518
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Feature request: strings in UDTs

Post by dodicat »

Looks complicated for a help file
(extends zstring?)
You still use two udt's and ubyte array.
Two udt's and ubyte array

Code: Select all

    
Type GiffHeader
    As String  signature
    As String  w
    As String  h
End Type

Type GiffHeaderReader
    As Ubyte signature(1 To 6)'option for signature length
    As Ubyte w(1 To 2)        'option short
    As Ubyte h(1 To 2)        '    ""
    Declare Sub load(As String)
    Declare Sub TransferTo(As GiffHeader)
End Type

Sub GiffHeaderReader.load(file As String)
    Var  f=Freefile
    If Open (file For Binary Access Read As #f)=0 Then
        If Lof(f) > 0 Then Get #f, , This
        Close #f
    Else
        Print file;" not found":Sleep:End
    End If
End Sub

Sub GiffHeaderReader.TransferTo(z As GiffHeader)
    For m As Long=Lbound(signature) To Ubound(signature): z.signature+= Chr(signature(m)):Next
    For m As Long=Lbound(w) To Ubound(w): z.w+= Chr(w(m)):Next
    For m As Long=Lbound(h) To Ubound(h): z.h+= Chr(h(m)):Next
End Sub

Sub savefile(filename As String,p As String)
    Dim As Integer n
    n=Freefile
    If Open (filename For Binary Access Write As #n)=0 Then
        Put #n,,p
        Close
    Else
        Print "Unable to load " + filename
    End If
End Sub

Dim As String z = "GIF89a" + Mkshort(10) + Mkshort(11)  +"WHATEVER ELSE A GIFF FILE MIGHT CONTAIN" 

savefile("dummy.gif",z)

Dim As GiffHeaderReader r
Dim As GiffHeader h
r.load("dummy.gif")
r.TransferTo(h)
            
Print "signature","width","height"            
Print h.signature,Cvshort(h.w),Cvshort(h.h)
print "Press any key to end . . ."
Sleep
Kill("dummy.gif")
                         
 
fxm
Moderator
Posts: 11200
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Feature request: strings in UDTs

Post by fxm »

dodicat wrote: Jun 26, 2022 9:54 Looks complicated for a help file
(extends zstring?)
My intention is not at all to add this code as an example of the documentation, but just as an example on the forum.
The advantage of my UDT (maybe advanced structure) is its complete compatibility with a Zstring, but without adding a terminal character in used memory.
erik
Posts: 25
Joined: Dec 28, 2020 17:27
Location: Krasnoyarsk
Contact:

Re: Feature request: strings in UDTs

Post by erik »

The string must always be null terminated. Otherwise, it won't work with all string functions, because string functions expect a null character.
In your case, you should use an array of bytes, not a ZString string:

Code: Select all

Type Foo
    Vector(15) As UByte
End Type
marcov
Posts: 3357
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Feature request: strings in UDTs

Post by marcov »

erik wrote: Jun 29, 2022 19:36 The string must always be null terminated.
I'm not sure this is still the case with the -n- functions. Some languages (like Modula-2) employ a scheme that a terminating zero is not necessary if the string exactly fits in the room for it. If that room is somehow guessable or calculable to the compiler, it can pass it to a wrapper that passes it as allocation size to the -n- functions?
Lost Zergling
Posts: 522
Joined: Dec 02, 2011 22:51
Location: France

Re: Feature request: strings in UDTs

Post by Lost Zergling »

If I'm not mistaken, string functions doesn't care about Chr(0) wich is not the case with Zstrings functions (ie ValueOf).
This because string type is using a string descriptor wich stores and maintain string len. You can check it out by building a string containing Chr(0) : it will be correct using string access and truncated using a zstring ptr access. Nevertheless, regular strings are terminated also by a Chr(0), IF I'm not mistaken.
This may seem useless except that a zstring ptr access is possible, meaning compatibility by default in most cases.
angros47
Posts: 2013
Joined: Jun 21, 2005 19:04

Re: Feature request: strings in UDTs

Post by angros47 »

erik wrote: Jun 29, 2022 19:36 The string must always be null terminated. Otherwise, it won't work with all string functions, because string functions expect a null character.
Not necessarily. The only information needed by the string function is where the string ends: and that information can be provided by putting a zero at the end of the string, or by telling the string length to the routine (for example, in the array descriptor)

When a string has fixed length the routines processing it should always work on the whole string (in QBASIC, for example, an empty fixed length string inside a type was a sequence of spaces, but the length never changed, there was no way to shorten it). FreeBasic can work with all the three methods: pre-defined length, or length in the string descriptor, or zero-terminated string. C++, too, uses a string descriptor with the length.

C, on the other end, expects a zero as terminator. And most libraries use a C based interface, so adding a zero at the end of every string makes them compatible with external libraries. Without it, passing a string from an UDT to an external C function would likely cause a segmentation fault, or a data corruption.

Adding the zero prevents that issue, and normally doesn't cause other issues: in fact, the UDT can be declared exactly as it was in QBASIC (in memory it will be allocated differently, but as long as FreeBasic is consistent in how it allocates UDT the program would work the same in FreeBasic and QBASIC).

The only problem could arise if an UDT has been saved (using the PUT command) from a QBASIC file, and a FB program tries to read it: the UDT must be modified, otherwise size won't match and all records would be misaligned. Also, if the string uses all the available space it will lack the terminal zero, and this will cause several errors.
erik
Posts: 25
Joined: Dec 28, 2020 17:27
Location: Krasnoyarsk
Contact:

Re: Feature request: strings in UDTs

Post by erik »

All C functions expect a null‐terminated string. All WinAPI functions expect a null‐terminated string.
Therefore, the talk that the null character in the string is optional is not applicable in practice.
Lost Zergling
Posts: 522
Joined: Dec 02, 2011 22:51
Location: France

Re: Feature request: strings in UDTs

Post by Lost Zergling »

Yes, but not so sure in all cases. Suppose you built a hierarchical tree using 1-len strings to use it as index, in wich use case not using a terminal character and using memcopy instead (of c-close zstrings functions) may let you expect/hope more speed and less memory load. Thus, till your code is 'standalone' in that meaning that it does not rely onto specific c function nor WinAPI and is mastering its own functionnal perimeter, then this question, hypothetically, could be relevant.
dodicat
Posts: 7518
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Feature request: strings in UDTs

Post by dodicat »

For a udt comprising only of fixed length strings, then using fb's put will automatically add the chr(0) to a saved binary file, so retrieving is straight forward.
For a straight string text file, then an extra character must be added for each field to align properly.
If the text file comes from another source, and is not formatted with a character in the correct place then it won't load properly into the udt.
With an unformatted text file, a ubyte array will load the file OK.

Code: Select all

Type udt
      As String * 5 h
      As String * 7 g
      As String * 14 f
      declare sub show()
End Type

sub udt.show()
print "Loaded Fields by GET:"
print "'";h;"'"
print "'";g;"'"
print "'";f;"'"
end sub

Type ubyteread
      As Ubyte U(1 To 26)
      declare sub show()
End Type

sub ubyteread.show
      dim as udt z
      fb_memcopy(z.h,u(1),5)
      fb_memcopy(z.g,u(6),7)
      fb_memcopy(z.f,u(13),14)
      z.show
End sub

#macro load(file,datatype)
Scope
Dim As Long  f=Freefile
      If Open (file For Binary Access Read As #f)=0 Then
            #If Typeof(datatype)=Typeof(String)
           Dim As String text=String(Lof(f), 0)
           datatype=text
            #endif
            If Lof(f) > 0 Then Get #f, , datatype
            Close #f
      Else
            Print "Unable to load " + file:Sleep:End
      End If   
End Scope
#endmacro

#macro save(file,datatype)
Scope
      Dim As Long n=Freefile
      If Open (file For Binary Access Write As #n)=0 Then
            Put #n,,datatype
            Close
      Else
            Print "Unable to save " + file:Sleep:End
      End If
End Scope
#endmacro
            

#macro writestring(L)
#define setcolour  if L[n]=0 or L[n]=1 or L[n]=2 or L[n]=3 then color 2 else color 15
For n As Long=0 To Len(L)-1
      setcolour
      Print Iif(Color=2,Str(L[n]),Chr(L[n]));
Next
Color 15
#endmacro
'==============================================

dim as const string filepath=curdir+"\cheers.txt"

Dim As udt x=Type("Hello","Goodbye","Another string")
Dim As udt y
Dim As String L

Print "Fixed length strings"
save(filepath,x)
load(filepath,L)
writestring(L)
Print Tab (40);"= file content by put udt"
load(filepath,y)
y.show
Print


save(filepath,"Hello"+Chr(1)+"Goodbye"+Chr(2)+"Another string"+Chr(3))
load(filepath,L)
writestring(L)
Print Tab (40);"= Text file content by manually adding  characters"
load(filepath,y)
y.show
Print


save(filepath,"Hello"+"Goodbye"+"Another string")
load(filepath,L)
writestring(L)
Print Tab (40);"= Text file content raw"
load(filepath,y)
y.show
Print

Print "Ubyte array"
save(filepath,"Hello"+"Goodbye"+"Another string")
Dim As ubyteread u
load(filepath,u)
writestring(L)
Print Tab (40);"= Text file content raw"
u.show

Print "Press any key to end . . ."
Sleep
Kill filepath

 
angros47
Posts: 2013
Joined: Jun 21, 2005 19:04

Re: Feature request: strings in UDTs

Post by angros47 »

In the old QBASIC there was also another reason for not using the 0 terminator: since QBASIC didn't have the BYTE data type , the only way to store a single byte variable inside an UDT was to store it as STRING*1. Adding a zero after that would have made it useless. FreeBasic has the BYTE and UBYTE data types, so that trick is not needed.
marcov
Posts: 3357
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Feature request: strings in UDTs

Post by marcov »

angros47 wrote: Jun 29, 2022 22:21 C, on the other end, expects a zero as terminator. And most libraries use a C based interface, so adding a zero at the end of every string makes them compatible with external libraries. Without it, passing a string from an UDT to an external C function would likely cause a segmentation fault, or a data corruption.
Read a manual page for any -n- function and see:
strnlen linux manpage wrote: RETURN VALUE top

The strnlen() function returns strlen(s), if that is less than
maxlen, or maxlen if there is no null terminating ('\0') among
the first maxlen characters pointed to by s.
Relying solely on \0 is old fashioned
Post Reply