Feature request: strings in UDTs

[fzabkar]

I notice that there is a reference to the possibility that fixed length strings inside Types may one day not require a 0x00 terminator. Is this still on the to-do list?

[angros47]

At this point, I doubt: changing that would break too much existing FreeBasic code.

By the way, a ZSTRING doesn't have that issue: its length is exactly as declared, inside a type

[fzabkar]

I propose to add a new parameter along the lines of the FIELD parameter. This parameter, if present, would tell the compiler to treat the strings inside the UDT as normal FB strings or zstrings. I can't see how this would break any code.

[angros47]

What is the point of using a FIELD parameter when you can already use ZSTRING? It's literally just one character to add.

Maybe in QB mode the strings inside a TYPE should be treated as ZSTRINGS, just to make UDT compatible with QB (in case one is porting a program that was supposed to read a file with the commands GET and PUT)

[fxm]

By the way, a ZSTRING doesn't have that issue: its length is exactly as declared, inside a type

Yes, but the maximum number of useful characters for the user is 'length - 1' (because of the mandatory terminal character).

[fxm]

In the following two cases, the terminal character is always mandatory:

• Dim z As Zstring * N '' sizeof(z) = N, but only N-1 useful characters at most for the user
  
  Dim s As String * N '' N useful characters at most for the user, but sizeof(s) = N+1

[fzabkar]

Thanks. It looks like this terminal character is going to be a continuing source of annoyance for me. :-(

[Lost Zergling]

Hello. Annoying.. Is it because you need to filter Chr(0) from zstrings, because you try to optimize memory load or other reason(s)?

[fzabkar]

It's annoying because it's counterintuitive and counterproductive. If I specify that I want to read or write a string consisting of n characters, then I don't want to think about anything else other than those n characters.

I do a lot of reverse engineering of firmware. This firmware often contains headers with mixed text and binary data. There are usually no terminators in these headers.

I really like the way that FB handles strings outside of Types. It's just too bad that I have to readjust my brain to handle the latter.

[Lost Zergling]

Perhaps your requirement is a custom function 'valueof' wich would accept a (mandatory) lenght parameter ?
*(myzstrptr, length). This could be achieved using ptrs and perhaps a new or overloaded operator.
(reading n＋1, storing, replacing by Chr(0), reading*, restoring n+1 initial value)
Then handling zstring ptrs (or zstring fixed len) instead of strings into types, allocating n and not n＋1 len ?
strings are designed according to the spirit of language meaning they can be handled no matter the lenght.
I think the Chr(0) is for compliance with c-like zstrings, so it can support Valueof, notably.
ps : I did have a thought.around this question because I tried to optimize as much as possible memory load in lzle types with fixed len strings (tagmode 1)
Finally I decided not to investigate further, inducing too much complexity and anticipating speed slowdown.

[fzabkar]

Thanks.

The documentation for Type has an example using pointers and ubytes. I think I will use that method if I need to, eg when the string has a "non-binary" length.

[Lost Zergling]

ubyte ptr onto a zstring area. Perhaps check memcopy or FB_memcopy also if relevant, shall be much faster.

[fxm]

Perhaps check memcopy or FB_memcopy also if relevant, shall be much faster.

Yes (only since fbc 1.08.0 for using 'FB_memcopy()'):

Code: Select all

'' Using a ubyte array, we need an auxiliary function to convert it to a string
Function ub2str( ub() As UByte ) As String
    Dim As String res = Space(UBound(ub) - LBound(ub) + 1)
    FB_memcopy(res[0], ub(LBound(ub)), UBound(ub) - LBound(ub) + 1)
    Function = res
End Function

[angros47]

By the way, a ZSTRING doesn't have that issue: its length is exactly as declared, inside a type

Yes, but the maximum number of useful characters for the user is 'length - 1' (because of the mandatory terminal character).

I made some tests: it is actually possible to use all the characters, removing the terminal zero, if the string is accessed through a pointer:

Code: Select all

type foo
	a as zstring*5
	b as byte
end type

dim test as foo
dim s as zstring ptr=@test.a
*s="12345"

print test.a

Of course, this will cause some issues, because the assignation will still add a zero, that will corrupt the next field:

Code: Select all

test.b=12
*s="12345"
print test.b

will give 0 as output (because the terminal zero was written in the next location, where "b" is supposed to be).
If we do:

Code: Select all

*s="12345"
test.b=65

We will get "12345A" as output (since the PRINT command didn't meet a 0, so it kept going on, and the ASCII character for 65 is "A")

To be able to set a string without the final zero, we must use MID (that is supposed to overwrite only part of a string, so it doesn't try to truncate it). So, either

Code: Select all

mid(*s,1)="12345"

or

Code: Select all

mid(test.a,1)="12345"

That are perfectly equivalent.

But if we do that, we will see that the string will remain empty: in fact, MID doesn't write past the ending of a string, so if the string is empty (and the first character is a zero) it stops immediately. We must first prepare the string with:

Code: Select all

*s=space(5)

to replace the zeroes with space.

So:

Code: Select all

type foo
	a as zstring*5
	b as byte
end type

dim test as foo
dim s as zstring ptr=@test.a
*s=space(5)

test.b=65
mid(test.a,1)="12345"

test.b will be 65, it won't be overwritten, and test.a will be "12345", so all five characters are used. But there is still one problem: if we print test.a, we will get "12345A", because we have no zero to tell we reached the end of the string.

So, to read test.a, we'll have to use:

Code: Select all

print left(test.a,5)

And done! We are able to store a string inside an UDT without wasting one byte for the terminal zero. Is it worth the effort?

[fxm]

My proposal:
An advanced UDT of size = 5 bytes, and allowing up to 5 useful zstring characters

Code: Select all

Type myZstring5 Extends Zstring
        Declare Constructor(Byref z As Const Zstring = "")
        Declare Operator Let(Byref z As Const Zstring)
        Declare Operator Cast() Byref As Const Zstring
        Declare Operator [](Byval i As Integer) Byref As Ubyte
    Private:
        Const N = 5
        Dim As Ubyte u(0 To myZstring5.N - 1)
End Type

Constructor myZstring5(Byref z As Const Zstring = "")
    This = z
End Constructor

Operator myZstring5.Let(Byref z As Const Zstring)
    FB_memcopyclear(This.u(0), myZstring5.N, z[0], Len(z))
End Operator

Operator myZstring5.Cast() Byref As Const Zstring
    Static As Zstring * (myZstring5.N + 1) z
    FB_memcopyclear(z[0], myZstring5.N + 1, This.u(0), myZstring5.N)
    Operator = z
End Operator

Operator myZstring5.[](Byval i As Integer) Byref As Ubyte
    Operator = This.u(i)
End Operator

Operator Len(Byref mz5 As myZstring5) As Uinteger
    Operator = Len(Str(mz5))
End Operator


Dim As myZstring5 mz5 = "12345"
Print mz5, "Len() = " & Len(mz5), "Sizeof() = " & Sizeof(mz5)
mz5 = "123"
Print mz5, "Len() = " & Len(mz5), "Sizeof() = " & Sizeof(mz5)
mz5 = "abcdefg"
Print mz5, "Len() = " & Len(mz5), "Sizeof() = " & Sizeof(mz5)
mz5[4] = Asc("!")
Print mz5, "Len() = " & Len(mz5), "Sizeof() = " & Sizeof(mz5)
Print Left(mz5, 3)
Print "12345" & mz5

Sleep