Extending Wstring and Zstring with UDTs

General discussion for topics related to the FreeBASIC project or its community.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Extending Wstring and Zstring with UDTs

Post by coderJeff »

We see the same problem with the USTRING aka DWSTR type where fbc's implicit STR conversions cause loss of wide characters.

José solved this issue in his wide string classes user code by overloading the dereference operator to get a WSTRING ptr (which I thought was a clever use of the operator).

Here is a minimal implementation with only the relavant parts (from DWSTR, I think), using a dynamic memory allocation. Can be tried with or without the 'extends wstring' to show the differences and I've included the output in the listing.

Code: Select all

type T '' extends wstring
	buffer as wstring ptr
	declare constructor( byref s as const wstring )
	declare destructor()
	declare operator cast() byref as const wstring
end type

constructor T( byref s as const wstring )
	buffer = callocate( (len(s) + 1) * sizeof(wstring) )
	*buffer = s
end constructor

destructor T()
	deallocate buffer
end destructor

operator T.cast() byref as const wstring
	operator = *buffer
end operator

operator * (byref s as const T) as wstring ptr
	operator = s.buffer
end operator

sub print_string( byref title as const string, byref x as const wstring, byval indent as integer = 0 )
	print left( title & space(20), 20 ) & ": "; space( indent*(1+sizeof(wstring)*2) );
	for i as integer = 0 to len(x) -1
		print hex( x[i], sizeof(wstring)*2 ); " ";
	next
	print
end sub

dim x as T = !"  \u3041\u3043\u3045\u3047\u3049  "
print_string( "x", x )

dim w as wstring * 50 = x
print_string( "w", w )

'' fbc wraps UDT's in a str() conversion for quirk string functions

print_string( "ltrim(x)", ltrim(x), 2 )
print_string( "rtrim(x)", rtrim(x) )
print_string( "trim(x)", trim(x), 2 )

'' double-deref gets the wstring data

print_string( "ltrim(**x)", ltrim(**x), 2 )
print_string( "rtrim(**x)", rtrim(**x) )
print_string( "trim(**x)", trim(**x), 2 )

/'
OUTPUT (without extends wstring):

x                   : 0020 0020 3041 3043 3045 3047 3049 0020 0020
w                   : 0020 0020 3041 3043 3045 3047 3049 0020 0020
ltrim(x)            :           003F 003F 003F 003F 003F 0020 0020
rtrim(x)            : 0020 0020 003F 003F 003F 003F 003F
trim(x)             :           003F 003F 003F 003F 003F
ltrim(**x)          :           3041 3043 3045 3047 3049 0020 0020
rtrim(**x)          : 0020 0020 3041 3043 3045 3047 3049
trim(**x)           :           3041 3043 3045 3047 3049

OUTPUT (with extends wstring):

x                   : 0020 0020 3041 3043 3045 3047 3049 0020 0020
w                   : 0020 0020 3041 3043 3045 3047 3049 0020 0020
ltrim(x)            :           3041 3043 3045 3047 3049 0020 0020
rtrim(x)            : 0020 0020 3041 3043 3045 3047 3049
trim(x)             :           3041 3043 3045 3047 3049
ltrim(**x)          :           3041 3043 3045 3047 3049 0020 0020
rtrim(**x)          : 0020 0020 3041 3043 3045 3047 3049
trim(**x)           :           3041 3043 3045 3047 3049
'/
fyi, I ran this on a Win PC, Linux should output 4 byte values in hex, though I didn't try it to confirm.
Josep Roca
Posts: 564
Joined: Sep 27, 2016 18:20
Location: Valencia, Spain

Re: Extending Wstring and Zstring with UDTs

Post by Josep Roca »

Excellent work. In my CWSTR class, I only have needed to change:

Code: Select all

#if __FB_VERSION__ < "1.07.0"
TYPE CWSTR
#else
TYPE CWSTR EXTENDS WSTRING
#endif
fxm
Moderator
Posts: 12083
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Extending Wstring and Zstring with UDTs

Post by fxm »

fxm wrote:For 'Type Ustring Extends Zstring', the added features allow to now support in addition:
- 'Strptr'
- 'Lset/Rset'
- 'Select Case'
which were the only incompatible without 'Extends Zstring'.

To overload the unary operator Len, since the member data of Ustring are usually private, I found nothing better than this:

Code: Select all

Operator Len ( Byref u As Ustring ) As Integer
  Operator = Len( Type<String>( u ) )
End Operator
Example:

Code: Select all

Type vZstring Extends Zstring
  Private:
    Dim As String s
  Public:
    Declare Constructor ()
    Declare Constructor (Byval pz As Zstring Ptr)
    Declare Operator Let (Byval pz As Zstring Ptr)
    Declare Operator Cast () Byref As Zstring
    Declare Operator [] (Byval index As Integer) Byref As Ubyte
    Declare Destructor ()
End Type

Constructor vZstring ()
  This.s = ""
End Constructor

Constructor vZstring (Byval pz As Zstring Ptr)
  This.s = *pz
End Constructor

Operator vZstring.Let (Byval pz As Zstring Ptr)
  This.s = *pz
End Operator

Operator vZstring.Cast () Byref As Zstring
  Return *Strptr(This.s)
End Operator

Operator vZstring.[] (Byval index As Integer) Byref As Ubyte
  Return This.s[index]
End Operator

Destructor vZstring ()
  This.s = ""
End Destructor

Operator Len (Byref v As vZstring) As Integer
  Return Len(Type<String>(v))        '' found nothing better than this ('vZstring.s' being private)
End Operator


Dim As vZstring v = "FreeBASIC"
Print "'" & v & "'",, Len(v)

Dim As Zstring * 256 z
z = *Strptr(v)                       '' 'error 24: Invalid data types' without 'Extends Zstring'
Print "'" & z & "'",, Len(z)

v &= Space(3)
Print "'" & v & "'", Len(v)
Rset v, "FreeBASIC"                  '' 'error 24: Invalid data types' without 'Extends Zstring'
Print "'" & v & "'", Len(v)

Select Case v                        '' 'error 24: Invalid data types' without 'Extends Zstring'
Case Type<vZstring>("FreeBASIC   ")
  Print "Left justified"
Case Type<vZstring>("   FreeBASIC")
  Print "Right justified"
End Select

Sleep
It remains to solve the compatibility (of 'Extends Zstring') with:
- Val/Valint/Vallng/Valuint/Valunlg
- Left/Right
presently:
error 98: Ambiguous call to overloaded function
Last edited by fxm on Jun 24, 2019 12:28, edited 5 times in total.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Extending Wstring and Zstring with UDTs

Post by coderJeff »

Thanks, fxm.

To avoid some complexity in the wiki topics, I was think of adding pages for
KeyPgExtendsWstring
KeyPgExtendsZstring
to give a separate treatment of the usage.

Then only linking (brief mention) from:
Type
Extends
Wstring
Zstring
fxm
Moderator
Posts: 12083
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Extending Wstring and Zstring with UDTs

Post by fxm »

1) For EXTENDS ZSTRING (KeyPgExtendsZstring), I can initiate the documentation page as the draft proposed below:
  • EXTENDS ZSTRING documentation page wrote:
    EXTENDS ZSTRING

    Specifies a type which inherits a Zstring behavior

    Syntax
    • Type|Union typename Extends Zstring [, base_typename]
      ...
      End Type|Union
    Description
    • Extends Zstring declares typename to inherit properties and behaviors of a Zstring. Purpose is to allow users to create custom string types (with i.e. dynamic memory management) that can integrate well in to existing fbc compiler built ins (good interoperability with fbc's Zstring type).

      This declaration of such a UDT with a suitable Cast operator will instruct compiler to convert the UDT to a Zstring (in addition, other suitable operators as Let, [] (Pointer index), Len, ..., can be also declared).

      Zstring behaviour can be inherited directly, or indirectly and singly from a base-type.
      Zstring behaviour can be inherited by a UDT also extending base_typename (a kind of pseudo multiple-inheritance).

      Note: By declaring a type (directly or indirectly) as Extends Zstring, this promotes it fully compatible even with Strptr/Sadd, Lset/Rset, and Select Case, but currently not yet with the built in functions Val/Valint/Vallng/Valuint/Valunlg and Left/Right.
    Example
    • Code: Select all

      Type myZstring Extends Zstring
        Public:
          Declare Constructor (Byval pz As Zstring Ptr = 0)
          Declare Operator Cast () Byref As Const Zstring
          Declare Operator Let (Byval pz As Zstring Ptr)
        Private:
          Dim As String s
      End Type
      
      Constructor myZstring (Byval pz As Zstring Ptr = 0)
        This.s = *pz
      End Constructor
      
      Operator myZstring.Cast () Byref As Const Zstring
        Return *Strptr(This.s)
      End Operator
      
      Operator myZstring.Let (Byval pz As Zstring Ptr)
        This.s = *pz
      End Operator
      
      Dim As myZstring z = "FreeBASIC"
      Print "'" & z & "'"
      
      z &= " compiler"
      Print "'" & z & "'"
      
      Sleep
      

      Code: Select all

      Type vZstring Extends Zstring
        Public:
          Declare Constructor (Byval pz As Zstring Ptr = 0)
          Declare Operator Cast () Byref As Zstring
          Declare Operator Let (Byval pz As Zstring Ptr)
          Declare Operator [] (Byval index As Integer) Byref As Ubyte
          Declare Destructor ()
        Private:
          Dim As Zstring Ptr p
          Dim As Uinteger l
      End Type
      
      Constructor vZstring (Byval pz As Zstring Ptr = 0)
        This.l = Len(*pz)
        This.p = Callocate(This.l + 1, SizeOf(ZString))
        *This.p = *pz
      End Constructor
      
      Operator vZstring.Cast () Byref As Zstring
        Return *This.p
      End Operator
      
      Operator vZstring.Let (Byval pz As Zstring Ptr)
        If This.l < Len(*pz) Then
          Deallocate(This.p)
          This.l = Len(*pz)
          This.p = Callocate(This.l + 1, SizeOf(ZString))
        End If
        *This.p = *pz
      End Operator
      
      Operator vZstring.[] (Byval index As Integer) Byref As Ubyte
        Return This.p[index]
      End Operator
      
      Destructor vZstring ()
        Deallocate(This.p)
      End Destructor
      
      Operator Len (Byref v As vZstring) As Integer
        Return Len(Type<String>(v))        '' found nothing better than this ('vZstring.s' being private)
      End Operator
      
      Dim As vZstring v = "FreeBASIC"
      Print "'" & v & "'", Len(v)
      
      Dim As Zstring * 256 z
      z = *Strptr(v)                       '' 'error 24: Invalid data types' without 'Extends Zstring'
      Print "'" & z & "'", Len(z)
      
      v &= Space(2)
      Print "'" & v & "'", Len(v)
      Rset v, "FreeBASIC"                  '' 'error 24: Invalid data types' without 'Extends Zstring'
      Print "'" & v & "'", Len(v)          ''     ('Cast' must return a modifiable reference)
      
      Select Case v                        '' 'error 24: Invalid data types' without 'Extends Zstring'
      Case Type<vZstring>(Trim(v) & "  ")
        Print "Left justified"
      Case Type<vZstring>("  " & Trim(v))
        Print "Right justified"
      End Select
      
      v[0] = Asc("-")
      Print "'" & v & "'", Len(v)
      
      'Print "'" & Right(v, 5) & "'"         '' 'Right' does not yet support types with 'Extends Zstring'
      Print "'" & Right(Type<String>(v), 5) & "'"  '' workaround
      
      Sleep
      
    Dialect differences
    • Not available in the -lang qb dialect unless referenced with the alias __Extends __Zstring, but unusable because no member procedure is allowed in this dialect.
    Differences from QB
    • New to FreeBASIC
    See also
    • - Type
      - Union
      - Extends
      - Zstring
      - Extends Wstring
    • (new documentation page to be finalized)
2) For EXTENDS WSTRING (KeyPgExtendsWstring),I can just initiate the template because I do not have the knowledge or even the practice of Wstring to write more information dedicated to such a string:
  • EXTENDS WSTRING documentation page wrote:
    EXTENDS WSTRING

    Specifies a type which inherits a Wstring behavior

    Syntax
    • Type|Union typename Extends Wstring [, base_typename]
      ...
      End Type|Union
    Description
    • Extends Wstring declares typename to inherit properties and behaviors of a Wstring. Purpose is to allow users to create custom string types (with i.e. dynamic memory management) that can integrate well in to existing fbc compiler built ins (good interoperability with fbc's Wstring type).

      This declaration of such a UDT with a suitable Cast operator will instruct compiler to convert the UDT to a Wstring (in addition, other suitable operators as Let, [] (Pointer index), Len, ..., can be also declared).

      Wstring behaviour can be inherited directly, or indirectly and singly from a base-type.
      Wstring behaviour can be inherited by a UDT also extending a base_typename (a kind of pseudo multiple-inheritance).
    Example Dialect differences
    • Not available in the -lang qb dialect unless referenced with the alias __Extends __Wstring, but unusable because no member procedure is allowed in this dialect.
    Differences from QB
    • New to FreeBASIC
    See also
    • - Type
      - Union
      - Extends
      - Wstring
      - Extends Zstring
    • (new documentation page to be completed)
Last edited by fxm on Jun 27, 2019 8:16, edited 7 times in total.
fxm
Moderator
Posts: 12083
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Extending Wstring and Zstring with UDTs

Post by fxm »

Done (see above for the two new documentation pages)

- KeyPgExtendsZstring → fxm [new page to be finalized]
- KeyPgExtendsWstring → fxm [new page to be completed]

- CatPgFullIndex → fxm [added 'Extends Zstring' and 'Extends Wstring' keywords]
- CatPgFunctIndex → fxm [added 'Extends Zstring' and 'Extends Wstring' keywords]
- CatPgUserDefTypes → fxm [added 'Extends Zstring' and 'Extends Wstring' declarations]
- KeyPgType → fxm [added 'Extends Zstring' and 'Extends Wstring' links]
- KeyPgExtends → fxm [added 'Extends Zstring' and 'Extends Wstring' links]
- KeyPgObject → fxm [added 'Extends Zstring' and 'Extends Wstring' links]
- KeyPgBaseInit → fxm [added 'Extends Zstring' and 'Extends Wstring' links]
- KeyPgBase → fxm [added 'Extends Zstring' and 'Extends Wstring' links]
- KeyPgOpIs → fxm [added 'Extends Zstring' and 'Extends Wstring' links]
- KeyPgVirtual → fxm [added 'Extends Zstring' and 'Extends Wstring' links]
- KeyPgAbstract → fxm [added 'Extends Zstring' and 'Extends Wstring' links]
- KeyPgZstring → fxm [added 'Extends Zstring' link]
- KeyPgWstring → fxm [added 'Extends Wstring' link]
- KeyPgImplements → fxm [added 'Extends Zstring' and 'Extends Wstring' links]
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: Extending Wstring and Zstring with UDTs

Post by D.J.Peters »

@fxm well done the examples are good also.

Joshy
D.J.Peters
Posts: 8586
Joined: May 28, 2005 3:28
Contact:

Re: Extending Wstring and Zstring with UDTs

Post by D.J.Peters »

Code: Select all

#include "crt.bi"

extern "C"
#ifdef __FB_LINUX__
  declare function AllocatedSize alias "malloc_usable_size" (byval p as any ptr) as size_t
#elseif defined(__FB_WIN32__)
  declare function AllocatedSize alias "_msize" (byval p as any ptr) as size_t   
#else
  #error 666: Build trarget must be Windows or Linux !
#endif
end extern
'
' main
'
dim as zstring * 26 zFixed = "FreeBASIC"
dim as zstring ptr zDynamic = callocate(26)
*zDynamic = "FreeBASIC"
Print "zFixed   len      : " & len(zFixed)
Print "zFixed   sizeof   : " & sizeof(zFixed)
Print "zDynamic len      : " & len(zDynamic)
Print "zDynamic sizeof   : " & sizeof(zDynamic)
Print "zDynamic allocated: " & AllocatedSize(zDynamic)

Sleep
fxm
Moderator
Posts: 12083
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Extending Wstring and Zstring with UDTs

Post by fxm »

Rather?
Print "zDynamic len : " & len(*zDynamic)
Print "zDynamic sizeof : " & sizeof(*zDynamic)


or:

Code: Select all

#include "crt.bi"

extern "C"
#ifdef __FB_LINUX__
  declare function AllocatedSize alias "malloc_usable_size" (byval p as any ptr) as size_t
#elseif defined(__FB_WIN32__)
  declare function AllocatedSize alias "_msize" (byval p as any ptr) as size_t   
#else
  #error 666: Build trarget must be Windows or Linux !
#endif
end extern
'
' main
'
dim as zstring * 26 zFixed = "FreeBASIC"
dim byref as zstring zDynamic = *cptr(zstring ptr, callocate(26))
zDynamic = "FreeBASIC"
Print "zFixed   len      : " & len(zFixed)
Print "zFixed   sizeof   : " & sizeof(zFixed)
Print "zDynamic len      : " & len(zDynamic)
Print "zDynamic sizeof   : " & sizeof(zDynamic)
Print "zDynamic allocated: " & AllocatedSize(@zDynamic)

Sleep
deallocate(@zDynamic)
Juergen Kuehlwein
Posts: 284
Joined: Mar 07, 2018 13:59
Location: Germany

Re: Extending Wstring and Zstring with UDTs

Post by Juergen Kuehlwein »

I removed the original Ustring pull request and made two new ones. The first one only for adding Ustring, the second one for adding Ustring and applying white space changes for consistent indentation. All of it in separate commits as desired.

JK
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Extending Wstring and Zstring with UDTs

Post by coderJeff »

fxm wrote:Done (see above for the two new documentation pages)
Thanks fxm, that's awesome. Yeah, z|wstring are basically the same, except there is no nice way to generate output for wstring that would be consistent on every platform for the examples.

Something I though was interesting...

With the UDT, you can build an overloaded operator that takes byval as zstring ptr parameter, for example:

Code: Select all

Declare Operator Let ( byval pz As ZString ptr )
But, for behaviour that is more consistent with string, would probably use byref as const zstring.

Code: Select all

Declare Operator Let ( byref as const zstring )
Like a string can't be assigned from pointer. Just something I though was interesting. The user has a little flexibility to build their own rules.
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Extending Wstring and Zstring with UDTs

Post by coderJeff »

Juergen Kuehlwein wrote:I removed the original Ustring pull request and made two new ones. The first one only for adding Ustring, the second one for adding Ustring and applying white space changes for consistent indentation. All of it in separate commits as desired.
Thank you, I see them.
add ustring.bi #155
White space #156

I can work with that. I did a first review of ustring.bi.
fxm
Moderator
Posts: 12083
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Extending Wstring and Zstring with UDTs

Post by fxm »

coderJeff wrote: Something I though was interesting...

With the UDT, you can build an overloaded operator that takes byval as zstring ptr parameter, for example:

Code: Select all

Declare Operator Let ( byval pz As ZString ptr )
But, for behaviour that is more consistent with string, would probably use byref as const zstring.

Code: Select all

Declare Operator Let ( byref as const zstring )
Like a string can't be assigned from pointer. Just something I though was interesting. The user has a little flexibility to build their own rules.
Does this mean that you prefer that I modify the Let operator declaration in one of my 2 examples of 'Extends Zstring' to present both possibilities?
Or altogether modify the 2 examples?
coderJeff
Site Admin
Posts: 4313
Joined: Nov 04, 2005 14:23
Location: Ontario, Canada
Contact:

Re: Extending Wstring and Zstring with UDTs

Post by coderJeff »

fxm wrote:Does this mean that you prefer that I modify the Let operator declaration in one of my 2 examples of 'Extends Zstring' to present both possibilities?
Or altogether modify the 2 examples?
Yeah, that sounds good, to modify 1 to show both possibilities.

Personally, I would probably use the 'byref' variant more often for initializing, assignment, etc, as it would be unlikely to be passed in a null pointer. But then, initializing and assignment from pointer might be a nice convenience depending on the type of program. I don't know think I can to say if one is better than the other, just different.
fxm
Moderator
Posts: 12083
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Extending Wstring and Zstring with UDTs

Post by fxm »

coderJeff wrote: Personally, I would probably use the 'byref' variant more often for initializing, assignment, etc, as it would be unlikely to be passed in a null pointer.
In my codes, I do not check the null Zstring pointers (pz) because their dereferencing (*pz) returns an empty (and valid) Zstring.
Post Reply