LZLE List Engine with user friendly powerfull syntax - BETA 0.997a
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax
Link to Tutorial : viewtopic.php?f=9&t=26551&p=245161#p245161
I think I'd appreciate some more returns, tests, remarks, questions and so on. Every subjects open.
About licence : coded in pointers, mainly need pointers and datatypes to work. My intention is that the idea benefits the use and the spirit of Basic langage, not benefit to other language () and I find it legitimate that an idea can be protected for what it was made of.
I think I'd appreciate some more returns, tests, remarks, questions and so on. Every subjects open.
About licence : coded in pointers, mainly need pointers and datatypes to work. My intention is that the idea benefits the use and the spirit of Basic langage, not benefit to other language () and I find it legitimate that an idea can be protected for what it was made of.
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax
Minor fixes
#1 AllOf : Added "If pFirstNode=pFIRSTFIRSTNode Then : If pLastNode<>pWhyteMove Then : this.Root : End If : End If"
to automate the need to call Root after BlindTag in Main Flat List
#2 NodeFlat : Added "If pFirstNode->pBranch=pFlatRoot Then : Return 0 : End If"
to prevent an element already reduce to be reduced twice
#3 NodeFlat : Two Replaced "this.NodeFlat : this.NodeRecycle2" by
"If pNode->Tag(1)="" Or pNode->Tag(1)=LIST_DEL Then : this.NodeFlat : this.NodeRecycle2 : End If"
to stop recursive reduce on parents which are Keys when Tracking
Evolutions
*1 NodeFlat : Added two "Or bPickReduce=1" and NFrecursive Property to enable/force recursive NodeFlat
behaviour outside the tree index parsing scope.
*2 Flat control : Added fMove(Integer) property on flat list : allowing user to code its own sorting features.
#1 AllOf : Added "If pFirstNode=pFIRSTFIRSTNode Then : If pLastNode<>pWhyteMove Then : this.Root : End If : End If"
to automate the need to call Root after BlindTag in Main Flat List
#2 NodeFlat : Added "If pFirstNode->pBranch=pFlatRoot Then : Return 0 : End If"
to prevent an element already reduce to be reduced twice
#3 NodeFlat : Two Replaced "this.NodeFlat : this.NodeRecycle2" by
"If pNode->Tag(1)="" Or pNode->Tag(1)=LIST_DEL Then : this.NodeFlat : this.NodeRecycle2 : End If"
to stop recursive reduce on parents which are Keys when Tracking
Evolutions
*1 NodeFlat : Added two "Or bPickReduce=1" and NFrecursive Property to enable/force recursive NodeFlat
behaviour outside the tree index parsing scope.
*2 Flat control : Added fMove(Integer) property on flat list : allowing user to code its own sorting features.
Re: LZLE List Engine with user friendly powerfull syntax
FYI, the tutorial is currently at the *wrong place*, it belongs here (in project itself).
Reason: even if not explicitly stated, *Documentation* referres to: FB-Documentation.
(not anybody's doc./tutorial/whatever)
Reason: even if not explicitly stated, *Documentation* referres to: FB-Documentation.
(not anybody's doc./tutorial/whatever)
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax
Hello,
Ok, I take note of it. This mistake was not wanted. I'll move the documentation subject to a new "General topic" and I'll update the links. Topic in documentation will be outdated. I rather prefer a separate topic for project itself and documentation because for me these are different topics to talk about : one (the project) is dedicated to bugs, technical or low level issues, the other one (documentation) is dedicated to how to use, code exemples, and conceptual remarks/requests. I was wondering if I should put this contribution in projects or in libraries, nevertheless as ambition is to propose an implementation overlay with a set of instructions, it seems to me more in the spirit to see it as a project. Welcome using it for some testing in 64 bits, or any suggestions.
Ok, I take note of it. This mistake was not wanted. I'll move the documentation subject to a new "General topic" and I'll update the links. Topic in documentation will be outdated. I rather prefer a separate topic for project itself and documentation because for me these are different topics to talk about : one (the project) is dedicated to bugs, technical or low level issues, the other one (documentation) is dedicated to how to use, code exemples, and conceptual remarks/requests. I was wondering if I should put this contribution in projects or in libraries, nevertheless as ambition is to propose an implementation overlay with a set of instructions, it seems to me more in the spirit to see it as a project. Welcome using it for some testing in 64 bits, or any suggestions.
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax
Evolution
New Property "HashSort" : MyList.HashSort(1) => Every MyList.HashTag("Key") will be sorted (ascending)
For sorting : Just reduce list to protected "flat" list using NodeFlat(1) in a loop, then MyList.HashSort(1) and RestoreHash in a loop will restore sorted list.
Can be combined with other features like Snatching, RevParsing and so on. Few modifications, mainly using reduce & rollbackreduce to sort.
Code exemple :
New Property "HashSort" : MyList.HashSort(1) => Every MyList.HashTag("Key") will be sorted (ascending)
For sorting : Just reduce list to protected "flat" list using NodeFlat(1) in a loop, then MyList.HashSort(1) and RestoreHash in a loop will restore sorted list.
Can be combined with other features like Snatching, RevParsing and so on. Few modifications, mainly using reduce & rollbackreduce to sort.
Code exemple :
Code: Select all
#Include once "D:\Basic\LZListsEngine20180301.bi"
Dim MaListe As List
MaListe.BranchCountDown(1)
MaListe.HashSort(1) ' ***************************Sorting here
Dim u as ulongint : Dim i as ulongint
MaListe.HashTag("a") : MaListe.Val("2")
Print "1"
'For u=1100000000 To 1100000030
For u=1100000030 To 1100000000 step -1
MaListe.HashTag(str(u)) : MaListe.Val(str(u-1100000000))
Next u
For u=28 To 20 Step -1
MaListe.HashTag(str(u)) : MaListe.Val(str(u))
Next u
Print "node count=" & Str(MaListe.Nodecount) & " count=" & Str(MaListe.count)
Print "++"
MaListe.First
MaListe.HashTag("8") : MaListe.Val("55")
MaListe.Root
MaListe.RootNode
Print "* " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count)
'While MaListe.KeyStep=1
While MaListe.HashStep=1
Print "* " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count)
Wend
Print "---------------------"
sleep
Print "*Count=" & MaListe.Count
MaListe.Root
Print "#Count=" & MaListe.Count
While MaListe.HashStep=1
If (Cint(MaListe.Val) > 22 And Cint(MaListe.Val) <25) Or Cint(MaListe.Val) =30 Then
MaListe.NodeFlat
ElseIf Cint(MaListe.Val) =0 Then
MaListe.NodeFlat
End If
Wend
MaListe.Root
Print "Count=" & MaListe.Count
sleep
MaListe.Root
MaListe.RootNode
While MaListe.HashStep=1
Print "% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) '& " suivant=" & MaListe.NextNode ': sleep
Wend
Print "F 01"
sleep
MaListe.Root
For i=1 To MaListe.AllOf : MaListe.BlindStep
Print "+ " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count)
Next i
sleep
Print "F 02"
sleep
MaListe.Root
While MaListe.HashStep=1
Print "% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) '& " suivant=" & MaListe.NextNode ': sleep
Wend
Print "F 03"
sleep
MaListe.Root
MaListe.EndNode
Print "# EndNode=" & MaListe.Tag
Print "root1"
MaListe.Root
Print "root2"
MaListe.FlatStack :
Print "--------------------"
sleep
MaListe.Root
While MaListe.HashStep=1
Print "~ " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
Wend
Print "--"
MaListe.Last
Print MaListe.HashTag & " =Last ?"
MaListe.Root
For u=1100000040 To 1100000050
MaListe.HashTag(str(u)) : MaListe.Val(str(u-1100000000))
Next u
MaListe.Root
While MaListe.HashStep=1
Print "$ " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
Wend
MaListe.HashTag("52000067") : MaListe.Val("9")
sleep
MaListe.Root
MaListe.HashTag("32000067") : MaListe.Val("98")
MaListe.HashTag("62000067") : MaListe.Val("5")
MaListe.Root
While MaListe.HashStep=1
Print "# " & MaListe.HashTag & " - " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
Wend
MaListe.EndNode
Print MaListe.HashTag & " =EndNode ? " & MaListe.Tag
MaListe.Last
Print MaListe.HashTag & " =Last ? " & MaListe.Tag :
sleep
MaListe.Root
MaListe.EndNode : 'MaListe.Val("??")
Print MaListe.HashTag & " =EndNode ?" :
MaListe.Last
Print MaListe.HashTag & " =Last ?" : sleep
MaListe.HashTag("9876") : MaListe.Val("9")
MaListe.Root
While MaListe.HashStep=1
Print "##? " & MaListe.HashTag & " - " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
Wend
Print "Fin 1"
sleep
MaListe.Root
While MaListe.HashStep=1
If Cint(MaListe.Val) < 18 Or ( Cint(MaListe.Val)<49 And Cint(MaListe.Val)>40 ) Then ' Or ( Cint(MaListe.Val)<50 And Cint(MaListe.Val)>40 )
' Print "Flating " & MaListe.HashTag ': sleep
MaListe.NodeFlat
Else
' Print "Parsing " & MaListe.HashTag ': sleep
' MaListe.NodeRecycle
End If
Wend
'MaListe.NodeRecycle
print "FIN"
'sleep
sleep :' system
' MaListe.HashTag("1100000000")
'If 1=0 Then
'MaListe.HashTag("32000067")
MaListe.HashTag("1100000030")
MaListe.BlindStep(-1) : Print "30->" & MaListe.Tag
MaListe.BlindStep(1) : Print "30->" & MaListe.HashTag
MaListe.Last : Print "30 Last->" & MaListe.HashTag
MaListe.HashTag("110000004") : Print "110000004 is branch=" & MaListe.BranchCount
MaListe.HashTag("1100000040")
MaListe.BlindStep(-1) : Print "40->" & MaListe.Tag
MaListe.BlindStep(1) : Print "40->" & MaListe.HashTag
MaListe.Last : Print "40 Last->" & MaListe.HashTag
sleep
MaListe.HashTag("1100000041")
MaListe.Root : MaListe.First
Print "++**++"
While MaListe.HashStep=1
Print "%% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
Wend
Print "Pt A01" : sleep
'MaListe.Root
MaListe.HashTag("110000001")
Print "%" & MaListe.HashTag & "%"
Print "Pt A02" : sleep
Print "~01"
MaListe.HashTag("1100000010")
MaListe.Root
Print "++**++"
While MaListe.HashStep=1
Print "%-% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
Wend
Print "~02"
sleep
Print "===================="
MaListe.Root
MaListe.FlatStack ': MaListe.Branch
Print "Count=" & MaListe.Count & " AllOf=" & MaListe.AllOf
Print "~03"
MaListe.RHMethod(-1)
For i=1 To MaListe.AllOf : MaListe.BlindStep
Print "restoring " & MaListe.Tag ': sleep
MaListe.RestoreHash
Next i
' MaListe.NodeRecycle
Print "--------------------"
Print "Restored "
sleep
MaListe.Root
MaListe.RootNode
MaListe.Val ("totor")
Print "*** " & MaListe.HashTag & " tag=" & MaListe.Tag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
While MaListe.HashStep=1
' Print "% " & MaListe.Tag & " = " & MaListe.Val
Print "** " & MaListe.HashTag & " tag=" & MaListe.Tag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode ' : sleep
Wend
Print "--------------------"
sleep
'system
'MaListe.Root
'MaListe.BlindTag("")
Print "* flating"
MaListe.Root
'While MaListe.HashStep=1
' Print "* flating -> " & MaListe.HashTag ' : sleep
' MaListe.NodeFlat
'Wend
'sleep
i=0
'dim i as integer=0
MaListe.Root
Print "count=" & MaListe.Count : sleep
Print MaListe.Tag
MaListe.Root
Print "# " & MaListe.Tag : MaListe.BlindStep
Print "# " & MaListe.Tag : MaListe.BlindStep
Print "# " & MaListe.Tag : MaListe.BlindStep
Print "# " & MaListe.Tag : MaListe.BlindStep
'Print "**********"
'MaListe.BlindStep : Print MaListe.Tag
Print "**********"
Print "value=" & str(u) & " / " & str(i)
Print "FlatCount=" & MaListe.Count
Print "GarbageCount=" & MaListe.GarbageCount
Print "NodeCount=" & MaListe.NodeCount
Print "**********"
sleep
u=1
MaListe.Root
MaListe.RootNode
Print "% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ': sleep
While MaListe.HashStep=1
' Print "% " & MaListe.Tag & " = " & MaListe.Val
Print "% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ': sleep
u+=1
Wend
sleep
Print "% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ': sleep
' MaListe.BlindStep
'Print "?= " & MaListe.tag
'MaListe.BlindStep
'Print "?= " & MaListe.tag
'MaListe.BlindStep
'Print "?= " & MaListe.tag
Print "Node visibles parcourables = " & str(u) & " Garbage=" & MaListe.GarbageCount '& " Flat=" & MaListe.FlatCount
sleep
' Print "Node visibles parcourables = " & str(u) & " Garbage=" & MaListe.GarbageCount '& " Flat=" & MaListe.FlatCount
MaListe.First 'GarbagePtr
Print "?? " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ': sleep
sleep
Print str(MaListe.NodeCount) & " totaux"
sleep
MaListe.EndNode
Print "*? " & MaListe.hashTag & " / " & MaListe.Tag
'u=0 : MaListe.Root : While MaListe.HashStep=1 : u+=1 : Wend
Print MaListe.DropAll & " / " & MaListe.NodeCount & " / Total parse=" & u
print "destroy"
MaListe.Destroy
Print "Fin **********"
sleep : system
Re: LZLE List Engine with user friendly powerfull syntax
Ok, since I'm also coding some data structures I need, I decided to give this a go. These are my results (all code compiled with fbc 1.05.0):Lost Zergling wrote:I think I'd appreciate some more returns, tests, remarks, questions and so on. Every subjects open.
32-bit
- example 1: CRASHES at the end
- example 2: CRASHES at the end
- example 3: CRASHES at the end
- example 4: CRASHES at the end
- example 5: HANGS at the end
- example 6: CRASHES at the end
- example 7: CRASHES at the end, sometimes taking hostages (my code editor)
- tuto001: OK
- tuto002: OK
- tuto003: OK
- tuto004: OK
- tuto005: OK
- tuto006: CRASHES at the end
- tuto007: has typo, fixed --> CRASHES at the end
- hashSort: CRASHES at the end
- example 1: CRASHES HARD at the end: took my code editor with it
- example 2: CRASHES RANDOMLY (sometimes it ends just by blind luck)
- example 3: CRASHES at the end
- example 4: CRASHES at the end
- example 5: CRASHES at the end
- example 6: CRASHES at the end
- example 7: CRASHES at the end
- tuto001: OK
- tuto002: OK
- tuto003: OK
- tuto004: OK
- tuto005: OK
- tuto006: CRASHES at the end
- tuto007: has typo, fixed --> CRASHES at the end
- hashSort: prints a '1', then CRASHES immediately
- make the method private and call it from the destructor only
- include a flag member variable to prevent deallocating the list twice (once explicitly, and the other implicitly at object destruction)
Another issue arises with shallow copies. This seemingly innocuous code will crash:
Code: Select all
#include once "lzle.bi"
scope
dim as List myList '' Ok
dim as List myList2 = mylist '' Not ok, creates a shallow copy of myList
'' From now on, using either object will wreak havoc and probably crash
end scope
/'
And here, the program invariably crashes, for destroying either object will cause
all the pointers from the other to become invalid since both objects share the
same explicitly allocated memory
'/
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (hopefully)
Hello Paul Doe
Thank you for interest. I'm on hollidays and I do not have a 64 bit machine for testing.
Be sure you test the last updated version (updated on 19/04)(twice the same day) (I just made a test and ex 2 does not crash at end on w7 32 bits, so I couldn't reproduce bugs you mentionned).
Theses end crashes where due to a multiple deallocation attemps on "garbagepointer", fixed : thanks to sancho3 posts.
dim as List myList2 = mylist lead to crash but should not be needed because there are alternate ways handling the object datas wich are Snatch, GarbageSnatch and FlatSnatch.
dim as List myList2 = mylist this syntax is not compatible with the way lzle has been designed, you're thrue.
And also it is not compliant with FBC object implementation, wich is a problem and a serious issue on wich I may investigate.
I think the copy constructor should be disabled.
You should use :
Dim As List MyList1, MyList2
.. working on list1...
myList2.Snatch(myList1) -> transferring datas from List1 to List2 (current node and sub levels)
myList2.GarbageSnatch(myList1) -> transferring dereferenced elements from List1 to List2
myList2.FlatSnatch(myList1) -> transferring protected flat list from List1 to List2
Then comes the random crash you mentionned : this looks like the critical fix on the 13/04 version.
Maybe you can check this ? Are your tests on Windows platforms ?
Do you have some other exemples of random crashes ?
I'm using FBide : sometimes I need to secure my tests by doing a compile job then a launch by double clic from Windows explorer : compile & run is not always 100% secure.
You mentionned : hashSort: prints a '1', then CRASHES immediately (64 bits) : on wich code exemple does this occurs ?
PS : do not forget to deactivate the licence message : sleep might lead to unexpectable results in some times.
Thank you for interest. I'm on hollidays and I do not have a 64 bit machine for testing.
Be sure you test the last updated version (updated on 19/04)(twice the same day) (I just made a test and ex 2 does not crash at end on w7 32 bits, so I couldn't reproduce bugs you mentionned).
Theses end crashes where due to a multiple deallocation attemps on "garbagepointer", fixed : thanks to sancho3 posts.
dim as List myList2 = mylist lead to crash but should not be needed because there are alternate ways handling the object datas wich are Snatch, GarbageSnatch and FlatSnatch.
dim as List myList2 = mylist this syntax is not compatible with the way lzle has been designed, you're thrue.
And also it is not compliant with FBC object implementation, wich is a problem and a serious issue on wich I may investigate.
I think the copy constructor should be disabled.
You should use :
Dim As List MyList1, MyList2
.. working on list1...
myList2.Snatch(myList1) -> transferring datas from List1 to List2 (current node and sub levels)
myList2.GarbageSnatch(myList1) -> transferring dereferenced elements from List1 to List2
myList2.FlatSnatch(myList1) -> transferring protected flat list from List1 to List2
Then comes the random crash you mentionned : this looks like the critical fix on the 13/04 version.
Maybe you can check this ? Are your tests on Windows platforms ?
Do you have some other exemples of random crashes ?
I'm using FBide : sometimes I need to secure my tests by doing a compile job then a launch by double clic from Windows explorer : compile & run is not always 100% secure.
You mentionned : hashSort: prints a '1', then CRASHES immediately (64 bits) : on wich code exemple does this occurs ?
PS : do not forget to deactivate the licence message : sleep might lead to unexpectable results in some times.
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (hopefully)
:sigh:
Let's go through this, carefully and slowly:
This is valid but not really needed. However, take for instance the tuto002.bas file (as it stands, it compiles ok): as soon as you add 'MyList.destroy()', the program crashes. This is of course totally previsible (you're deallocating memory twice), but in that case, the destroy() method should not be allowed to be public if it's called from the destructor. A possible solution:
That way, you'll avoid these issues. And most of the examples you provided call the destroy() method explicitly at the program end.
There. Now, doing this:
Will issue a compile time error, which is far better than simply crashing the app, or hope that the user of the library don't do something stupid (that's indeed valid and even expected syntax and behavior, unless you have a very good reason to not allow it, which I'll discuss later).
Let's see:
Does anybody beside me experience these crashes?
Let's go through this, carefully and slowly:
You're welcome. The version I'm using is the one in the first post, so I assume it's the latest one. The examples crash on both compilers, 32 and 64 bit versions. I'm using 1.05.0 and/or 1.06.0, compiling with the gcc backend. This is the complete command line switches of my usual compile: fbc -s console -mt -gen gcc -Wc -Ofast. I have a Win10 box, latest update (don't remember the number, sorry).Lost Zergling wrote:Hello Paul Doe
Thank you for interest. I'm on hollidays and I do not have a 64 bit machine for testing.
Be sure you test the last updated version (updated on 19/04)(twice the same day) (I just made a test and ex 2 does not crash at end on w7 32 bits, so I couldn't reproduce bugs you mentionned).
These end crashes are due to the code calling 'list.destroy' twice in succession:Lost Zergling wrote:Theses end crashes where due to a multiple deallocation attemps on "garbagepointer", fixed : thanks to sancho3 posts.
Code: Select all
Destructor List : this.Destroy : End Destructor
'' Excerpt from exemple 1.bas follow
Print MaListe.DropAll & " / " & MaListe.NodeCount & " / Total parse=" & u
print "destroy"
MaListe.Destroy
Print "Fin **********"
Code: Select all
constructor List()
m_destroyed = false '' m_destroyed is a private member variable
end constructor
destructor List()
if( m_destroyed = false ) then
m_destroyed = true
destroy()
end if
end destructor
Of course. But the posibility is there, and that amounts to poor design. Fortunately, the solution is also simple:Lost Zergling wrote:dim as List myList2 = mylist lead to crash but should not be needed because there are alternate ways handling the object datas wich are Snatch, GarbageSnatch and FlatSnatch.
dim as List myList2 = mylist this syntax is not compatible with the way lzle has been designed, you're thrue.
Code: Select all
type List
public:
'' ...
private:
'' Put the copy constructor and the let() assignment operators here
declare constructor( byref as List )
declare operator let( byref as List )
'' ...
end type
Code: Select all
dim as List a, b
b = a
Object orientation in FreeBasic has, like almost every other language, it's quirks and traps. Eventually, you'll grasp each and every one of them. For now, those two simple fixes I propose resolve some simple implementation oversights. However...Lost Zergling wrote:And also it is not compliant with FBC object implementation, wich is a problem and a serious issue on wich I may investigate.
I think the copy constructor should be disabled.
...this is the meat of the question. I already told you my specs, and I use FbEdit (never had any trouble compiling anything with it).Lost Zergling wrote:Then comes the random crash you mentionned : this looks like the critical fix on the 13/04 version.
Maybe you can check this ? Are your tests on Windows platforms ?
Do you have some other exemples of random crashes ?
I'm using FBide : sometimes I need to secure my tests by doing a compile job then a launch by double clic from Windows explorer : compile & run is not always 100% secure.
You mentionned : hashSort: prints a '1', then CRASHES immediately (64 bits) : on wich code exemple does this occurs ?
Let's see:
- the 'hashSort' example is the one you provided immediately above my previous post.
- I have already detailed my results, in the aforementioned post. The worst offender is 'exemple 1.bas' which seems totally random. Sometimes it stops halfway, sometimes it goes on a little further, but it seems pretty random to me.
- have a good look at 'exemple 5.bas'. The fact that the behavior of the example changes depending on which compiler you use (32-bit hangs, 64-bit crashes) probably means there's a pointer amidst the mess that has the wrong dimensions. You see, integer ptrs are 4 bytes on 32-bit, and 8 bytes on 64-bit. The integer (and it's uinteger equivalent) are the only ones that change size depending on the compiler used (as some members, like MrSwiss, cheerfully remind us from time to time =D)
- the random crashes are probably due to off-by-one errors in pointer dereferences. Depending on how much memory it's allocated (and where), the dereference may *just* land in the pad area (normally 1 to 3 bytes) and you may get away with it. That would explain the extremely random nature of the crashes.
- the 'hashSort' example is another one of the random crashers. Sometimes it prints a '1' and dies, other times it goes on a little longer, and sometimes it even finishes, provided you remembered to comment that 'MaListe.Destroy' call at the end...
Does anybody beside me experience these crashes?
Last edited by paul doe on Apr 27, 2018 5:02, edited 2 times in total.
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (hopefully)
There seems to be a problem buried in the HashTag() method also. The 'hashSort' example I mentioned before crashes (most of the times) on a hashTag() call in the second loop:
There seems to be an older version of it, commented at the side. Do you happen to have the older version? Is it working correctly? What does hashTag() do? Traverse the tree until the 'hashtag' is found, and returns whether the tag was found?
Code: Select all
'' CRASHES randomly
#Include once "lzle.bi"
Dim MaListe As List
MaListe.BranchCountDown(1)
MaListe.HashSort(1)
Dim u as ulongint : Dim i as ulongint
MaListe.HashTag("a") : MaListe.Val("2")
Print "1"
? "Executing first loop..."
For u=1100000030 To 1100000000 step -1
? " Executing HashTag..."
MaListe.HashTag(str(u))
? " Done"
? " Executing Val..."
MaListe.Val(str(u-1100000000))
? " Done."
Next u
? "Done."
? "Executing second loop..."
For u=28 To 20 Step -1
? " Executing HashTag..."
MaListe.HashTag(str(u))
? " Done."
? " Executing Val..."
MaListe.Val(str(u))
? " Done."
Next u
? "Done."
Print "node count=" & Str(MaListe.Nodecount) & " count=" & Str(MaListe.count)
Print "++"
MaListe.First
MaListe.HashTag("8") : MaListe.Val("55")
MaListe.Root
MaListe.RootNode
Print "* " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count)
'While MaListe.KeyStep=1
While MaListe.HashStep=1
Print "* " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count)
Wend
Print "---------------------"
'
Print "*Count=" & MaListe.Count
MaListe.Root
Print "#Count=" & MaListe.Count
While MaListe.HashStep=1
If (Cint(MaListe.Val) > 22 And Cint(MaListe.Val) <25) Or Cint(MaListe.Val) =30 Then
MaListe.NodeFlat
ElseIf Cint(MaListe.Val) =0 Then
MaListe.NodeFlat
End If
Wend
MaListe.Root
Print "Count=" & MaListe.Count
'
MaListe.Root
MaListe.RootNode
While MaListe.HashStep=1
Print "% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) '& " suivant=" & MaListe.NextNode ': '
Wend
Print "F 01"
'
MaListe.Root
For i=1 To MaListe.AllOf : MaListe.BlindStep
Print "+ " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count)
Next i
'
Print "F 02"
'
MaListe.Root
While MaListe.HashStep=1
Print "% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) '& " suivant=" & MaListe.NextNode ': '
Wend
Print "F 03"
'
MaListe.Root
MaListe.EndNode
Print "# EndNode=" & MaListe.Tag
Print "root1"
MaListe.Root
Print "root2"
MaListe.FlatStack :
Print "--------------------"
'
MaListe.Root
While MaListe.HashStep=1
Print "~ " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
Wend
Print "--"
MaListe.Last
Print MaListe.HashTag & " =Last ?"
MaListe.Root
For u=1100000040 To 1100000050
MaListe.HashTag(str(u)) : MaListe.Val(str(u-1100000000))
Next u
MaListe.Root
While MaListe.HashStep=1
Print "$ " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
Wend
MaListe.HashTag("52000067") : MaListe.Val("9")
'
MaListe.Root
MaListe.HashTag("32000067") : MaListe.Val("98")
MaListe.HashTag("62000067") : MaListe.Val("5")
MaListe.Root
While MaListe.HashStep=1
Print "# " & MaListe.HashTag & " - " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
Wend
MaListe.EndNode
Print MaListe.HashTag & " =EndNode ? " & MaListe.Tag
MaListe.Last
Print MaListe.HashTag & " =Last ? " & MaListe.Tag :
'
MaListe.Root
MaListe.EndNode : 'MaListe.Val("??")
Print MaListe.HashTag & " =EndNode ?" :
MaListe.Last
Print MaListe.HashTag & " =Last ?" : '
MaListe.HashTag("9876") : MaListe.Val("9")
MaListe.Root
While MaListe.HashStep=1
Print "##? " & MaListe.HashTag & " - " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
Wend
Print "Fin 1"
'
MaListe.Root
While MaListe.HashStep=1
If Cint(MaListe.Val) < 18 Or ( Cint(MaListe.Val)<49 And Cint(MaListe.Val)>40 ) Then ' Or ( Cint(MaListe.Val)<50 And Cint(MaListe.Val)>40 )
' Print "Flating " & MaListe.HashTag ': '
MaListe.NodeFlat
Else
' Print "Parsing " & MaListe.HashTag ': '
' MaListe.NodeRecycle
End If
Wend
'MaListe.NodeRecycle
print "FIN"
''
' :' system
' MaListe.HashTag("1100000000")
'If 1=0 Then
'MaListe.HashTag("32000067")
MaListe.HashTag("1100000030")
MaListe.BlindStep(-1) : Print "30->" & MaListe.Tag
MaListe.BlindStep(1) : Print "30->" & MaListe.HashTag
MaListe.Last : Print "30 Last->" & MaListe.HashTag
MaListe.HashTag("110000004") : Print "110000004 is branch=" & MaListe.BranchCount
MaListe.HashTag("1100000040")
MaListe.BlindStep(-1) : Print "40->" & MaListe.Tag
MaListe.BlindStep(1) : Print "40->" & MaListe.HashTag
MaListe.Last : Print "40 Last->" & MaListe.HashTag
'
MaListe.HashTag("1100000041")
MaListe.Root : MaListe.First
Print "++**++"
While MaListe.HashStep=1
Print "%% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
Wend
Print "Pt A01" : '
'MaListe.Root
MaListe.HashTag("110000001")
Print "%" & MaListe.HashTag & "%"
Print "Pt A02" : '
Print "~01"
MaListe.HashTag("1100000010")
MaListe.Root
Print "++**++"
While MaListe.HashStep=1
Print "%-% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
Wend
Print "~02"
'
Print "===================="
MaListe.Root
MaListe.FlatStack ': MaListe.Branch
Print "Count=" & MaListe.Count & " AllOf=" & MaListe.AllOf
Print "~03"
MaListe.RHMethod(-1)
For i=1 To MaListe.AllOf : MaListe.BlindStep
Print "restoring " & MaListe.Tag ': '
MaListe.RestoreHash
Next i
' MaListe.NodeRecycle
Print "--------------------"
Print "Restored "
'
MaListe.Root
MaListe.RootNode
MaListe.Val ("totor")
Print "*** " & MaListe.HashTag & " tag=" & MaListe.Tag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode
While MaListe.HashStep=1
' Print "% " & MaListe.Tag & " = " & MaListe.Val
Print "** " & MaListe.HashTag & " tag=" & MaListe.Tag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ' & " suivant=" & MaListe.NextNode ' : '
Wend
Print "--------------------"
'
'system
'MaListe.Root
'MaListe.BlindTag("")
Print "* flating"
MaListe.Root
'While MaListe.HashStep=1
' Print "* flating -> " & MaListe.HashTag ' : '
' MaListe.NodeFlat
'Wend
''
i=0
'dim i as integer=0
MaListe.Root
Print "count=" & MaListe.Count : '
Print MaListe.Tag
MaListe.Root
Print "# " & MaListe.Tag : MaListe.BlindStep
Print "# " & MaListe.Tag : MaListe.BlindStep
Print "# " & MaListe.Tag : MaListe.BlindStep
Print "# " & MaListe.Tag : MaListe.BlindStep
'Print "**********"
'MaListe.BlindStep : Print MaListe.Tag
Print "**********"
Print "value=" & str(u) & " / " & str(i)
Print "FlatCount=" & MaListe.Count
Print "GarbageCount=" & MaListe.GarbageCount
Print "NodeCount=" & MaListe.NodeCount
Print "**********"
'
u=1
MaListe.Root
MaListe.RootNode
Print "% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ': '
While MaListe.HashStep=1
' Print "% " & MaListe.Tag & " = " & MaListe.Val
Print "% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ': '
u+=1
Wend
'
Print "% " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ': '
' MaListe.BlindStep
'Print "?= " & MaListe.tag
'MaListe.BlindStep
'Print "?= " & MaListe.tag
'MaListe.BlindStep
'Print "?= " & MaListe.tag
Print "Node visibles parcourables = " & str(u) & " Garbage=" & MaListe.GarbageCount '& " Flat=" & MaListe.FlatCount
'
' Print "Node visibles parcourables = " & str(u) & " Garbage=" & MaListe.GarbageCount '& " Flat=" & MaListe.FlatCount
MaListe.First 'GarbagePtr
Print "?? " & MaListe.HashTag & " = " & MaListe.Val & " branch " & str(MaListe.BranchCount) & " count=" & str(MaListe.Count) ': '
'
Print str(MaListe.NodeCount) & " totaux"
'
MaListe.EndNode
Print "*? " & MaListe.hashTag & " / " & MaListe.Tag
'u=0 : MaListe.Root : While MaListe.HashStep=1 : u+=1 : Wend
Print MaListe.DropAll & " / " & MaListe.NodeCount & " / Total parse=" & u
Print "Fin **********"
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (hopefully)
Hello Paul,
I have tested the random crash exemple, wich is the meat of the question. Well, I could reproduce it on W7 32 bits : your latest exemple
'' CRASHES randomly
I did change the following line :
Dim As uInteger uNodeCOUNT, uGarbCt, uCount, uContainerGarbCt, uContainerGivenCt
To
Dim As uLong uNodeCOUNT, uGarbCt, uCount, uContainerGarbCt, uContainerGivenCt
And it does not crash any longer.
Let's have a look to this post : https://freebasic.net/forum/viewtopic.php?f=17&t=26618 (thx to MrSwiss)
What I do not understand is why the bug is in 32 bits also and why it can be corrected also using the uLong instead of uInteger.
It appears that I have assumed my ability to solve this issue the way I had imagine.
My fault, I thought I could use uInteger while uLong is safe. But perhaps or probably is there something else I do not see ?
My fault again, from 19/04 to 21/04 I made slight changes and I didn't report it hoping no one would be impacted.
I thank you again for returns this really helps me for evolutions.
A by pass on Destroy is an idea, I wish to maintain a public access to Destroy() (for fbc 0.20 and above).
Same I wish to maintain the object implementation compliant with older versions.
Currently, I have no time working on further documentation because I'm investigating on random crashes, result sets and ascending compatibility issues on undocumented features. Probably this uLong instead of uInteger will solve some issues.
I have tested the random crash exemple, wich is the meat of the question. Well, I could reproduce it on W7 32 bits : your latest exemple
'' CRASHES randomly
I did change the following line :
Dim As uInteger uNodeCOUNT, uGarbCt, uCount, uContainerGarbCt, uContainerGivenCt
To
Dim As uLong uNodeCOUNT, uGarbCt, uCount, uContainerGarbCt, uContainerGivenCt
And it does not crash any longer.
Let's have a look to this post : https://freebasic.net/forum/viewtopic.php?f=17&t=26618 (thx to MrSwiss)
What I do not understand is why the bug is in 32 bits also and why it can be corrected also using the uLong instead of uInteger.
It appears that I have assumed my ability to solve this issue the way I had imagine.
My fault, I thought I could use uInteger while uLong is safe. But perhaps or probably is there something else I do not see ?
My fault again, from 19/04 to 21/04 I made slight changes and I didn't report it hoping no one would be impacted.
I thank you again for returns this really helps me for evolutions.
A by pass on Destroy is an idea, I wish to maintain a public access to Destroy() (for fbc 0.20 and above).
Same I wish to maintain the object implementation compliant with older versions.
Currently, I have no time working on further documentation because I'm investigating on random crashes, result sets and ascending compatibility issues on undocumented features. Probably this uLong instead of uInteger will solve some issues.
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
I also obtain crashes with Ulong in 32-bit and 64-bit.
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
On my side : W7 32 bits, fbc 0.23 : compile & run : no crash. Accelerate execution : crash. What's important is compile & run.
I continue my investigations,...
When it was running OK one time, then it becomes ok even though with "Accelerate execution" => Looks like a cache issue on pointers at compile time or somewhat else ? What a puzzle! I don't know what I could do more to fix this issue.
I continue my investigations,...
When it was running OK one time, then it becomes ok even though with "Accelerate execution" => Looks like a cache issue on pointers at compile time or somewhat else ? What a puzzle! I don't know what I could do more to fix this issue.
Last edited by Lost Zergling on Apr 27, 2018 11:09, edited 1 time in total.
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
- First bug:
when allocating memory for a zstring, you must allocate an additional ubyte for the terminal character (0).
.....
Dim As zString Ptr Listptemp=Allocate(iLen+1), Listptemp2=Allocate(iLen+1), zp1, zp2, zp3=Allocate(1) ' * Alternative to Mid, left & right ==>+20-25% speed tested
.....
Dim As zString Ptr Listptemp=Allocate(Len(str_Tag)+1), zp1, zp2, zp3=Allocate(1)
- Second bug in 'Property List.HashTag(str_Tag As String) As Byte':
'len(this.HashTag)' may be greater than 'ilen' (up to 8 characters greater), so allocated memory overflows when executing '*Listptemp=Str_tmp'
- Eventual some next bugs because I stopped here my code review.
when allocating memory for a zstring, you must allocate an additional ubyte for the terminal character (0).
.....
Dim As zString Ptr Listptemp=Allocate(iLen+1), Listptemp2=Allocate(iLen+1), zp1, zp2, zp3=Allocate(1) ' * Alternative to Mid, left & right ==>+20-25% speed tested
.....
Dim As zString Ptr Listptemp=Allocate(Len(str_Tag)+1), zp1, zp2, zp3=Allocate(1)
- Second bug in 'Property List.HashTag(str_Tag As String) As Byte':
'len(this.HashTag)' may be greater than 'ilen' (up to 8 characters greater), so allocated memory overflows when executing '*Listptemp=Str_tmp'
- Eventual some next bugs because I stopped here my code review.
Last edited by fxm on Apr 27, 2018 11:15, edited 1 time in total.
-
- Posts: 538
- Joined: Dec 02, 2011 22:51
- Location: France
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
Ok I take note. Thx fxm.
Re: LZLE List Engine with user friendly powerfull syntax - Alpha1 Frozen (Latest FIX 27/04)
I allocated thus:
Dim As zString Ptr Listptemp=Allocate(iLen*sizeof(string)), Listptemp2=Allocate(iLen*sizeof(string)), zp1, zp2, zp3=Allocate(1*sizeof(string)) ' * Alternative to Mid, left & right ==>+20-25% speed tested
The ofending property is List.HashTag
The other offender is
List.HasHashTag
but is not used in your test, but needs fixed also.
Maybe an overkill to fxm's adding only one byte.
Dim As zString Ptr Listptemp=Allocate(iLen*sizeof(string)), Listptemp2=Allocate(iLen*sizeof(string)), zp1, zp2, zp3=Allocate(1*sizeof(string)) ' * Alternative to Mid, left & right ==>+20-25% speed tested
The ofending property is List.HashTag
The other offender is
List.HasHashTag
but is not used in your test, but needs fixed also.
Maybe an overkill to fxm's adding only one byte.