StringArray Sort (case independent)

Post your FreeBASIC tips and tricks here. Please don’t post your code without including an explanation.
dodicat
Posts: 5913
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: StringArray Sort (case independent)

Postby dodicat » Aug 17, 2019 21:53

My question was not for JK. (Any .o or .a or .exe (with symbols) apply)
32 bit versus 64 bit has been discussed.
This system is 64 bits, but the shell command nm only catches 32 bit files.
(nm should show all the functions used inside library files)
So, a function of the command shell in a 64 bit system can only handle 32 bit files.
Strange?
MrSwiss
Posts: 3221
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: StringArray Sort (case independent)

Postby MrSwiss » Aug 17, 2019 22:35

dodicat wrote:So, a function of the command shell in a 64 bit system can only handle 32 bit files. Strange?
Well, I can't find "nm" in command-shell (cmd.exe) nor in C:\Windows\System32\...
Seems to be a external utility ... (which can't be related directly to the shell itself).

If you are running Linux, I simply don't know.
dodicat
Posts: 5913
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: StringArray Sort (case independent)

Postby dodicat » Aug 17, 2019 22:56

Call this file this.bas

Code: Select all

 
'this.bas
sub getnames()
 shell "nm -g this.exe"
end sub

Print "starting"
getnames
sleep


Compile it with the -g flag (to keep the symbols)
Then run it.
do it with 32 bits then 64 bits.
MrSwiss
Posts: 3221
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: StringArray Sort (case independent)

Postby MrSwiss » Aug 17, 2019 23:06

Okay, first try FBC 64 results:
starting
'nm' is not recognized as an internal or external command,
operable program or batch file.
The second result with FBC 32, is identical ...
dodicat
Posts: 5913
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: StringArray Sort (case independent)

Postby dodicat » Aug 17, 2019 23:33

Looks like nm.exe resides in mingw, which I have a 32 bit version on path.
I apologize for wasting time.
dafhi
Posts: 1245
Joined: Jun 04, 2005 9:51

Re: StringArray Sort (case independent)

Postby dafhi » Aug 18, 2019 2:14

jj2007 wrote:..QSort() is a stable mergesort ..

:O

i've actually been working on an in-place merge for about a week. seems my brain wants me to solve this one.
jj2007
Posts: 1215
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: StringArray Sort (case independent)

Postby jj2007 » Aug 18, 2019 8:03

dafhi wrote:i've actually been working on an in-place merge for about a week. seems my brain wants me to solve this one.
My version took a bit longer, actually; and yesterday I threw out the call to StringsDiffer() and implemented Juergen's table solution instead (see Aug 06, 2019 9:30). Much faster for the case-insensitive sort.
Juergen Kuehlwein
Posts: 210
Joined: Mar 07, 2018 13:59
Location: Germany

Re: StringArray Sort (case independent)

Postby Juergen Kuehlwein » Aug 18, 2019 8:25

Mr. Swiss is right. I posted an idea (conversion table) and code in his thread, because it was relevant to his opening post. But further discussion about my code should take place here: viewtopic.php?f=17&p=263449#p263449. Thanks!


JK
jj2007
Posts: 1215
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: StringArray Sort (case independent)

Postby jj2007 » Aug 20, 2019 0:06

I updated my library algo QSort(), see Fast and powerful string sort. The trick with the conversion table gave it a 2.8x speed boost - thanks a lot, Juergen! As mentioned earlier, it's a stable merge sort, and it has a number of other features, such as sorting by column (for spreadsheets), sorting by value (for numbers - slow), and ignoring leading whitespace.

For those who are interested in assembly, here 1. how the conversion table gets filled and 2. how it gets used:

Code: Select all

   mov edx, offset qsCaseTable
   xor ecx, ecx
   .if ecx==[edx]   ; allows to fill the buffer with other data
      push 256
      push edx
      .Repeat
         mov [edx+ecx], cl
         inc cl
      .Until Zero?
      call CharLowerBuff   ; a few microseconds, once
   .endif

Code: Select all

         .Repeat      ; case-insensitive; strings to compare are in esi and edi
            movzx eax, byte ptr [esi+ecx]      ; 730 ms/mio lines on a Core(TM) i5-2450M CPU @ 2.50GHz
            movzx edx, byte ptr [edi+ecx]
            inc ecx
            movzx edx, qsCaseTable[edx]
            movzx eax, qsCaseTable[eax]
            test edx, edx
            .Break .if Zero?
            sub eax, edx
         .Until !Zero?   ; exit if a difference was found
MrSwiss
Posts: 3221
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: StringArray Sort (case independent)

Postby MrSwiss » Aug 20, 2019 0:52

jj2007 wrote:For those who are interested in assembly ...
Be at least honest: For those who are interested in historical assembly ...
Bye, bye, until the time, you've learned 64 bit ASM ... (or use FB instead).
jj2007
Posts: 1215
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: StringArray Sort (case independent)

Postby jj2007 » Aug 20, 2019 7:58

Post code, MrSwiss. If you are still a coder...
dodicat
Posts: 5913
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: StringArray Sort (case independent)

Postby dodicat » Aug 20, 2019 13:07

jj2007.
With avira
I scaned your .exe files while still compressed.
I scanned them uncompressed.

I checked them with nm.exe to see your functions, but you have them stripped of all symbols.
Avira still quarantined MbsortText.exe when I dragged a text file on it.
HEUR/APC was the name of the offender.
Seemingly it is a false positive caused by some new capricorn engine used by avira.
But nonetheless I cannot run your .exe files here.
Pity!
jj2007
Posts: 1215
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: StringArray Sort (case independent)

Postby jj2007 » Aug 20, 2019 13:20

It is obviously a false positive - I build the exes from scratch every day. No option to exclude a folder from scanning? I haven't use an AV for some years, the last one was Windows Defender, and it had such an option. If I had any intention to earn money with my coding, I would sue the hell out of them.

No symbols is standard, except when debugging is needed. When I have doubts about an exe, I run PeView to see which APIs are being used. For example, if you see UrlDownloadToFile, be cautious ;-)

Return to “Tips and Tricks”

Who is online

Users browsing this forum: No registered users and 30 guests