Weird problem: CreateFile seems to return equal handles for different files

Windows specific questions.
Post Reply
jj2007
Posts: 2326
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Weird problem: CreateFile seems to return equal handles for different files

Post by jj2007 »

Code: Select all

#include "Windows.bi"
Dim As handle fh2, fh1
fh1=CreateFile("..\examples\unicode\hello_chinese.bas", GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0)
Print "The handle #1: ", fh1
fh2=CreateFile("..\examples\unicode\hello_greek.bas", GENERIC_READ, FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, 0)
Print "The handle #2: ", fh2
if fh1=fh2 Then Print "The handles are equal, and that is NOT ok"
sleep
The exe sits one level below the main FreeBasic folder, so I assumed that the two files do exist. Output:

Code: Select all

The handle #1:              18446744073709551615
The handle #2:              18446744073709551615
The handles are equal, and that is NOT ok
Thorough investigation reveals that the "handle" 18446744073709551615 is... minus 1. ERROR_INVALID_HANDLE. The files do exist, but since I launched the exe from the main FB level, the ..\ was wrong. Now I wonder why Print spits out the full unsigned QWORD instead of a simple -1...

P.S.: With Dim As longlong fh2, fh1, it works correctly. Maybe the type handle is not properly defined (it's type HANDLE as any ptr in WinNt.bi). There is also a not-so-well-known difference between HFILE and HANDLE, see this SOF thread.
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by marcov »

Windows handles are unsigned, that is normal.
jj2007
Posts: 2326
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by jj2007 »

As usual, opinions differ, marcov ;-)
According to docs.microsoft..., converting from 32 to 64 bit handles involves sign-extending the value, so this means handles are NOT unsigned
64-bit versions of Windows use 32-bit handles for interoperability. When sharing a handle between 32-bit and 64-bit applications, only the lower 32 bits are significant, so it is safe to truncate the handle (when passing it from 64-bit to 32-bit) or sign-extend the handle (when passing it from 32-bit to 64-bit)
Note the ambiguity: use 32-bit handles ... but you must sign-extend them to 64 bits... hilarious.
dodicat
Posts: 7976
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by dodicat »

In fb a handle is any ptr.
The equal return values indicate that the files do not exist, or at least they are not found.
If things go to plan then the handles are different.

Code: Select all

#include "Windows.bi"
#include "file.bi"
Dim As handle fh2, fh1
#print typeof(fh1)

fh1=CreateFile("hello_chinese.bas", GENERIC_READ,FILE_SHARE_READ or FILE_SHARE_DELETE, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0)
Print "The handle #1: ", fh1
fh2=CreateFile("hello_greek.bas", GENERIC_READ, FILE_SHARE_READ or FILE_SHARE_DELETE, NULL, CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0)
Print "The handle #2: ", fh2
if fh1=fh2 Then Print "The handles are equal, and that is NOT ok"
print fileexists("hello_chinese.bas")
print fileexists("hello_greek.bas")

kill ("hello_chinese.bas")
kill "hello_greek.bas"
print iif(fileexists("hello_chinese.bas") or fileexists("hello_greek.bas"),"Delete these files manually","OK, files have been deleted")
print fileexists("hello_chinese.bas")
print fileexists("hello_greek.bas")
sleep 
That's as far Is I understand the situation anyway.
My result:

Code: Select all

The handle #1:              616
The handle #2:              204
-1
-1
OK, files have been deleted
 0
 0
 
jj2007
Posts: 2326
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by jj2007 »

dodicat wrote:In fb a handle is any ptr.
The equal return values indicate that the files do not exist, or at least they are not found.
If things go to plan then the handles are different.
Yes, that's all correct. But try your example with 0 instead of CREATE_ALWAYS...
dodicat
Posts: 7976
Joined: Jan 10, 2006 20:30
Location: Scotland

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by dodicat »

If I use 0 then no files are created, so no files are loaded, so the pointers is not set to specific values.
(The pointer values seem arbitrarily set by windows, and do not represent addresses).
Looks like care must be taken in choosing parameter 4 in order to load an existing file or creating a new one.
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by marcov »

jj2007 wrote:As usual, opinions differ, marcov ;-)
According to docs.microsoft..., converting from 32 to 64 bit handles involves sign-extending the value, so this means handles are NOT unsigned
64-bit versions of Windows use 32-bit handles for interoperability. When sharing a handle between 32-bit and 64-bit applications, only the lower 32 bits are significant, so it is safe to truncate the handle (when passing it from 64-bit to 32-bit) or sign-extend the handle (when passing it from 32-bit to 64-bit)
Note the ambiguity: use 32-bit handles ... but you must sign-extend them to 64 bits... hilarious.
I checked meanwhile, and in the SDK I can find pointer.

The conversion is not related to the signed type of the handle type, but due to the Windows kernel being at the top of the memory range.
erik
Posts: 39
Joined: Dec 28, 2020 17:27
Location: Krasnoyarsk
Contact:

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by erik »

jj2007 wrote:Maybe the type handle is not properly defined
The header file "winnt.bi" is derived from the header file "winnt.h". Open " winnt.h" and look at the definition of the HANDLE data type:

Code: Select all

#ifdef STRICT
typedef void *HANDLE;
#else
typedef PVOID HANDLE;
#endif
typedef HANDLE *PHANDLE;
As you can see, HANDLE is defined as a pointer to void, in FreeBASIC it equvalent "Any Ptr".
Last edited by erik on Mar 04, 2021 12:43, edited 1 time in total.
fxm
Moderator
Posts: 12081
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by fxm »

If 'CreateFile()' fails, the return value is 'INVALID_HANDLE_VALUE', value corresponding to '-1', i.e.: '&hFFFFFFFFFFFFFFFF' in your 64-bit case (18446744073709551615).
jj2007
Posts: 2326
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by jj2007 »

dodicat wrote:If I use 0 then no files are created, so no files are loaded
Indeed. As a matter of fact, GENERIC_READ and CREATE_ALWAYS are not really compatible: either you want to read a file (then it must exist, and it should have more than zero bytes), or you want to write to it. In the latter case, Windows will throw an error if you did not specify CREATE_ALWAYS and the file does not yet exist:

Code: Select all

hFile=CreateFile("blafoo.bas", GENERIC_WRITE, FILE_SHARE_WRITE, NULL, 0, FILE_ATTRIBUTE_NORMAL, 0)
If blafoo.bas does not exist, CreateFile will return INVALID_HANDLE_VALUE (which is defined as -1 in WinBase.h - for the C/C++ fans).
jj2007
Posts: 2326
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by jj2007 »

fxm wrote:If 'CreateFile()' fails, the return value is 'INVALID_HANDLE_VALUE', value corresponding to '-1', i.e.: '&hFFFFFFFFFFFFFFFF' in your 64-bit case (18446744073709551615).
fxm, thank you so much for explaining numbers to me!

Yes, you are probably right that INVALID_HANDLE_VALUE corresponds, vaguely, to 18446744073709551615. My point was, though, that the definition of a handle should be signed integer, so that Print MyHandle would spit out -1 alias INVALID_HANDLE_VALUE instead of the somewhat unwieldy, almost bulky, 18446744073709551615, that might scare beginners away from FreeBasic.

The Sherlock Holmes in me bases this judgment on two elements:
1. WinBase.h: #define INVALID_HANDLE_VALUE ((HANDLE)(LONG_PTR)-1)
2. The recommendation of Microsoft (y'know, the company that sells Windows):
"64-bit versions of Windows use 32-bit handles for interoperability. When sharing a handle between 32-bit and 64-bit applications, only the lower 32 bits are significant, so it is safe to truncate the handle (when passing it from 64-bit to 32-bit) or sign-extend the handle (when passing it from 32-bit to 64-bit)".
You might actually be in trouble if you passed the 32-bit handle &H80000001h as &H80000001h (i.e. unsigned) to the 64-bit OS. Let me know if it works, though - I am always willing to learn.
jj2007
Posts: 2326
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by jj2007 »

marcov wrote:The conversion is not related to the signed type of the handle type, but due to the Windows kernel being at the top of the memory range.
Are you sure? Any reference to that? Which mem range, 32-bit or 64-bit? My Win7-64 CreateFileA, for example, sits at &H772F1720, which is not that high... actually, it's a positive 32-bit number. Remember also that Windows handles have nothing to do with memory locations. They do not point anywhere.
marcov
Posts: 3455
Joined: Jun 16, 2005 9:45
Location: Netherlands
Contact:

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by marcov »

jj2007 wrote:
marcov wrote:The conversion is not related to the signed type of the handle type, but due to the Windows kernel being at the top of the memory range.
Are you sure? Any reference to that?
Dave Cutler c.s ? Search for "Dave Cutler Windows NT".
Which mem range, 32-bit or 64-bit? My Win7-64 CreateFileA, for example, sits at &H772F1720, which is not that high... actually, it's a positive 32-bit number. Remember also that Windows handles have nothing to do with memory locations. They do not point anywhere.
It could be that they used pointer because that scales with CPU type. The origins of the 64-bit port of Windows is quite old (dating back to MIPS,Alpha)
jj2007
Posts: 2326
Joined: Oct 23, 2016 15:28
Location: Roma, Italia
Contact:

Re: Weird problem: CreateFile seems to return equal handles for different files

Post by jj2007 »

marcov wrote:Dave Cutler c.s ? Search for "Dave Cutler Windows NT"
I couldn't find his view on x64, but here is a detailed description of the layout, saying kernel addresses are indeed in the negative range. Unfortunately, most debuggers can't follow syscall, see below in ZwCreateFile. In any case, there is no link between handles and kernel addresses, and I consider my two arguments re signed definition of INVALID_HANDLE_VALUE and sign extension valid ;-)

Btw I've never seen a negative CreateFile handle. They are, surprisingly, most of the time in the BYTE range.

Code: Select all

0000000077559CF0 < | 4C 8B D1                | mov r10,rcx                                               |
0000000077559CF3   | B8 52 00 00 00          | mov eax,52                                                | 52:'R'
0000000077559CF8   | 0F 05                   | syscall                                                   |
0000000077559CFA   | C3                      | ret                                                       |
Post Reply