when does FileFlush() return?

[SamL]

The goal is to guarantee data is on the disk before continuing in case of a power loss so any corruption can be undone.

I'm learning how to use FileFlush(filenum,1) and have a question.

I turned off my system buffer for my HDD in windows,
Device manager>Disk drives>(my drive)>properties>Policies, un check "Enable write cashing on the device"

so now my system buffer is off, I have the application buffer and the HDD Cache to flush.

Where is the file data guaranteed to be after a FileFlush(filenum,1) call?
a. after the application file buffer flush is done?
b. after the system file buffer flush is done?
c. after the HDD cache write to disk is done?


Regards,
SamL

[badidea]

Not sure if there are any hard guarantees. FileFlush() calls fb_hFileFlushEx() which calls FlushFileBuffers() on Windows. Which should make sure that data is actually written to disk, but read this: Revised notes on the reliability of FlushFileBuffers

[St_W]

On Windows, FreeBasic uses the FlushFileBuffers Win32 API method internally, see:
https://github.com/swurzinger/fbc/blob/ ... flush.c#L9

Documentation on that method from Microsoft on MSDN: https://docs.microsoft.com/en-us/window ... ilebuffers
While the documentation doesn't go into too much detail there's more info on Raymon Chen's Blog "Old new thing" (which I can highly recommend in general, btw): https://devblogs.microsoft.com/oldnewth ... 0/?p=95505

According to that you can generally expect the data to be physically written to disk (or whatever storage type is used), but there can be exceptions (depending on hardware, windows version, drivers, system configuration and possibly more).

//edit: seems "badidea" was a little bit faster than me

[badidea]

Interesting, simultaneous post. With almost the same content.

[jj2007]

you can generally expect the data to be physically written to disk

Good question. I would expect the following to happen:
- program writes to disk
- data are now in disk-internal cache
- oops, power failure!
- disk has some condensators that allow to flush the cache automagically

[SamL]

FileFlush() looks like the best i can do and its pretty good command. Calling it often will slow down the code

I did see in the C Standard Library Functions here https://www.freebasic.net/wiki/ProPgCruntime#LOLEVIO there is _close(handle as integer) as integer Close a file opened for unbuffered I/O.

I assume that means _open(filename as zstring ptr, oflag as integer, pmode as uinteger) is an unbuffered open and may write directly to the OS buffer or disk cache if OS buffer is off. but i'm not sure and need to research that more.

Thanks for the links i read through them and searched up more info and it appears that any buffer flush command leaves the HDD cache with the option to flush but its not guaranteed when. some hard drives ignore requests to dump the cache some don't. and i think only some high-end hard drives have any kind reply back that lets the code know that the data is physically recorded. i am using a 12 TB Ironwolf NAS drive and it seems to dump the buffers and HDD cache as soon as i call FileFlush(). but on the exact timing of the function return, I don't know if the data is secure or not.

my conclusion is FileFlush() must return after the Application and system buffer's are flushed, then it's up to the drive to decide if it's going to flush the cache. good drives will flush asap, cheap ones ignore the flush command and get to it later if they are busy.

battery backup and code to handle shutting down databases on power loss is the automagical solution. also a good backup solution is required to prevent total data file loss due to corruption.

for now, I'm hashing every data entry and checking hashes on data and index nodes per requests to verify if data corruption exists also keeping verified regular backups.

any other thoughts on subject would be appreciated

thanks for the input!
Regards,
SamL

[jj2007]

you can generally expect the data to be physically written to disk

Good question. I would expect the following to happen:
- program writes to disk
- data are now in disk-internal cache
- oops, power failure!
- disk has some condensators that allow to flush the cache automagically

See https://www.quora.com/Do-hard-disk-driv ... er-is-lost