About sharing data between running executables

General FreeBASIC programming questions.
grindstone
Posts: 726
Joined: May 05, 2015 5:35
Location: Germany

Re: About sharing data between running executables

Postby grindstone » Jul 17, 2015 9:23

@marcov: You're surely right, but the question was: what's the easiest to code? (quick and dirty). The file method may be not very elegant, but it's amazing capable if you know how to do it.
marcov
Posts: 2969
Joined: Jun 16, 2005 9:45
Location: Eindhoven, NL
Contact:

Re: About sharing data between running executables

Postby marcov » Jul 17, 2015 9:36

grindstone wrote:@marcov: You're surely right, but the question was: what's the easiest to code? (quick and dirty). The file method may be not very elegant, but it's amazing capable if you know how to do it.


Well the internet is build on sockets, and that is also quite capable if you know how to do it :-)

But my point was more that filebased is hard to do right
MichaelW
Posts: 3500
Joined: May 16, 2006 22:34
Location: USA

Re: About sharing data between running executables

Postby MichaelW » Jul 18, 2015 8:04

Starting with the memory-mapped file examples I linked earlier in this thread, I created cut-down versions (with no error handling) that time access to the mapped memory. In actual practice you would probably need to synchronize access to the mapped memory, and one simple method of doing this is to use a mutex.

Code: Select all

''=============================================================================
#include "windows.bi"
#include "crt.bi"
''=============================================================================

#define FILE_SIZE 1000000000

dim as HANDLE hMapFile
dim as integer ptr p
dim as double t

''--------------------------------------------------------------------------
'' Create a named file mapping object. Specifying INVALID_HANDLE_VALUE for
'' the file handle causes the function to create a file mapping object that
'' is backed by the system paging file rather than by a file, so the object
'' is actually used to access shared memory.
''
'' If the function fails the return value is zero. If the function succeeds
'' the return value is a handle to a file mapping object, but note that if
'' the object existed before the function call GetLastError will return
'' ERROR_ALREADY_EXISTS.
''--------------------------------------------------------------------------

hMapFile = CreateFileMapping( cast(HANDLE,INVALID_HANDLE_VALUE), _
                              NULL, _
                              PAGE_READWRITE, _
                              0, _
                              FILE_SIZE, _
                              "FileMappingObject" )
print hMapFile

''---------------------------------------------------------------------------
'' Map a view of the shared memory into our address space. If the function
'' fails the return value is zero. If the function succeeds the return value
'' is the starting address of the mapped view.
''---------------------------------------------------------------------------

p = MapViewOfFile( hMapFile, _
                   FILE_MAP_ALL_ACCESS, _
                   0, _
                   0, _
                   0 )
print hex(p,16)

''----------------------------------------------------------------------
'' Do a timed initialization of the shared memory and display the time.
''----------------------------------------------------------------------

t = timer

for i as integer = 0 to FILE_SIZE/sizeof(integer)
    p[i] = i
next

print timer-t

UnmapViewOfFile( p )
CloseHandle( hMapFile )

sleep

Code: Select all

''=============================================================================
#include "windows.bi"
#include "crt.bi"
''=============================================================================

#define FILE_SIZE 1000000000

dim as HANDLE hMapFile
dim as integer ptr p
dim as double t

''----------------------------------------------------------------------------
'' Open our named file mapping object. If the function fails the return value
'' is zero. If the function succeeds the return value is an open handle to
'' the specified file mapping object.
''----------------------------------------------------------------------------

hMapFile = OpenFileMapping( FILE_MAP_ALL_ACCESS, _
                            false, _
                            "FileMappingObject" )

hMapFile = CreateFileMapping( cast(HANDLE,INVALID_HANDLE_VALUE), _
                              NULL, _
                              PAGE_READWRITE, _
                              0, _
                              FILE_SIZE, _
                              "FileMappingObject" )
print hMapFile

''---------------------------------------------------------------------------
'' Map a view of the shared memory into our address space. If the function
'' fails the return value is zero. If the function succeeds the return value
'' is the starting address of the mapped view.
''---------------------------------------------------------------------------

p = MapViewOfFile( hMapFile, _
                   FILE_MAP_ALL_ACCESS, _
                   0, _
                   0, _
                   0 )
print hex(p,16)

''----------------------------------------------------------------------
'' Do a timed initialization of the shared memory and display the time.
''----------------------------------------------------------------------

t = timer

for i as integer = 0 to FILE_SIZE/sizeof(integer)
    p[i] = i
next

print timer-t

UnmapViewOfFile( p )
CloseHandle( hMapFile )

sleep
grindstone
Posts: 726
Joined: May 05, 2015 5:35
Location: Germany

Re: About sharing data between running executables

Postby grindstone » Jul 18, 2015 8:22

marcov wrote:But my point was more that filebased is hard to do right
Not at all. You only have to open a file for writing in the emitting process and write the data to it, and open that same file in the receiving process for reading and read it periodically. That's all. No library, no care for timing, the only thing that both processes have to know is the filename. It's nearly foolproof.
MichaelW
Posts: 3500
Joined: May 16, 2006 22:34
Location: USA

Re: About sharing data between running executables

Postby MichaelW » Jul 18, 2015 8:49

grindstone wrote:It's nearly foolproof.

Without some sort of synchronization, how does the receiving process know that the write operation is complete, and how does the emitting process know that the read operation is complete?
grindstone
Posts: 726
Joined: May 05, 2015 5:35
Location: Germany

Re: About sharing data between running executables

Postby grindstone » Jul 18, 2015 10:14

Here's a little "quick and dirty" example. If you want, you can call the 'Close #ff' - statement "synchronisation", because afterwards the additional content of the file is accessable by the receiver:

Code: Select all

Print "TRANSMITTER"
Dim As String sf = ExePath + "\sharefile.txt"

Sub fwrite(text As String)
   Dim As Integer ff
   Print text
   ff = FreeFile
   Open ExePath + "\sharefile.txt" For Append As #ff
   Print #ff, text;
   Close #ff
   Sleep 1000
End Sub

Kill (sf)

Print
Print "Start RECEIVER and press a key"
Sleep
fwrite("The ")
fwrite("big ")
fwrite("brown ")
fwrite("fox ")
fwrite("jumped ")
fwrite("over ")
fwrite("the ")
fwrite("lazy ")
fwrite("dog ")

Do
Loop While Kill (sf)

Code: Select all

Dim As String text
Print "RECEIVER"

Do
Loop While Open (ExePath + "\sharefile.txt" For Input As #1)

Do
   Input #1,text
   If Len(text) Then
      Print text;
   EndIf
Loop Until InKey = " "
Tourist Trap
Posts: 2880
Joined: Jun 02, 2015 16:24

Re: About sharing data between running executables

Postby Tourist Trap » Jul 18, 2015 11:05

grindstone wrote:
marcov wrote:But my point was more that filebased is hard to do right
Not at all. You only have to open a file for writing in the emitting process and write the data to it, and open that same file in the receiving process for reading and read it periodically. That's all. No library, no care for timing, the only thing that both processes have to know is the filename. It's nearly foolproof.

I have the same fear than marcov here. If one program accessing the common file doesn't release its handle on it (after a crash for example) you can get stuck. I've observated this even with adobe pdf reader, leaving a locked file standing by until you restart the system. That's why sending every data to one pool where it is processed to disk by one process only seems so appealing. Don't you think?
grindstone
Posts: 726
Joined: May 05, 2015 5:35
Location: Germany

Re: About sharing data between running executables

Postby grindstone » Jul 18, 2015 11:17

I can't confirm that. Whren I let the receiver crash by inserting

Code: Select all

? text[100000000]
at the end of the code, the file is deleted by the transmitter as soon as the receiver has terminated, even though it's not been closed. And when I let the transmitter crash, the receiver can be ended with no problems.
speedfixer
Posts: 398
Joined: Nov 28, 2012 1:27
Location: California

Re: About sharing data between running executables

Postby speedfixer » Jul 19, 2015 16:04

re: file based:

Marcov is correct - a proper lock/notice system must be used.

I use a filename as a lock flag. When a module desires access, it renames the flagfile. A success of the rename function *from the OS* means ONLY this process owns the flag file - which will lock out any other program/module from access. No competition.

Any other process will have to wait for the flagfile to become available. Error traps and timeouts put in this process waiting code.

When the module completes its function in the common data file, it renames the flagfile. You can make this more or less complex.
Almost every Linux already has ramdrives available to you.

No external libraries needed.

David

VERY late edit:

I failed to mention:
I use linux only. Modern linux makes memory files VERY easy to use.
I can share/map locations between systems simply which means no network management.
For me - all is very fast with no memory/network to manage.
dnb
Last edited by speedfixer on Sep 06, 2017 20:44, edited 1 time in total.
vdecampo
Posts: 2982
Joined: Aug 07, 2007 23:20
Location: Maryland, USA
Contact:

Re: About sharing data between running executables

Postby vdecampo » Jul 20, 2015 15:00

Also consider using a named semaphore which is like a mutex but accessible to all processes. The you can use WaitForSingleObject to get a lock on it. You can then set a flag in the mapped file to indicate changes to the data.

-Vince

Return to “General”

Who is online

Users browsing this forum: No registered users and 3 guests