MyTerminal threads prog

For issues with communication ports, protocols, etc.
Post Reply
dasyar
Posts: 372
Joined: Dec 04, 2008 15:31

MyTerminal threads prog

Post by dasyar »

Below is prototype, console mode, of a freeBasic program. It is a general outline of a Python program which I am trying to replicate in freeBasic.

A brief overview, I have a device that is streaming data every thirty seconds via the serial port. The Python program, within a thread, is capturing the data and is adding a date and time to a line of data which is then appended to a csv file. I am also working on a way of analyzing the data that is in the file, probably will be using a spreadsheet.

The Python program, in the foreground, can start a log file, stop a log file, capture an instance of incoming data, and other things. I am trying to see if I can replicate this in freeBasic console mode, then if I am satisfied with the results try, to make it a GUI program

Thanks

Code: Select all

' tthread.bas
'
' November 5, 2017
'

Dim Shared As String inBuff, buffer
Dim As String baud,port



baud = "115200"
port = "/dev/ttyUSB0:"

Open Com port & baud & ",n,8,1,cs0,ds0,cd0,rs" As #1
If Err <> 0 Then
	Print "Error opening", port
End If
Print port+" open"


' Thread for incoming data stream
' Runs in the background???
Sub commthread(param As Any Ptr)
    print "mythread is running"

'Open "test1.csv" For Append As #2 
Open "test1.csv" For Append As #2   
    ' Keep the thread running
    do
    
		' Check Com port for activity
		While LOC(1) > 0
			
			buffer = Input(LOC(1),#1)			
			
		Wend
		

    sleep 200,1
    'Put #2, ,buffer
    
    loop
End Sub

'' Launch commthread thread
Dim As Any Ptr thread1 = ThreadCreate(@commthread, 0, 64)

'' Main
' Runs in the foreground???
Print
do
' Terminal user I/O
	input "> ",inBuff
	If inBuff = "quit" Then
		Exit Do
	Elseif inBuff = "data" then
		Print buffer  ' Print out what is in the com buffer
	Else
		Print " Invalid Command"
	End If
	
	Sleep 1,0
loop until multikey(&H01)  ' Esc to end program

Print "Program End!"
Close #1
End
dasyar
Posts: 372
Joined: Dec 04, 2008 15:31

Re: MyTerminal threads prog

Post by dasyar »

It seems that I have run into problem with this program. The main point of this program was to be able to capture a data stream and have it stored to a csv file, to be used like a database, using LibreOffice Calc. I guess maybe I posted this program much to early, but I guess somebody might get some use out of it, in terms of an idea as to how to use a thread(s).
dasyar
Posts: 372
Joined: Dec 04, 2008 15:31

Re: MyTerminal threads prog

Post by dasyar »

In another thread I got a solution for the serial Com data streaming, which I added to this program. I gave the program a quick test and it looks like it is working as expected. The created csv file also works as expected when I open it up in a calc program, all the like items are in a separate column.

The only problem that showed up is with my data command, because I have the buffer2 = "", in my thread, the data cannot be accessed within my foreground UI. One way of getting around this is if I could get the last entry in my csv file, that should be a recent enough snapshot of the data stream. I need some input from the experts on file manipulation as to how this could be done.

Code: Select all

' tthread.bas
'
' November 5, 2017
'

Dim Shared As String buffer,buffer2
Dim As String inBuff,buffer1
Dim As String baud,port
Dim Shared As Long ST1 = 0  ' Start logfile


baud = "115200"
port = "/dev/ttyUSB0:"

Open Com port & baud & ",n,8,1,cs0,ds0,cd0,rs" As #1
If Err <> 0 Then
	Print "Error opening", port
End If
Print port+" open"


' Thread for incoming data stream
' Runs in the background???
Sub commthread(param As Any Ptr)
    print "mythread is running"

'Open "test1.csv" For Append As #2   
    ' Keep the thread running
    do
    If ST1 = 0 Then    ' If ST1 = 1, start logging
		goto breakout
    End If
    Open "test1.csv" For Append As #2 
		' This is the capture and log
		do
			buffer2 += Input(1,#1)
			sleep 5,1
		loop until Right(buffer2,2) = Chr(13,10)
		Print #2,date;",";time;",";buffer2;		
		buffer2 = ""
		
		Close #2
    sleep 200,1

	breakout:
    loop
End Sub

'' Launch commthread thread
Dim As Any Ptr thread1 = ThreadCreate(@commthread, 0, 64)

'' Main
' Runs in the foreground???
Print
do
' Terminal user I/O
	input "> ",inBuff
	If inBuff = "quit" Then
		Exit Do
	Elseif inBuff = "data" then
		Print Date+" "+Time
		'Print buffer2  ' Print out what is in the com buffer
		'buffer = ""
		'Open "test1.csv" For Output As #3 
		'Get #3,40, buffer
		'Print buffer
		'Close #2
	Elseif inBuff = "startlog" Then
		ST1 = 1
		Print "Log started"
		Print Date
		Print Time
	Elseif inBuff = "stoplog" Then
		ST1 = 0
		Print "Log Stopped"
		Print Date
		Print Time
	Elseif inBuff = "help" Then
		Print "Menu - quit, help, data, startlog, stoplog"
	Else
		Print " Invalid Command"
	End If
	
	Sleep 1,0
loop until multikey(&H01)  ' Esc to end program

Print "Program End!"
Close #1
End
grindstone
Posts: 862
Joined: May 05, 2015 5:35
Location: Germany

Re: MyTerminal threads prog

Post by grindstone »

Do you mean something like this?

Code: Select all

Dim As String sample
Open "xyz.csv" For Binary Access Read As #1
Seek #1, Lof(1) - 5000 'set file pointer to end-of-file - 5000
sample = Input(5000, #1) 'get the last 5000 bytes
Close #1
dasyar
Posts: 372
Joined: Dec 04, 2008 15:31

Re: MyTerminal threads prog

Post by dasyar »

@grindstone, thanks for all the help.

I added the example code to the data command, and it is working as expected. I am getting very close to taking my Python program offline, and using this freeBasic program.

I am now wondering if freeBasic could be used to do some data analysis on my csv file. With the Python program I am using a library called Pandas, which provides some analysis tools. I am not sure if there is a bi file with some similar capabilities, or would I have to create my own stuff. It would be kind of nice, if within my program I could do some data analysis. That would make it more convenient, that way I would not have to deal with a database or spreadsheet program. Any ideas?

Code: Select all

' tthread.bas
'
' November 5, 2017
'

Dim Shared As String buffer,buffer2,logname, dateit, timeit
Dim As String inBuff,buffer1
Dim As String baud,port
Dim Shared As Long ST1 = 0  ' Start logfile


logname = "1117"  ' Temp location, will add logname command
baud = "115200"
port = "/dev/ttyUSB0:"

Open Com port & baud & ",n,8,1,cs0,ds0,cd0,rs" As #1
If Err <> 0 Then
	Print "Error opening", port
End If
Print port+" open"


' Thread for incoming data stream
' Runs in the background???
Sub commthread(param As Any Ptr)
    print "mythread is running"

 
    ' Keep the thread running
    do
    If ST1 = 0 Then    ' If ST1 = 1, start logging
		goto breakout
    End If
    Open "/media/pi/2tbdrive/solstadat/" & logname & "log.csv" For Append As #2 
		' This is the capture and log
		do
			buffer2 += Input(1,#1)
			sleep 5,1
		loop until Right(buffer2,2) = Chr(13,10)
		Print #2,date;",";time;",";buffer2;		
		buffer2 = ""
		
		Close #2
    'sleep 200,1

	breakout:
    loop
End Sub

'' Launch commthread thread
Dim As Any Ptr thread1 = ThreadCreate(@commthread, 0, 64)

'' Main
' Runs in the foreground???
Print "For menu type help"
do
' Terminal user I/O
	input "> ",inBuff
	If inBuff = "quit" Then
		Exit Do
	Elseif inBuff = "data" then
		If ST1 = 0 Then
			Print "Data stream is offline"
		Else
		Open "/media/pi/2tbdrive/solstadat/" & logname & "log.csv" For Binary Access Read As #3 
		Seek #3, Lof(3) - 120
		buffer = Input(120, #3)
		Print buffer
		Close #3
		End If
	Elseif inBuff = "startlog" Then
		ST1 = 1
		Print "Log started"
		Print Date
		Print Time
		timeit = Time
		dateit = Date
	Elseif inBuff = "stoplog" Then
		ST1 = 0
		Print "Log Stopped"
		Print Date
		Print Time
	Elseif inBuff = "logstate" Then
		If ST1 = 0 Then
			Print "Datalog is off"
		End If
		If ST1 = 1 Then
			Print "Datalog is running"
			Print "Log started: "
			Print dateit
			Print timeit
			
		End If
	Elseif inBuff = "time" Then
		Print time
	Elseif inBuff = "date" Then
		Print date
	Elseif inBuff = "help" Then
		Print "Menu - quit, help, data, startlog, stoplog"
		Print "       date, time, logstate "
	Else
		Print " Invalid Command"
	End If
	
	Sleep 1,0
loop until multikey(&H01)  ' Esc to end program

Print "Program End!"
Close #1
End

grindstone
Posts: 862
Joined: May 05, 2015 5:35
Location: Germany

Re: MyTerminal threads prog

Post by grindstone »

One little hint: You can use Continue Do instead of the GoTo - statement. Furthermore I would recommend to insert a Sleep 1 at the end of the loop for saving ressources:

Code: Select all

' Thread for incoming data stream
' Runs in the background???
Sub commthread(param As Any Ptr)
	Print "mythread is running"


	' Keep the thread running
	Do
		If ST1 = 0 Then    ' If ST1 = 1, start logging
			'GoTo breakout '<<<<<<< not needed
			Continue Do '<<<<<<< use this instead
		End If
		Open "/media/pi/2tbdrive/solstadat/" & logname & "log.csv" For Append As #2
		' This is the capture and log
		Do
			buffer2 += Input(1,#1)
			Sleep 5,1
		Loop Until Right(buffer2,2) = Chr(13,10)
		Print #2,Date;",";Time;",";buffer2;
		buffer2 = ""

		Close #2
		'sleep 200,1

		'breakout: '<<<<<<<<<<< not needed
		Sleep 1 '<<<<<<<<<< for saving ressources
	Loop
End Sub
And I'm afraid you'll have to write your own .csv analysis tools.
dasyar
Posts: 372
Joined: Dec 04, 2008 15:31

Re: MyTerminal threads prog

Post by dasyar »

So far the freeBasic program is a very good replication of my Python program, short of some data analysis tools. Now I am going to start the conversion of of the program to a a GUI solution, if it is a practical solution. I am still not sure if it will be more practical than the console version. But as soon as I start implementing some of widgets in a GUI screen, it should become apparent very quickly if this would be a practical and intuitive solution.

Comments are welcome at this point.
dasyar
Posts: 372
Joined: Dec 04, 2008 15:31

Re: MyTerminal threads prog

Post by dasyar »

Just a brief update, since I was unable to work around a problem for the Linux version, I cleaned up the code for a Windows version. I have been running the Windows version for a couple of days without any problems popping up.

I sort of like this freeBasic version better than the Python version because the way it breaks out of a thread. The freeBasic version breaks out of the thread immediately while the Python version has a lingering affect when it tries to break out of a thread.
dasyar
Posts: 372
Joined: Dec 04, 2008 15:31

Re: MyTerminal threads prog

Post by dasyar »

It has been awhile since I worked with freebasic. I looked at this thread and decided to create, sort of a template for working with threading that has a rudimentary UI.

I am working/testing this program on Raspberry Pi, the code should be easily transferrable to other OSs, with a few modifications.

I did mention in the OP that I started with a program that I created in Python, just to see how it compares, in functionality, using freebasic. The slight difference is, in the Python program I use multiprocessing, instead of threading. Freebasic does not have a multiprocessing capability, that I know of. Freebasic threading, it is noted, that if you have "cores" available it will use it.

One test that I did run, in the thread, I changed the 'sleep 1' to 'sleep 1000', and in the UI you can see the affect. The Raspberry Pi has four cores, so I am not sure why I was seeing a ~one second delay in the UI responses. I wonder if freebasic did have a multiprocessing function, if that would make a big difference.

As a side note, I found installing freebasic for the Raspberry Pi, from the downloads section, a very easy install session. The problem I had was installing freebasic for Linux, to be a problem. After installation, of the Linux version, when I tried fbc, the system kept coming up with an error that it could not find fbc or it could not start it.

Code: Select all

'' fb_thread.bas
'' 
'' Jan 03, 2021
'' Sort of a template for the use of threading

dim as string baud,port
baud = "115200"
port = "/dev/ttyS0:"

dim shared as string logdate,logtime, dateit, timeit
dim shared as long ST1 = 0

logdate = date
logtime = time

'' Open and check the status of the comm port
open com port & baud &",n,8,1,cs0,ds0,cd0,rs" as #1
if Err <> 0 then
	print "Error Opening", port
end if
print port+" Open!"

dim as string inBuff

'' Thread 
''
sub commthread(param as any ptr)
	print "Thread is running"
	
	do
		'' Check to see status of log start/stop
		if ST1 = 0 then  '' Start with log stopped
			continue do
		end if
		'' Open xxxxlog.csv in logdata folder
		open "/home/pi/programs/freebasic/logdata/"& logdate & "log.csv" for append as #2
		print #2,date;",";time  '' For testing purpose, insert date and time into .csv file
		close #2
		sleep 1
	loop
end sub

'' Start the thread
dim as any ptr thread1 = ThreadCreate(@commthread, 0, 64)

'' Main
'' This is the UI
do
	input "> ",inBuff
	if inBuff = "quit" then
		exit do
	'' Start the logging in the thread
	elseif inBuff = "startlog" then
		ST1 = 1
		print "Log Started!"
		print date
		print Time
		timeit = Time 
		dateit = Date
	'' Stop the logging in the thread
	elseif inBuff = "stoplog" then
		ST1 = 0
		print "Log Stopped!"
		print date
		print time
	elseif inBuff = "logstate" then
		if ST1 = 0 then
			print "Datalog is off!"
		end if
		if ST1 = 1 then
		print "Datalog is running!"
		print "Log started: "
		print dateit
		print timeit
		end if
	else
		print "Invalid Command!"
	end if

sleep 1
loop until multikey(&H01)  '' Can use the Esc key to stop


Print "Program End!"
sleep 1000  '' Wait one second 
Close #1

End
Post Reply