[SOLVED] turning working KNN code into a static library

New to FreeBASIC? Post your questions here.
Post Reply
ron77
Posts: 212
Joined: Feb 21, 2019 19:24

[SOLVED] turning working KNN code into a static library

Post by ron77 »

hello :-/
can you pass on a function as a parameter of another function in FB?

python code source https://machinelearningmastery.com/impl ... ch-python/

code in python:

Code: Select all

# Evaluate regression algorithm on training dataset
def evaluate_algorithm(dataset, algorithm):
	test_set = list()
	for row in dataset:
		row_copy = list(row)
		row_copy[-1] = None
		test_set.append(row_copy)
	predicted = algorithm(dataset, test_set)
	print(predicted)
	actual = [row[-1] for row in dataset]
	rmse = rmse_metric(actual, predicted)
	return rmse
my poor translation to FB:

Code: Select all

 redim SHARED test_set(0) AS DOUBLE
FUNCTION elvaluate_algo(x() AS DOUBLE, y() AS DOUBLE, algorithem AS DOUBLE) AS DOUBLE
   
   FOR i AS INTEGER = 0 TO UBOUND(x) - 1
      IAPPEND test_set(), x(i)
   NEXT
   DIM predicted = algorithem(x(),y(), test_set())
   (don't know!)
END FUNCTION
ANY HELP WILL BE GREATLY APRECIATED
ron77
Last edited by ron77 on Dec 09, 2020 18:46, edited 11 times in total.
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: [help] converting a function from python to FB

Post by Tourist Trap »

ron77 wrote:hello :-/
can you pass on a function as a parameter of another function in FB?
Hi ron77. Answer is yes, as a pointer like in:
https://www.freebasic.net/wiki/ProPgProcedurePointers

In your example however, you will have to specify fully the signature of the "algorithm" function :

Code: Select all

algorithm(dataset, test_set)
What is the type of dataset? What is the type of test_set? What is to be returned?

I never tried to use overload with pointers to functions by the way and I wonder if you can define just many "algorithm" functions as you want the arguments to vary. I think I'll ask a different question elsewhere about this.

To get back to your case, I just use the example of the doc with "algorithm" name instead:

Code: Select all

Function algo(a As dataset_type, b As test_set_type) As return_type
    Return a + b
End Function


Dim algorithm As Function  (As dataset_type, As test_set_type) As return_type= @algo


sub MySUB(byval p as any ptr)
	var algorithm =  cast(Function  (As dataset_type, As test_set_type)
    	print algorithm (3, 2)
end sub

MySUB(algorithm )
I know it may look akward a little until you define dataset_type and so on.
Last edited by Tourist Trap on Oct 21, 2020 13:24, edited 2 times in total.
fxm
Moderator
Posts: 12131
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: [help] converting a function from python to FB

Post by fxm »

Code: Select all

REDIM SHARED test_set(0) AS DOUBLE

TYPE function_type AS FUNCTION(() As DOUBLE, () AS DOUBLE, () AS DOUBLE) As DOUBLE

FUNCTION elvaluate_algo(x() AS DOUBLE, y() AS DOUBLE, BYVAL algorithem AS function_type) AS DOUBLE
    ' .....
    DIM AS DOUBLE predicted = algorithem(x(),y(), test_set())
    ' .....
End Function
ron77
Posts: 212
Joined: Feb 21, 2019 19:24

Re: [help] converting a function from python to FB

Post by ron77 »

thank you fxm and touist trap...

Code: Select all

REDIM SHARED test_set(0) AS DOUBLE

TYPE function_type AS FUNCTION(() As DOUBLE, () AS DOUBLE, () AS DOUBLE) As DOUBLE

FUNCTION elvaluate_algo(x() AS DOUBLE, y() AS DOUBLE, BYVAL algorithem AS function_type) AS DOUBLE
   
   FOR i AS INTEGER = 0 TO UBOUND(x) - 1
      IAPPEND test_set(), x(i)
   NEXT
    DIM AS DOUBLE predicted = algorithem(x(),y(), test_set())
   ??? dont know
END FUNCTION
how do i continue?

part of the code i'm busting my head to convert from python:

Code: Select all

# Calculate root mean squared error
def rmse_metric(actual, predicted):
	sum_error = 0.0
	for i in range(len(actual)):
		prediction_error = predicted[i] - actual[i]
		sum_error += (prediction_error ** 2)
	mean_error = sum_error / float(len(actual))
	return sqrt(mean_error)

# Evaluate regression algorithm on training dataset
def evaluate_algorithm(dataset, algorithm):
	test_set = list()
	for row in dataset:
		row_copy = list(row)
		row_copy[-1] = None
		test_set.append(row_copy)
	predicted = algorithm(dataset, test_set)
	print(predicted)
	actual = [row[-1] for row in dataset]
	rmse = rmse_metric(actual, predicted)
	return rmse
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: [help] converting a function from python to FB

Post by Tourist Trap »

ron77 wrote: how do i continue?
From my part, I don't understand why you replace your DATASET, that looks to be a matrix, by simply one 1-D array of X, and one 1-D array of Y. But I may be missing something?

The most difficult stuff will be to convert the for...next loops of python in FB ones. Passing the functions as argument is not difficult.
Last edited by Tourist Trap on Oct 21, 2020 13:22, edited 1 time in total.
fxm
Moderator
Posts: 12131
Joined: Apr 22, 2009 12:46
Location: Paris suburbs, FRANCE

Re: [help] converting a functions from python to FB

Post by fxm »

Before continuing, perhaps you should study the FreeBASIC language a little more with simple examples.
ron77
Posts: 212
Joined: Feb 21, 2019 19:24

Re: [help] converting a functions from python to FB

Post by ron77 »

i thank you gentlemen for your kind help
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: [help] converting a functions from python to FB

Post by Tourist Trap »

ron77 wrote:i thank you gentlemen for your kind help
Hi ron77,

maybe you can tell us what exactly you try to implement. It may exist a basic version of your python program, or at least a C version. Python has very hi-level syntax (hidding all the real work under the hood) that makes it difficult to translate quickly. So sometimes it can be faster just trying to describe your goal and try to find a way in Basic, C, or just in FB after we understand what is it to be done.
ron77
Posts: 212
Joined: Feb 21, 2019 19:24

Re: [help] converting a functions from python to FB

Post by ron77 »

hello tourist trap :)

here you'll find more about what i'm trying to accomplish:

viewtopic.php?f=14&t=28901

viewtopic.php?f=8&t=28907

like i keep saying any help will be greatly appreciated
Tourist Trap
Posts: 2958
Joined: Jun 02, 2015 16:24

Re: [help] converting a functions from python to FB

Post by Tourist Trap »

ron77 wrote: like i keep saying any help will be greatly appreciated
Hi,

If it will be only about simple linear regression, you'll find good examples from dodicat (and maybe others) from this forum, done so in Freebasic. Myself I did this this way :

Code: Select all

'linear regression UDT

#include "fbgfx.bi"

const as integer    scrW => 600
const as integer    scrH => 450
randomize TIMER


type XYFEEDER
   'allow linear regression on a static subset
    declare constructor()
    declare constructor(byval as integer, byval as integer)
    declare operator let(byval as integer)
    declare sub AddInstanceToArray()
    declare sub DrawRegLine()
    declare sub DrawRegLineAt(byval as integer, byval as integer)
        as integer  _x
        as integer  _y
    declare static sub CleanUpArray()
    declare static function RefreshSum() as boolean
    declare static function RefreshMean() as boolean
    declare static function ComputeLinearRegression() as boolean
    static as integer       sumX
    static as integer       sumY
    static as double      meanX
    static as double      meanY
    static as double        beta0
    static as double        slope
    static as double        regQualityParameter
    static as integer       arrayCount
    static as XYFEEDER ptr  xyFeederArrayOfPtr(any)
end type
dim as integer      XYFEEDER.sumX
dim as integer      XYFEEDER.sumY
dim as double      XYFEEDER.meanX
dim as double      XYFEEDER.meanY
dim as double      XYFEEDER.beta0
dim as double      XYFEEDER.slope
dim as double      XYFEEDER.regQualityParameter
dim as integer      XYFEEDER.arrayCount
dim as XYFEEDER ptr   XYFEEDER.xyFeederArrayOfPtr(any)
constructor XYFEEDER()
    with THIS
        ._x => 0
        ._y => 0
    end with
end constructor
constructor XYFEEDER(byval X as integer, byval Y as integer)
    with THIS
        ._x => X
        ._y => Y
    end with
end constructor
operator XYFEEDER.let(byval LetValue as integer)
    THIS._x = LetValue
    THIS._y = LetValue
end operator
sub XYFEEDER.AddInstanceToArray()
    XYFEEDER.arrayCount += 1
    redim preserve _
    XYFEEDER.xyFeederArrayOfPtr(uBound(XYFEEDER.xyFeederArrayOfPtr) + 1)
    XYFEEDER.xyFeederArrayOfPtr(uBound(XYFEEDER.xyFeederArrayOfPtr)) => @THIS
end sub
sub XYFEEDER.DrawRegLine()
   line (THIS._x, THIS._y)-_
       (THIS._x + 20, XYFEEDER.beta0 + XYFEEDER.slope*(THIS._x + 20)), _
       rgb(100,230,100)
end sub
sub XYFEEDER.DrawRegLineAt(byval X as integer, byval Y as integer)
   line (X, Y)-_
       (X + 180, XYFEEDER.beta0 + XYFEEDER.slope*(X + 180)), _
       rgb(200,230,100)
end sub
sub XYFEEDER.CleanUpArray()
    erase XYFEEDER.xyFeederArrayOfPtr
    XYFEEDER.arrayCount = 0
end sub
function XYFEEDER.RefreshSum() as boolean
   dim as boolean  returnValue
   '
    XYFEEDER.sumX = 0
    XYFEEDER.sumY = 0
    for index as integer = lBound(XYFEEDER.xyFeederArrayOfPtr) to _
                        uBound(XYFEEDER.xyFeederArrayOfPtr)
        sumX += XYFEEDER.xyFeederArrayOfPtr(index)->_x
        sumY += XYFEEDER.xyFeederArrayOfPtr(index)->_y
    next index
    '---->
    return returnValue
end function
function XYFEEDER.RefreshMean() as boolean
   dim as boolean  returnValue
   '
   XYFEEDER.RefreshSum()
   XYFEEDER.meanX = XYFEEDER.sumX/XYFEEDER.arrayCount
   XYFEEDER.meanY = XYFEEDER.sumY/XYFEEDER.arrayCount
    '---->
    return returnValue
end function
function XYFEEDER.ComputeLinearRegression() as boolean
    dim as boolean  returnValue
    XYFEEDER.RefreshMean()
    dim as double   num => 0
    for index as integer = lBound(XYFEEDER.xyFeederArrayOfPtr) to uBound(XYFEEDER.xyFeederArrayOfPtr)
       num += _
       (XYFEEDER.xyFeederArrayOfPtr(index)->_x - XYFEEDER.meanX)*XYFEEDER.xyFeederArrayOfPtr(index)->_y
    next index
    dim as double   den => 0
    for index as integer = lBound(XYFEEDER.xyFeederArrayOfPtr) to uBound(XYFEEDER.xyFeederArrayOfPtr)
       den += _
       (XYFEEDER.xyFeederArrayOfPtr(index)->_x - XYFEEDER.meanX)^2
    next index
    XYFEEDER.slope = num/den
    XYFEEDER.beta0 = XYFEEDER.meanY - XYFEEDER.slope*XYFEEDER.meanX
   '
   num = 0
    for index as integer = lBound(XYFEEDER.xyFeederArrayOfPtr) to uBound(XYFEEDER.xyFeederArrayOfPtr)
       num += (XYFEEDER.xyFeederArrayOfPtr(index)->_x - XYFEEDER.meanX)* _
             (XYFEEDER.xyFeederArrayOfPtr(index)->_y - XYFEEDER.meanY)
    next index   
   dim as double den1 => den
   dim as double den2 => 0
    for index as integer = lBound(XYFEEDER.xyFeederArrayOfPtr) to uBound(XYFEEDER.xyFeederArrayOfPtr)
       den2 += _
       (XYFEEDER.xyFeederArrayOfPtr(index)->_y - XYFEEDER.meanY)^2
    next index
   XYFEEDER.regQualityParameter = num/sqr(den1* den2)
    '---->
    return returnValue
end function


'___------------------------------------------------------------___
'___------------------------------------------------------------___
screenRes scrW, scrH, 32

dim as XYFEEDER       xy(1 to 10)

for index as integer = 1 to 10
   with xy(index)
      ._x => index*50
      ._y => scrH/2 + rnd()*(scrH/8) + 4*index
      circle (._x, ._y), 4
      xy(index).AddInstanceToArray()
      XYFEEDER.ComputeLinearRegression()
      xy(index).DrawRegLine()
      ? "beta0", XYFEEDER.beta0, "r", XYFEEDER.regQualityParameter
      draw string (._x - 8, ._y - 10), str(index) &","& left(str(XYFEEDER.slope*100), 4)
   end with
next index

xy(1).DrawRegLineAt(0, XYFEEDER.beta0)
circle (0, XYFEEDER.beta0), 8
circle (0, XYFEEDER.beta0), 4

XYFEEDER.CleanUpArray()

getKey()

'(eof)
If you really want to translate your Python code anyway. That's not a bad exercice at all. But you'll have to make things by steps.
[*]1 - Define your datatypes in input - will you use user defined type of simple arrays?
[*]2 - Learn how to mimic FOR..LOOPS of Python using features of FB
[*]3 - Put all the stuff together, and that's it.

For now I think you still will have to deal with point 2. I'll see later if I can adapt an implementation of mine of a FOR_EACH loop. It can be funny to do :)
ron77
Posts: 212
Joined: Feb 21, 2019 19:24

Re: [help] converting functions from python to FB

Post by ron77 »

thank you so much :) i'll take a look at the the forum you yet i'm committed and egger to continue converting the python code to FB...

i wasn't sure if anyone else did linear regression in FB before but i suspected that it must have been done before me yet i'm hoping to make a linear regression ML library in FB...


as for what you said in your replay:
1. the input will be by csv files containing the data and will be loaded into the program by UDT OOP (contribute of Paul doe)
2. point to is correct i will need help in it and also in translating the python code to FB
3.well the intension is to make a library of ML linear regression in FB so when (hopefully) we'll have a working linear regression program code in FB that's just one step the next is to make out of it a library (libLINREG) with a bi file and a libLINREG.a file so any one in FB won't have to code this headache over and over when trying to add some linear regression calculation to his data / program


p.s. - this is almost the full code that needed to be converted to FB

Code: Select all

# Standalone simple linear regression example
from math import sqrt

# Calculate root mean squared error
def rmse_metric(actual, predicted):
	sum_error = 0.0
	for i in range(len(actual)):
		prediction_error = predicted[i] - actual[i]
		sum_error += (prediction_error ** 2)
	mean_error = sum_error / float(len(actual))
	return sqrt(mean_error)

# Evaluate regression algorithm on training dataset
def evaluate_algorithm(dataset, algorithm):
	test_set = list()
	for row in dataset:
		row_copy = list(row)
		row_copy[-1] = None
		test_set.append(row_copy)
	predicted = algorithm(dataset, test_set)
	print(predicted)
	actual = [row[-1] for row in dataset]
	rmse = rmse_metric(actual, predicted)
	return rmse

# Calculate the mean value of a list of numbers
def mean(values):
	return sum(values) / float(len(values))

# Calculate covariance between x and y
def covariance(x, mean_x, y, mean_y):
	covar = 0.0
	for i in range(len(x)):
		covar += (x[i] - mean_x) * (y[i] - mean_y)
	return covar

# Calculate the variance of a list of numbers
def variance(values, mean):
	return sum([(x-mean)**2 for x in values])

# Calculate coefficients
def coefficients(dataset):
	x = [row[0] for row in dataset]
	y = [row[1] for row in dataset]
	x_mean, y_mean = mean(x), mean(y)
	b1 = covariance(x, x_mean, y, y_mean) / variance(x, x_mean)
	b0 = y_mean - b1 * x_mean
	return [b0, b1]

# Simple linear regression algorithm
def simple_linear_regression(train, test):
	predictions = list()
	b0, b1 = coefficients(train)
	for row in test:
		yhat = b0 + b1 * row[0]
		predictions.append(yhat)
	return predictions

# Test simple linear regression
dataset = [[1, 1], [2, 3], [4, 3], [3, 2], [5, 5]]
rmse = evaluate_algorithm(dataset, simple_linear_regression)
print('RMSE: %.3f' % (rmse))
:)
ron77
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: [help] converting code from python to FB

Post by MrSwiss »

ron77 wrote:can you pass on a function as a parameter of another function in FB?
This clearly indicates what is commonly known as a callback function/procedure.
You'll find discussions about it by searching the term: callback.

There is by contrast, the method of layering procedures (proc. = Sub or Function).
Main aim here is, to prevent code duplication ...

A explained example is found in Tips & Tricks section:
Time_t - layering code & encapsulation
ron77
Posts: 212
Joined: Feb 21, 2019 19:24

Re: [help] converting code from python to FB

Post by ron77 »

hello MrSwiss :)

i thank you for your help and i invite you to join in the project: viewtopic.php?f=8&t=28907

ron77
MrSwiss
Posts: 3910
Joined: Jun 02, 2013 9:27
Location: Switzerland

Re: [help] converting code from python to FB

Post by MrSwiss »

Hi ron77,

thanks for the invitation but no thank you (python isn't even on my list of scripting-languages).
All scripting-languages I've ever used:
  • PHP (server side) and JavaScript (client side) for web development
    LUA for process automation within compiled FB or other code
badidea
Posts: 2591
Joined: May 24, 2007 22:10
Location: The Netherlands

Re: [help] converting code from python to FB

Post by badidea »

Don't use 0 TO UBOUND(x) - 1 if you want to loop all values in x().
Post Reply