Neural Networks 1
Neural Networks 1
To be updated from time to time.
Re: Neural Networks 1
Refer to Beginners, Easy Matrix for other routines, include
files and library code .
As an introduction I'd suggest you look at this series
of videos.
https://www.youtube.com/watch?v=aircAruvnKk
To a fair extent I'm modelling my NN programs using
that information.
The MNIST handwritten numerals data, mentioned there
is something I've already successfully trialed using
python; eventually I might use this with the BASIC
program.
In the meantime though, I intend to use a much smaller
data set to check the validity of my code, this will
consist of a binary representation of unsigned integers.
The format of the target values will also be a binary
representation of integers.
I don't claim that the Matrix routines in use are perfect.
However much of the code I build upon that has some worth,
even if it's instructional .
Also from this video :
https://www.youtube.com/watch?v=Ilg3gGewQ5U
Others have been suggested by gunslinger .
https://www.youtube.com/watch?v=hfMk-kjRv4c
files and library code .
As an introduction I'd suggest you look at this series
of videos.
https://www.youtube.com/watch?v=aircAruvnKk
To a fair extent I'm modelling my NN programs using
that information.
The MNIST handwritten numerals data, mentioned there
is something I've already successfully trialed using
python; eventually I might use this with the BASIC
program.
In the meantime though, I intend to use a much smaller
data set to check the validity of my code, this will
consist of a binary representation of unsigned integers.
The format of the target values will also be a binary
representation of integers.
I don't claim that the Matrix routines in use are perfect.
However much of the code I build upon that has some worth,
even if it's instructional .
Code: Select all
'
' bit_rep2.bas
'
' Numerical representation of bits.
'
' and
'
' Binary representation of numbers.
'
'
#cmdline "-exx"
#include once "easy_matx.bi"
'
' next include, not required for this frame work .
'
' #include once "easy_maty.bi"
'
declare sub int2bit(bsq as Matrix, x as uinteger, nb as integer)
declare function ubxlb(x as uinteger, lb as integer) as uinteger
declare function bits2int(bsq as Matrix, nb as integer) as uinteger
'
' ----------------------------------------------------------------------
'
dim as uinteger v(0 to 3), w(0 to 3), x(0 to 15), y(0 to 15), i, k, m
'
dim as integer la, pwr
la = 2
'pwr = (2^(la)-1)^2
pwr = 2^(2*la)-1
dim as Matrix ab = Matrix(1, la*2)
'
' Sequence through bit representations .
'
for i=0 to pwr
' print " ";i;
int2bit(ab , i, la ) ' input
prt_m(ab)
print " x "
m = ubxlb(i , la)
int2bit(ab , m, la ) ' target, convert
prt_m(ab)
'
' train , retain previous weights, to adjust then save for
' next [ input , target ] pair .
' output , convert, check .
'
next i
print
print " ---------------------------------------------------------- "
m = bits2int(ab, la)
print " m = ";m
'
' For all valid input data samples .
' test data -> pretrained NN -> output , convert, compare expected.
end
'
' ======================================================================
'
'
'
' |0|1||2|3|
' 1 2 4 8
'
end
'
' ======================================================================
'
'
function bits2int(bsq as Matrix, nb as integer) as uinteger
'
' Matrix bits to uinteger .
'
dim as uinteger x
dim as single bt
dim as integer i, j, nx, ny ' , lb, ub
'
nx = ubound(bsq.m, 1)
ny = ubound(bsq.m, 2)
'
for j = 0 to ny
for i=0 to nx
bt = bsq.m(i,j)
bt = int(bt + 0.5)
bt = bt*(2^j)
x = x + bt
next i
next j
'
return x
'
end function
' ----------------------------------------------------------------------
'
sub int2bit(bsq as Matrix, x as uinteger, nb as integer)
'
' Convert an integer to bits of length lb
' , assign to matrix elements .
'
dim as integer i, j, nx, ny ' , lb, ub
'
nx = ubound(bsq.m, 1)
ny = ubound(bsq.m, 2)
'
for j = 0 to nb-1
for i=0 to nx
bsq.m(i,j) = -Bit(x,j)
bsq.m(i,j+nb) = -Bit(x,j+nb)
next i
next j
'
'
end sub
'
' ----------------------------------------------------------------------
'
function ubxlb(x as uinteger, lb as integer) as uinteger
'
' Upper bits x Lower bits , multiplication.
'
dim as integer pwr
dim as uinteger a, b, c
'
pwr = 2^lb
'
b = int(x/pwr)
a = x - pwr*b
c = a * b
' print " ";a;" , ";b;" , ";c
'
return c
'
'
end function
Also from this video :
https://www.youtube.com/watch?v=Ilg3gGewQ5U
Others have been suggested by gunslinger .
https://www.youtube.com/watch?v=hfMk-kjRv4c
Re: Neural Networks 1
I used the previous code to train, then test my
existing NN .
It tends to remember the last training epoch
using those weights, as the default; irrespective of the input.
Somethings is amok.
If this is over fitting, then there are ways to
compensate for this; ways I'm not yet familiar
with.
Early Termination looks like one possibility .
Time to examine NN code from different languages and replicate
the previous training and test arrangement within those.
With FreeBASIC I only need to read about a method and I'm able to
eventually visualize what needs to be done.
existing NN .
It tends to remember the last training epoch
using those weights, as the default; irrespective of the input.
Somethings is amok.
If this is over fitting, then there are ways to
compensate for this; ways I'm not yet familiar
with.
Early Termination looks like one possibility .
Time to examine NN code from different languages and replicate
the previous training and test arrangement within those.
With FreeBASIC I only need to read about a method and I'm able to
eventually visualize what needs to be done.