Neural Networks 1

New to FreeBASIC? Post your questions here.
Post Reply
Luxan
Posts: 222
Joined: Feb 18, 2009 12:47
Location: New Zealand

Neural Networks 1

Post by Luxan »

To be updated from time to time.
Luxan
Posts: 222
Joined: Feb 18, 2009 12:47
Location: New Zealand

Re: Neural Networks 1

Post by Luxan »

Refer to Beginners, Easy Matrix for other routines, include
files and library code .


As an introduction I'd suggest you look at this series
of videos.


https://www.youtube.com/watch?v=aircAruvnKk

To a fair extent I'm modelling my NN programs using
that information.
The MNIST handwritten numerals data, mentioned there
is something I've already successfully trialed using
python; eventually I might use this with the BASIC
program.
In the meantime though, I intend to use a much smaller
data set to check the validity of my code, this will
consist of a binary representation of unsigned integers.
The format of the target values will also be a binary
representation of integers.

I don't claim that the Matrix routines in use are perfect.
However much of the code I build upon that has some worth,
even if it's instructional .

Code: Select all

'
'  bit_rep2.bas
'
'   Numerical representation of bits.
'
'             and
'
'   Binary representation of numbers.
'
'
#cmdline "-exx"
#include once "easy_matx.bi"
'
' next include, not required for this frame work .
'
' #include once "easy_maty.bi"
'
declare sub int2bit(bsq as Matrix, x as uinteger, nb as integer)
declare function ubxlb(x as uinteger, lb as integer) as uinteger

declare function bits2int(bsq as Matrix, nb as integer) as uinteger
'
' ----------------------------------------------------------------------
'
dim as uinteger v(0 to 3), w(0 to 3), x(0 to 15), y(0 to 15), i, k, m
'
dim as integer la, pwr
la = 2
'pwr = (2^(la)-1)^2
pwr = 2^(2*la)-1
dim as Matrix ab = Matrix(1, la*2)
'
'             Sequence through bit representations .
'
for i=0 to pwr 
    ' print " ";i;
     int2bit(ab , i, la ) ' input
     prt_m(ab)
     print "  x  "
     m = ubxlb(i , la)
     int2bit(ab , m, la ) ' target, convert
     prt_m(ab)    
     '
     '  train , retain previous weights, to adjust then save for 
     ' next [ input , target ] pair .
     '  output , convert, check .
     ' 
  next i

print
print " ---------------------------------------------------------- "

m = bits2int(ab, la) 
print " m = ";m
'
'  For all valid input data samples .
'  test data -> pretrained NN -> output , convert, compare expected.


end
'
' ======================================================================
'
'
'
'   |0|1||2|3|
'    1 2  4 8
'
end
'
' ====================================================================== 
'
'
function bits2int(bsq as Matrix, nb as integer) as uinteger
'
'               Matrix bits to uinteger .  
'
dim as uinteger x
dim as single bt
dim as integer i, j, nx, ny ' , lb, ub
'
nx = ubound(bsq.m, 1)
ny = ubound(bsq.m, 2)
'
for j = 0 to ny
 for i=0 to nx 
     bt = bsq.m(i,j)
     bt = int(bt + 0.5)
     bt = bt*(2^j)
     x = x + bt 
 next i
next j 
' 
                 return x
'
end function

' ----------------------------------------------------------------------
'
sub int2bit(bsq as Matrix, x as uinteger, nb as integer)
'
'         Convert an integer to bits of length lb
'     , assign to matrix elements .
'
dim as integer i, j, nx, ny ' , lb, ub
'
nx = ubound(bsq.m, 1)
ny = ubound(bsq.m, 2)
'
for j = 0 to nb-1
 for i=0 to nx 
    bsq.m(i,j) = -Bit(x,j) 
    bsq.m(i,j+nb) = -Bit(x,j+nb) 
 next i
next j 
'
'
end sub
'
' ----------------------------------------------------------------------
'
function ubxlb(x as uinteger, lb as integer) as uinteger
'
'   Upper bits x Lower bits , multiplication.
'
dim as integer pwr
dim as uinteger a, b, c
'
           pwr = 2^lb
'
             b = int(x/pwr)
             a = x - pwr*b
             c = a * b
          '   print " ";a;" , ";b;" , ";c
'
    return c
'
'
end function


Also from this video :

https://www.youtube.com/watch?v=Ilg3gGewQ5U


Others have been suggested by gunslinger .

https://www.youtube.com/watch?v=hfMk-kjRv4c
Luxan
Posts: 222
Joined: Feb 18, 2009 12:47
Location: New Zealand

Re: Neural Networks 1

Post by Luxan »

I used the previous code to train, then test my
existing NN .
It tends to remember the last training epoch
using those weights, as the default; irrespective of the input.
Somethings is amok.

If this is over fitting, then there are ways to
compensate for this; ways I'm not yet familiar
with.
Early Termination looks like one possibility .

Time to examine NN code from different languages and replicate
the previous training and test arrangement within those.

With FreeBASIC I only need to read about a method and I'm able to
eventually visualize what needs to be done.
Post Reply