Inside each neuron there is a bias (b In between any 2 neighboring neuron layers a set of weights are found X W ●wt=N Input layer Output layer x(i=l)o x(i=2)(=2) x(=)m( Neural Networks Ch9, ver. 9b
Inside each neuron there is a bias (b) • In between any 2 neighboring neuron layers, a set of weights are found Neural Networks Ch9. , ver. 9b 16 x(i =1) y w(i = 1) u f (u) w(I) x(i = 2) w(i = 2) x(i = I)
Inside each neuron X=input, y=output X( (=2)1(=2) x(=)w( y=f( u)with u=∑v(x小+b, b=bias, x=input, w= weight, u= internal signal Typically fo is a logistic(sigmod)function,i. e f(u) assume B=l for simplicity 1+e therefore=f(u) )x(+b 1+e Neural Networks Ch9, ver. 9b
Inside each neuron x=input, y=output Neural Networks Ch9. , ver. 9b • 17 − + − = = = = + = = = + = = = = = = = + i I i i x i u i I i e y f e f u f b x w u y f u w(i)x(i) b 1 ( ) ( ) b 1 1 1 therefore (u) ,assume 1for simplicity, 1 1 ( ) Typically ()is a logistic (sigmod) function, i.e. bias, input, weight, internalsignal (u)with , x(i = 1) y w(i = 1) u f (u) w(I) x(i = 2) w(i = 2) x(i = I)
Sigmoid function flu)= logsiglu) and its derivative f(u=dlogsiglu) Neural Networks ch9, ver. 9b 1+e-m1+e or simplicity set B df(u) http://mathworld.wolframcom/sigmoidFunctionhtml note Logistic sigmoid(logsig) au dx https://kawahara.ca/how-to-compute-the-derivative-of-a- hence sigmoid-function-fully-worked-examplel df(a)(1+e“丿d(1+e 1/(1+e) f(u)= (using chain rule) htd(1+e“)a e e e +e-)}+e (1+e)(1+e“)(1+e-“) +e f()-f(l) e us (n)=f(l)-f(n) http:/ink.springercom/chapter/10.1007%2b3-540-59497-3175#page-1,httpsimiloainfwordpresscom/2013/11/06/rectifier-nonlinearities/
Sigmoid function f(u)= logsig(u) and its derivative f’(u)=dlogsig(u) • Neural Networks Ch9. , ver. 9b 18 ( ) ( ) ( )(1 ( )) ( ) Thus, ( ) 1 ( ) (1 ) 1 1 (1 ) 1 (1 ) (1 ) (1 ) (1 ) 1 (1 ) (1 ) 1 (1 ) 1 ( ) (1 ) 1 ( ) ,(using chain rule) (1 ) (1 ) 1 1 ( ) ( ) ( ) ( ) , for simplicity set 1 1 1 1 1 ( ) ' 2 2 ' ' ' f u f u f u du df u f u f u e e e e e e e e e e e e e e f u du d e d e e d du df u f u Hence f u du df u e e f u u u u u u u u u u u u u u u u u u u u = = − = − + − + = + + − + + + = + + = + − = + − = + + + = = = = + = + = − − − − − − − − − − − − − − − − − − − http://link.springer.com/chapter/10.1007%2F3-540-59497-3_175#page-1 , https://imiloainf.wordpress.com/2013/11/06/rectifier-nonlinearities/ http://mathworld.wolfram.com/SigmoidFunction.html Logistic sigmoid (logsig) https://kawahara.ca/how-to-compute-the-derivative-of-asigmoid-function-fully-worked-example/ x x e dx de note : =
Back Propagation Neural Net ( BPnn Forward pass Forward pass is to find the output when an input is given For example: Assume we have used N=60,000 images MNIST database)to train a network to recognize c=10 numerals When an unknown image is given to the input, the output neuron corresponds to the correct answer will give the highest output level l=1 X 0 0 Input mage W 8 Input layer Output layer Hidden layers 10 output neurons for 0, 1, 2, .. 9 Neural Networks Ch9
Back Propagation Neural Net (BPNN) Forward pass • Forward pass is to find the output when an input is given. For example: • Assume we have used N=60,000 images (MNIST database) to train a network to recognize c=10 numerals. • When an unknown image is given to the input, the output neuron corresponds to the correct answer will give the highest output level. Neural Networks Ch9. , ver. 9b 19 10 output neurons for 0,1,2,..,9 Input image 0 0 0 1 0 0
Our simple demo program ° Training pattern age/Class #1 Tran image/Class #1 Tran image/Class #1 3 classes ( in 3 rows class1 Each class has 3 training Train imageless #2 Tran image Class #2 Tren image/Class#? samples(items in each class2 row) Train image/Class 43 After training an input class3 (assume it is test image #2 )is presented to the network the network should tell you it is class 2, etc (class 2) Unknown Inp Neural Networks Ch9. ver. 9b
Our simple demo program • Training pattern – 3 classes (in 3 rows) – Each class has 3 training samples (items in each row) • After training , an input (assume it is test image #2) is presented to the network, the network should tell you it is class 2, etc. Neural Networks Ch9. , ver. 9b 20 class1 class2 class3 Result :image (class 2) Unknown input