Basics of anns Network Architecture Hidden Layers/Nodes The number of neurons in the input and output layers is dictated by the physics of the problem The number of hidden layers and the number of nodes in each layer is more art than science More hidden layers make an ann smarter More neurons per layer make an ann more accurate Determined by the complexity of the problem and the number of patterns(pairs of input/output vectors) used for training the ann DCABES. October 2009
DCABES, October 2009 Basics of ANNs Network Architecture Hidden Layers/Nodes • The number of neurons in the input and output layers is dictated by the physics of the problem • The number of hidden layers and the number of nodes in each layer is more art than science! • More hidden layers make an ANN smarter • More neurons per layer make an ANN more accurate • Determined by the complexity of the problem and the number of patterns (pairs of input/output vectors) used for training the ANN
Basics of anns Hidden Layers/Nodes Heuristics D) I or 2 HLs adequate for most problems For the number of nodes various propositions Heuristic ource 2N, or 3M Kanellopoulos and Wilkinson(1997) 3Ni Hush(1989) 2N1+1 Hecht-Nielsen(1987) 2N/3 Wang(1994b) (N2+ND)2 Ripley ( 1993) NPr(Ni+No) Garson(1998) 2+NN+N(M2+M)-3 N+No Paola(1994) N: Number of inputs, N: Number of outputs, Nn: number of training samples r 5, 10] related to the noise of the data DCABES. October 2009
DCABES, October 2009 Basics of ANNs Hidden Layers/Nodes Heuristics (I) • 1 or 2 HLs adequate for most problems • For the number of nodes various propositions: Ni : Number of inputs, No : Number of outputs, Np : number of training samples r [5, 10] related to the noise of the data
Basics of anns Hidden Layers/Nodes heuristics (in) Following Kolmogorov's theorem: 1 HL and (2 N;+ 1)nodes adequate for most problems for 6 inputs 1 HL with 13 nodes DCABES. October 2009
DCABES, October 2009 Basics of ANNs Hidden Layers/Nodes Heuristics (II) Following Kolmogorov’s theorem: 1 HL and (2 Ni + 1) nodes adequate for most problems for 6 inputs 1 HL with 13 nodes
Basics of anns Learning Algorithm -Back Propagation ( D) The training process starts with a random distribution of the weights wii between layers passes through the nodes of the hls a unique and wrong solution vector is yielded at the output a measure of the error can be calculated E=∑(0-t k=1 tK: correct output value, Dk: predicted output value K: number of output neurons DCABES. October 2009
DCABES, October 2009 Basics of ANNs Learning Algorithm - Back Propagation (I) – The training process starts with a random distribution of the weights wij between layers passes through the nodes of the HLs a unique and wrong solution vector is yielded at the output. – a measure of the error can be calculated t k : correct output value, yk : predicted output value K: number of output neurons
Basics of anns Learning Algorithm- Back Propagation (ID) Error minimization( gradient descent method) OE Thus, change in the output vector de ae dyk N刂k-0rk (孙k-tk)×0k×(1-) correction in weights △"示k=7×△k+a×△ m-wik (n denoting iterations n: learning rate a: momentum term DCABES. October 2009
DCABES, October 2009 Basics of ANNs Learning Algorithm - Back Propagation (II) – Error minimization (Gradient Descent method) 0 – Thus, change in the output vector: – correction in weights: (n denoting iterations) – η: learning rate α: momentum term