20.5 Nerual NetworksThanks: Professors FrankHoffmann and Jiawei Han, andRussell and Norvig
20.5 Nerual Networks Thanks: Professors Frank Hoffmann and Jiawei Han, and Russell and Norvig
Biological Neural SystemsNeuron switching time : > 10-3 secs Number of neurons in the human brain: ~1010 Connections (synapses) per neuron : ~104-105Face recognition : 0.1 secsHigh degree of distributed and parallel computationHighly fault tolerentHighly efficientLearning is key
Biological Neural Systems ◼ Neuron switching time : > 10-3 secs ◼ Number of neurons in the human brain: ~1010 ◼ Connections (synapses) per neuron : ~104–105 ◼ Face recognition : 0.1 secs ◼ High degree of distributed and parallel computation ◼ Highly fault tolerent ◼ Highly efficient ◼ Learning is key
Excerpt from Russell and NorvigBrains10llneuronsof >20types,1014synapses,1ms-10mscycletimeSignalsare noisy "spike trains"of electrical potentialAxonalarborizationAxonfromanothercellSynapseDendriteAxonNucleusSynapsesCellbodyor Soma
Excerpt from Russell and Norvig
A NeuronakWkin, outputInputoutputZailinkslinksZWig* Ikinj =a; = output(inComputation: input signals → input function(linear)→ activationfunction(nonlinear)> output signal
A Neuron ◼ Computation: ◼ input signals → input function(linear) → activation function(nonlinear) → output signal aj output links ak output Input links Wk j ai = output(inj ) inj j inj =Wkj *Ik
Part 1. Perceptrons: Simple NNinputsweightsW1X1outputactivationW2ZX20a=Zi=1" W; XiWXi's range: [0, 1]n1 if a≥0y=oifa<0
Part 1. Perceptrons: Simple NN x1 x2 xn . . . w1 w2 wn a=i=1 n wi xi Xi’s range: [0, 1] 1 if a q y= 0 if a < q y { inputs weights activation output q