Modeling the Single Neuron n inputs Efficiency of synapses modeled by having a multiplicative factor on each of the inputs to the neuron Multiplicative factor associated weights on ∑)y input lines · Neuron' s tasks Calculates weighted sum of its inputs Compares sum to some internal threshold Xn Turn on if threshold exceeded
6 Modeling the Single Neuron • n inputs. • Efficiency of synapses modeled by having a multiplicative factor on each of the inputs to the neuron. • Multiplicative factor = associated weights on input lines. • Neuron’s tasks: – Calculates weighted sum of its inputs. – Compares sum to some internal threshold. – Turn on if threshold exceeded. Σ x1 x2 xn w1 w2 wn y
A Mathematical Model of neurons Neuron computes The thresholding function is altenative weighted sum known as the"step"function, or the "Heaviside"function. Threshold functian. SUM thresholding st 8. W:X Fire if sum exceeds a threshold e -y=l if SUM> 0 Threshold function. y= O if SUM≤
7 A Mathematical Model of Neurons • Neuron computes weighted sum: • Fire if SUM exceeds a threshold θ. – y=1 if SUM > θ – y=0 if SUM θ. n i i i SUM w x 1
Learning in Simple Neurons Need to be able to determine connection weights Inspiration comes from looking at real neural systems Reinforce good behavior and reprimand bad E.g., train a nn to recognize 2 characters H and F Output 1 when a H is presented and 0 when it sees a F If it produces an incorrect output, we want to reduce the chances of that happening again This is done by modifying the weights
8 Learning in Simple Neurons • Need to be able to determine connection weights. • Inspiration comes from looking at real neural systems. – Reinforce good behavior and reprimand bad. – E.g., train a NN to recognize 2 characters H and F – Output 1 when a H is presented and 0 when it sees a F. – If it produces an incorrect output, we want to reduce the chances of that happening again. – This is done by modifying the weights
Learning in Simple Neurons (2) Neuron given random initial weights At starting state, neuron knows nothing Present an H Neuron computes the weighted sum of inputs Compare weighted sum with threshold If exceeds threshold, output a 1 otherwise a 0 If output is 1, neuron is correct Do nothing Otherwise if neuron produces a 0 Increase the weights so that next time it will exceed the threshold and produces a 1
9 Learning in Simple Neurons (2) • Neuron given random initial weights. – At starting state, neuron knows nothing. • Present an H. – Neuron computes the weighted sum of inputs. – Compare weighted sum with threshold. – If exceeds threshold, output a 1 otherwise a 0. • If output is 1, neuron is correct. – Do nothing. • Otherwise if neuron produces a 0. – Increase the weights so that next time it will exceed the threshold and produces a 1
A Simple Learning Rule How much weight to increase? Can follow simple rule Add the input values to the weights when we want the output to be on Subtract the input values from the weights when we want the output to be off This learning rule is called the Hebb rule It is a variant on one proposed by donald Hebb and is called Hebbian learning It is the earliest and simplest learning rule for a neuron 10
10 A Simple Learning Rule • How much weight to increase? • Can follow simple rule: – Add the input values to the weights when we want the output to be on. – Subtract the input values from the weights when we want the output to be off. • This learning rule is called the Hebb rule: – It is a variant on one proposed by Donald Hebb and is called Hebbian learning. – It is the earliest and simplest learning rule for a neuron