Binary Neurons hard threshold 1.2 output Stimulus Response l4=∑ furs tu “Hard” threshold heaviside z≥⊙→ON -1 else→OFF 0= threshold ex: Perceptrons, Hopfield nns, boltzmann Machines Main drawbacks: can only map binary functions, biologically implausible 02/02/2021 Artificial Neural Networks
02/02/2021 Artificial Neural Networks - I 11 Binary Neurons ( ) = else OFF z ON f z “Hard” threshold = threshold hard threshold -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 -10 -8 -6 -4 -2 0 2 4 6 8 10 input output heaviside • ex: Perceptrons, Hopfield NNs, Boltzmann Machines • Main drawbacks: can only map binary functions, biologically implausible. off on = j i ij j u w x Stimulus ( ) i urest ui y = f + Response
Analog neurons sigmoid 1.output Stimulus Response u+u (1+exp(-×x)-1 soft” threshold 1+e ex: MLPs. Recurrent nNs rbF nns Main drawbacks: difficult to process time patterns, biologically implausible 02/02/2021 Artificial Neural Networks 12
02/02/2021 Artificial Neural Networks - I 12 Analog Neurons ( ) 1 1 2 − + = −z e f z “Soft” threshold sigmoid -1.2 -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 1.2 -10 -8 -6 -4 -2 0 2 4 6 8 10 input output 2/(1+exp(-x))-1 • ex: MLPs, Recurrent NNs, RBF NNs... •Main drawbacks: difficult to process time patterns, biologically implausible. off on = j i ij j u w x Stimulus ( ) i urest ui y = f + Response
Spiking neurons Stimulus n= spike and afterspike potential urest=resting potential p()=∑v,x e(t,u(t))= trace at time t of input at time t threshold xt=output of neuron at time t Wi=efficacy of synapse from neuron i to neuron J Response u(t=input stimulus at time t p0)=(a+m(=1)+∑cn() dz z≥Q&>0→ON else→OFF 02/02/2021 Artificial Neural Networks
02/02/2021 Artificial Neural Networks - I 13 Spiking Neurons ( ) = ( ) j i ij j u t w x t ( ( ( ))) = = + − + t i rest f ui y t f u t t t 0 ( ) ( ) , ( ) = else OFF ON dt dz z f z & 0 = spike and afterspike potential urest = resting potential (t,u()) = trace at time t of input at time = threshold xj (t) = output of neuron j at time t wij = efficacy of synapse from neuron i to neuron j u(t) = input stimulus at time t Response Stimulus
Spiking neuron dynamics neuron output 2.5 n(t-t) 1.5 0.5 10 304050/60708090100 -0.5 02/02/2021 Artificial Neural Networks 14
02/02/2021 Artificial Neural Networks - I 14 Spiking Neuron Dynamics neuron output - 1 -0.5 0 0.5 1 1.5 2 2.5 0 10 20 30 40 50 60 70 80 90 100 t V y(t) urest+(t-tf )
赫布律 加拿大心理学家 Donald hebb出版了《行为的组 织》一书,指出学习导致突触的联系强度和传 递效能的提高,即为“赫布律” 在此基础上,人们提出了各种学习规则和算法, 以适应不同网络模型的需要。有效的学习算法 使得神经网络能够通过连接权值的调整,构造 客观世界的内在表示,形成具有特色的信息处 理方法,信息存储和处理体现在网络的连接中。 02/02/2021 Artificial Neural Networks
02/02/2021 Artificial Neural Networks - I 15 赫布律 加拿大心理学家Donald Hebb出版了《行为的组 织》一书，指出学习导致突触的联系强度和传 递效能的提高，即为“赫布律” 。 在此基础上，人们提出了各种学习规则和算法， 以适应不同网络模型的需要。有效的学习算法， 使得神经网络能够通过连接权值的调整，构造 客观世界的内在表示，形成具有特色的信息处 理方法，信息存储和处理体现在网络的连接中