Chapter 6Architecture and Eguilibria6.2Global Equilibra:convergence andstabilityEquilibrium is steady state (for fixed-point attractors)Convergence is synaptic equilibrium. M=0 6.1Stability is neuronal equilibrium. X=06. 2More generally neural signals reach steady state eventhough the activations still change.We denote steady state in the neuronal field FFx=06.3Stability-Eguilibrium dilemmaNeuron fluctuate faster than synapses fluctuate2002.12.4Convergence undermines stability
2002.12.4 Chapter 6 Architecture and Equilibria 6.2 Global Equilibra:convergence and stability Equilibrium is steady state (for fixed-point attractors) Convergence is synaptic equilibrium. Stability is neuronal equilibrium. More generally neural signals reach steady state even though the activations still change. We denote steady state in the neuronal field Stability - Equilibrium dilemma : Neuron fluctuate faster than synapses fluctuate. Convergence undermines stability M = 0 6.1 • X = 0 6.2 • F x Fx = 0 6.3 •
Chapter 6Architecture and Eguilibria6.3 Synaptic convergence to centroids:AvQ AIgorithmsCompetitve learning adpatively qunatizes the input pattern space Rnp(x) charcaterizes the continuous distributions of patternAVQX→XcentroidWeshall provethat:Competitve AvQ synaptic vector m,convergeto pattern-classcentroid.They vibrate about the centroid in a Browmian2002.12.4motion
2002.12.4 Chapter 6 Architecture and Equilibria 6.3 Synaptic convergence to centroids:AVQ Algorithms We shall prove that: Competitve AVQ synaptic vector converge to pattern-class centroid. They vibrate about the centroid in a Browmian motion m j Competitve learning adpatively qunatizes the input pattern space charcaterizes the continuous distributions of pattern. n R p(x) X X centroid AVQ ^ →
Chapter6Architectureand Eguilibria6.3 Synaptic convergence to centroids:AvQAlgorithmsComptetiveAvQ StochasticDifferentialEquationsPatternRn6.6Rn=DUD2UD3...UDK6.7D,nD,=o,ifitjThe Random Indicator function I,,Io,Ie.(1ifxED6-8I,(x)=)o if xDSupervised learning algorithms depend explicitly on theindicator functions.Unsupervised learning algorthmsdon't require this pattern-class information.JD.xp(x)dxA6-9Centriodx"D,pdx2002.12.4
2002.12.4 Chapter 6 Architecture and Equilibria 6.3 Synaptic convergence to centroids:AVQ Algorithms , 6.7 . 6.6 1 2 3 if i j j D i D K Rn D D D D PatternRn = = The Random Indicator function Supervised learning algorithms depend explicitly on the indicator functions.Unsupervised learning algorthms don’t require this pattern-class information. Centriod D D D DK I ,I ,I ,.I 1 2 3 6 8 0 1 ( ) − = j j D i f x D i f x D I x j 6 9 ( ) ( ) ^ − = j D p x dx j D xp x dx j x Comptetive AVQ Stochastic Differential Equations
Chapter 6Architectureand Eguilibria6.3 Synaptic convergence to centroids:AvQAlgorithmsThe Stochastic unsupervised competitive learning law6-10m,=S,(yj)[x-m,]+njWe want to show that at equilibrium m, =x, or E(m,)=x,We assume6-11S,=Ip,(a)The equilibrium and convergence depend onapproximation (6-11),so 6-10reduces :6-12m,=ID,(x)[x-m,]+nj2002.12.4
2002.12.4 Chapter 6 Architecture and Equilibria 6.3 Synaptic convergence to centroids:AVQ Algorithms The Stochastic unsupervised competitive learning law: = ( )[ − ]+ 6 −10 • j j j mj nj m S y x We want to show that at equilibrium mj = xj mj = xj or E( ) S I (x) 6−11 Dj j We assume The equilibrium and convergence depend on approximation (6-11) ,so 6-10 reduces : = ( )[ − ]+ 6 −12 • j D mj nj m I x x j
Chapter6Architectureand Eguilibria6.3 Synaptic convergence to centroids:AvQAlgorithmsCompetitiveAvQAlgorithms1. Initialize synaptic vectors:m(O)= x(i) , i=1..m2.For random sample x(t),find the closet(winning)synapticvector m,(t)[m,(t)-x(0)|=min|m,(t)- x(0) 6-13wherel =x..+x.3.Update the wining synaptic vectors m,(t) by the UCL,SCL,orDCL learning algorithm.2002.12.4
2002.12.4 Chapter 6 Architecture and Equilibria 6.3 Synaptic convergence to centroids:AVQ Algorithms Competitive AVQ Algorithms 1. Initialize synaptic vectors: mi (0) = x(i) , i =1,.,m 2.For random sample ,find the closet(“winning”)synaptic vector x(t) m (t) j 2 2 1 2 . ( ) ( ) min ( ) ( ) 6 13 m i i j where x x x m t x t m t x t = + + − = − − 3.Update the wining synaptic vectors by the UCL ,SCL,or DCL learning algorithm. m (t) j