Chapter 3.Neural Dynamics II:Activation Models X3.6.1 Optimal Linear Associative Memory Matrices Define the matrix Euclidean norm M as M=Trace(MMT) Minimize the mean-squared error of forward recall,to find M that satifies the relation Y-4≤lY-M for all M 2002.10.8
2002.10.8 Chapter 3. Neural Dynamics II:Activation Models ※3.6.1 Optimal Linear Associative Memory Matrices Define the matrix Euclidean norm M as ( ) T M = Trace MM Minimize the mean-squared error of forward recall,to find that satifies the relation M ˆ Y − XM ˆ Y − XM for all M
Chapter 3.Neural Dynamics II:Activation Models X3.6.1 Optimal Linear Associative Memory Matrices Suppose further that the inverse matrix exists. Then o ol =Y-Y川 =y-xx-y So the OLAM matrix M correspond toM=-Y 2002.10.8
2002.10.8 Chapter 3. Neural Dynamics II:Activation Models Suppose further that the inverse matrix exists. Then −1 X Y - XX Y Y Y -1 = = − 0 = 0 So the OLAM matrix correspond to M ˆ M X Y ˆ −1 = ※3.6.1 Optimal Linear Associative Memory Matrices
Chapter 3.Neural Dynamics II:Activation Models If the set of vector ,is orthonormal xx=日等 Then the OLAM matrix reduces to the classical linear associative memory(LAM): M=XTY For is orthonormal,the inverse of is. 2002.10.8
2002.10.8 Chapter 3. Neural Dynamics II:Activation Models If the set of vector is orthonormal {X1 , , X m } = = if i j if i j X X T i j 0 1 M X Y T = ˆ Then the OLAM matrix reduces to the classical linear associative memory(LAM) : For is orthonormal, the inverse of is . X X T X
Chapter 3.Neural Dynamics II:Activation Models X3.6.2 Autoassociative OLAM Filtering Autoassociative OLAM systems behave as linear filters. In the autoassociative case the OLAM matrix encodes only the known signal vectors Xi.Then the OLAM matrix equation (3-78)reduces to M=XY M linearly filters"input measurement x to the output vector x'by vector matrix multiplication:xM=x. 2002.10.8
2002.10.8 Chapter 3. Neural Dynamics II:Activation Models ※3.6.2 Autoassociative OLAM Filtering Autoassociative OLAM systems behave as linear filters. In the autoassociative case the OLAM matrix encodes only the known signal vectors . Then the OLAM matrix equation (3-78) reduces to xi M X X * = M linearly “filters” input measurement x to the output vector by vector matrix multiplication: . x xM = x
Chapter 3.Neural Dynamics II:Activation Models X3.6.2 Autoassociative OLAM Filtering The OLAM matrix X'X behaves as a projection operator[Sorenson,1980].Algebraically,this means the matrix Mis idempotent:M2=M. Since matrix multiplication is associative,pseudo- inverse property(3-80)implies idempotency of the autoassociative OLAM matrix M: M2=MM -X'XY'Y =(Y'XY)Y =X'X =M 2002.10.8
2002.10.8 Chapter 3. Neural Dynamics II:Activation Models ※3.6.2 Autoassociative OLAM Filtering The OLAM matrix behaves as a projection operator[Sorenson,1980].Algebraically,this means the matrix M is idempotent: . X X * M = M 2 Since matrix multiplication is associative,pseudoinverse property (3-80) implies idempotency of the autoassociative OLAM matrix M: M X X X XX X X XX X M MM * = = = = = ( ) * * * * 2