4. 0.1 General matri Let y=X then dy= XdX+dXX or dY=18X+X81 Using the Eiy as before Equation (5), we see that the eigenvalues of IoX+XoI are Ai+A, so that detJ=Ⅱ1≤i≤n(A+))=2 n XII<(A+入)2. When n=2, we obtain4detx(trx)2 asseen example 8.2.2 For general powers, Y= XP,(p= 1, 2, 3,.. XkdX XP-l-k Axp1-)=p"(det X)P-II Of course by computing the Jacobian determinant explicitly in terms of the matrix elements, we can obtain an equivalent expression in terms of the elements of X rather than the A Formally, if we plug in p=-1, we find that we can compute det J for Y=X-I yielding the answer given after Theorem 8.1. Indeed one can take any scalar function y= f(a) such as r P, e, log a, sin r, etc The correct formula is det J = det((X)) We can prove these using the tools provided by wedge products. Often one thinks that to define Y= f(X) for matrices, one needs a power series, but this is not so. If X=ZAZ-I, then we can define f(X)=Zf(A)Z-l, where f(A)=diag(f(\1),., f(An).The formula bove is correct at any matrix X such that f is differentiable at the eigenvalues When X is of low rank. we can consider the moore-Penrose Inverse of x. the jacobian was calcula 2 and 3. There is a nonsymmetric and symmetric version Exercise. Compute the Jacobian determinant for and Y= X Advanced exercise: compute the Jacobian matriz for Y=x1/2 as an inverse of a sum of two Kronecker products. Numerically, a "Sylvester equation"solver is needed. To obtain the Jacobian matriz for Y=x-/ one can use the chain rule, given the answer to the previous problem. It is interesting that this is as hard as 4.0.2 Symmetric Matrices If X is symmetric, we have that det J=(det X)-(n+l) for the map Y=X-I 5 Jacobians of Matrix Factorizations(without wedge products) Let A be an n x n matrix. In elementary linear algebra, we learn about Gaussian elimination, Gram Schmidt orthogonalization, and the eigendecomposition. All of these ideas may Factly matrix factorizations, which in turn may be thought of as a change of variables
Q Q X Y X 4.0.1 General Matrices Let Y = X2 then dY = XdX + dXX or dY = I ⊗ X + XT ⊗ I Using the Eij as before Equation (5), we see that the eigenvalues of I ⊗ X + XT ⊗ I are λi + λj so that det J = 1≤i,j≤n(λi + λj ) = 2n det X i<j (λi + λj )2 . When n = 2, we obtain 4 det X(trX)2 as seen in example 8.2.2. For general powers, Y = Xp, (p = 1, 2, 3, . . .) p−1 dY = XkdX Xp−1−k k=0 n j and det J = p−1 λi kλp−1−k ! = p (det X) p−1 Y" λi p − λp #2 . j 1≤i,j≤n k=0 i<j λi − λj Of course by computing the Jacobian determinant explicitly in terms of the matrix elements, we can obtain an equivalent expression in terms of the elements of X rather than the λi. Formally, if we plug in p = −1, we find that we can compute det J for Y = X−1 yielding the answer given after Theorem 8.1. Indeed one can take any scalar function y = f(x) such as xp, ex , log x,sin x, etc. The correct formula is 2 Y Y n 2 f(λi) − f(λj ) det J = det(f′ (X)) Y f(λi) − f(λj ) = f′ · (λi) . λi − λj λi − λj i<j i=1 i<j We can prove these using the tools provided by wedge products. Often one thinks that to define Y = f(X) for matrices, one needs a power series, but this is not so. If X = ZΛZ−1, then we can define f(X) = Zf(Λ)Z−1, where f(Λ) = diag(f(λ1), . . . , f(λn)). The formula above is correct at any matrix X such that f is differentiable at the eigenvalues. When X is of low rank, we can consider the Moore-Penrose Inverse of X. The Jacobian was calculated in [2] and [3]. There is a nonsymmetric and symmetric version. Exercise. Compute the Jacobian determinant for Y = X1/2 and Y = X−1/2 . Advanced exercise: compute the Jacobian matrix for Y = X1/2 as an inverse of a sum of two Kronecker products. Numerically, a “Sylvester equation” solver is needed. To obtain the Jacobian matrix for Y = X−1/2 one can use the chain rule, given the answer to the previous problem. It is interesting that this is as hard as it is. 4.0.2 Symmetric Matrices If X is symmetric, we have that det J = (det X)−(n+1) for the map Y = X−1 . 5 Jacobians of Matrix Factorizations (without wedge products) Let A be an n × n matrix. In elementary linear algebra, we learn about Gaussian elimination, GramSchmidt orthogonalization, and the eigendecomposition. All of these ideas may be written compactly as matrix factorizations, which in turn may be thought of as a change of variables:
parameter count Gaussian elimination: (n-1)/2+n(n+1)/2 unit lower upper triangular triangular G 1)/2 triangular tion:A A envectors eigenvalues eigenvector value Each of these factorizations is a change of variables. Somehow the n- parameters in A are transformed into n2 other parameters, though it may not be immediately obvious what these parameters are(the n(n-1)/2 parameters in Q for example Our goal is to derive the Jacobians for those matrix factorizations 5.1 Jacobian of Gaussian Elimination(A=LU) In numerical linear algebra texts it is often observed that the memory used to store A on a computer may be overwritten with the n- variables in L and U. Graphically the n x n matrix A Indeed the same is true for other factorizations. This is not just a happy coincidence, but deeply related te the fact that n- parameters ought not need more storage. Theorem 2. If A= LU, the Jacobian of the change of variables is Proof 1: Let A= LU, then using da Ldu+dLU L(dUU-+L- dL)U (U 8L)((U Upper 1)dU +(I Lower L)-IdL The mapping du -duU-I only affects the upper triangular part. Similarly dL-L-dL for the lower triangular. We might think of the map in block format Since the Jacobian determinants of UT&L is Ilun and of(UTo I)-l is Ilui(and that of I e L is 1),we have that det J =llu
Y Q Q Q Gaussian Elimination: Gram-Schmidt: Eigenvalue Decomposition: A = L U · ↑ ↑ unit lower upper triangular triangular A = Q R · ↑ ↑ Orthogonal upper triangular A = X Λ X−1 · · ↑ ↑ eigenvectors eigenvalues parameter count n(n − 1)/2 + n(n + 1)/2 n(n − 1)/2 + n(n + 1)/2 (n2 − n) + n ↑ ↑ eigenvector eigenvalue Each of these factorizations is a change of variables. Somehow the n2 parameters in A are transformed into n2 other parameters, though it may not be immediately obvious what these parameters are (the n(n − 1)/2 parameters in Q for example). Our goal is to derive the Jacobians for those matrix factorizations. 5.1 Jacobian of Gaussian Elimination (A = LU) In numerical linear algebra texts it is often observed that the memory used to store A on a computer may be overwritten with the n2 variables in L and U. Graphically the n × n matrix A U A = L Indeed the same is true for other factorizations. This is not just a happy coincidence, but deeply related to the fact that n2 parameters ought not need more storage. Theorem 2. If A = LU, the Jacobian of the change of variables is n det J = u n−1 u22 n−2 . . . un−1,n−1 = u n−i 11 ii i=1 Proof 1: Let A = LU, then using dA = L dU + dL U = L(dUU −1 + L−1dL)U = (UT ⊗ L) (UT ⊗upper I) −1dU + (I ⊗lower L) −1dL The mapping dU dUU −1 only affects the upper triangular part. Similarly dL L−1 → → dL for the lower triangular. We might think of the map in block format: −1 dU dA = UT ⊗ L (UT ⊗upper I) I ⊗lower L dL n Since the Jacobian determinants of UT ⊗ L is uii and of (UT ⊗ I)−1 is u−i (and that of I ⊗ L is 1), we ii have that det J = u n−i ii . � That is the fancy slick proof. Here is the direct proof. It is of value to see both