Part 2.Linear Algebra Positive(Semi-)Definite Matrix ·Rank Inner Product,Norm,Matrix Norm Matrix Decomposition Advanced Optimization (Fall 2023) Lecture 1.Mathematical Background 11
Advanced Optimization (Fall 2023) Lecture 1. Mathematical Background 11 Part 2. Linear Algebra • Positive (Semi-)Definite Matrix • Rank • Inner Product, Norm, Matrix Norm • Matrix Decomposition
Positive (Semi-)Definite Matrix Definition 4(Positive Definite,PD).A matrix A E Rdxd is positive definite,if for all x≠0，xTAx>0,usually denoted as A>0. Definition 5(Positive Semi-Definite,PSD).A matrix A E Rdxd is positive semi-definite,if for all x∈Ra,xTAx≥0，usually denoted as A≥0. Advanced Optimization (Fall 2023) Lecture 1.Mathematical Background 12
Advanced Optimization (Fall 2023) Lecture 1. Mathematical Background 12 Positive (Semi-)Definite Matrix
Rank Rank:the dimension of the vector space spanned by its columns, or the maximal number of linearly independent columns. Example 5. 1 [1 -3 2R1+R2→R2 21 -3R1+R3→R3 1 2 1 0 13 0 1 3 35 0 0 -1-3 「12 1 10-5 R2+R3→R3 013 -2R2+R1→B1 0 000 00 The rank of matrix A is 2. Advanced Optimization (Fall 2023) Lecture 1.Mathematical Background 13
Advanced Optimization (Fall 2023) Lecture 1. Mathematical Background 13 Rank • Rank: the dimension of the vector space spanned by its columns, or the maximal number of linearly independent columns. The rank of matrix A is 2
Inner Product ·Vector Space:consider x,y∈Rd,then d x,y)=xTy=∑x班 2=1 ·Matrix Space:consider A,B∈Rmxm,then (A,B)=Tr(ATB)=∑∑AB i=1 i=1 Advanced Optimization (Fall 2023) Lecture 1.Mathematical Background 14
Advanced Optimization (Fall 2023) Lecture 1. Mathematical Background 14 • Vector Space: Inner Product • Matrix Space:
Norm Typically used vector norms. -(1-norm: xl1=lx+…+lzd -(2-norm: x2=xx)2=√好++话 or called Euclidean norm -loo-norm: lxlo=max {11l,...,ldly Advanced Optimization (Fall 2023) Lecture 1.Mathematical Background 15
Advanced Optimization (Fall 2023) Lecture 1. Mathematical Background 15 Norm • Typically used vector norms. or called Euclidean norm