1.3.2.Trace of Matrix Trace of square matrix A E Rxm is: tA- ∑An i=1 Properties of trace trA trA? tr(A+B)=trA trB,A,B E R"xn tr(入A)=λtrA,入∈R tr(AB)tr(BA) tr(ABC)=tr(BCA)=tr(CAB) 20/130
1.3.2. Trace of Matrix ▶ Trace of square matrix A ∈ R n×n is: tr A = Xn i=1 Aii ▶ Properties of trace trA = trA T tr(A + B) = trA + trB, A, B ∈ R n×n tr(λA) = λtrA, λ ∈ R tr(AB) = tr(BA) tr(ABC) = tr(BCA) = tr(CAB) 20 / 130
Outline (Level 2-3) Operational Properties o Symmetric Matrix o Trace of Matrix ●Vector Norm 21/130
Outline (Level 2-3) Operational Properties Symmetric Matrix Trace of Matrix Vector Norm 21 / 130
1.3.3.Vector Norm ‖x‖，the norm of vector x,is a function:f:R"→R,which satisfies the following 3 properties: x,y∈R",λ∈R,we have3 axioms: 1 iff(x)=0,then x =0,(separates points) 2 f(x+y)<f(x)+f(y),(triangle inequality or subadditivity) 3 f(Ax)=lf(x),(absolute homogeneity or absolute scalability) By the above 3 axioms,we havef(0)=0 and f(-v)=f(v),so that by the triangle inequality: f(x)≥0，(positivity) 22/130
1.3.3. Vector Norm ▶ ∥ x ∥, the norm of vector x , is a function: f : R n → R , which satisfies the following 3 properties: ∀x, y ∈ R n , λ ∈ R , we have 3 axioms: 1 if f (x) = 0, then x = 0, (separates points) 2 f (x + y) ≤ f (x) + f (y), (triangle inequality or subadditivity) 3 f (λx) = |λ| f (x), (absolute homogeneity or absolute scalability) By the above 3 axioms, we have f (0) = 0 and f (−v) = f (v), so that by the triangle inequality: f (x) ≥ 0, (positivity) 22 / 130
Common vector norm: 2norm:‖xl2=√∑1x7-Vxi,vector length h norm:‖xl1=1xl l norm:‖x‖o=max(lx,.,xnl) norm:‖xlp=(∑=1xP)p Matrix Norm: Frobenius Norm IA-\∑-v而=viu西-Veam团 Where vec(A)=a11,...,am1a12...am.2.mmn Matrix Frobenius norm Vtr(ATA)is similar to vector 12 norm VxTx 23/130
▶ Common vector norm: l2 norm：∥ x ∥2= pPn i=1 xi 2 = √ x Tx, vector length l1 norm：∥ x ∥1= Pn i=1 |xi | l∞ norm：∥ x ∥∞= max(|x1|, . . . , |xn|) lp norm：∥ x ∥p= (Pn i=1 |xi | p ) 1/p ▶ Matrix Norm： Frobenius Norm ∥ A ∥F= vuut Xm i=1 Xn j=1 Aij 2 = p tr(ATA) = p tr(AAT ) = p vec(A) Tvec(A) where vec(A)=[a1,1, . . . , am,1, a1,2, . . . , am,2, . . . , a1,n, . . . , am,n] T Matrix Frobenius norm p tr(ATA) is similar to vector l2 norm √ x Tx 23 / 130
Outline (Level 1-2) ○Linear Algebra o Notations and Fundamental Concepts o Matrix Multiplication o Operational Properties Linear Space o Quadratic Form and Positive Definite Matrix o Matrix Calculus 24/130
Outline (Level 1-2) 1 Linear Algebra Notations and Fundamental Concepts Matrix Multiplication Operational Properties Linear Space Quadratic Form and Positive Definite Matrix Matrix Calculus 24 / 130