版套是 NJUAT 南京大学 人工智能学院 SCHODL OF ARTIFICIAL INTELUGENCE,NANJING UNIVERSITY Lecture 1.Mathematical Background Advanced Optimization(Fall 2023) Peng Zhao zhaop@lamda.nju.edu.cn Nanjing University
Lecture 1. Mathematical Background Peng Zhao zhaop@lamda.nju.edu.cn Nanjing University Advanced Optimization (Fall 2023)
Outline 。Calculus ·Linear Algebra Probability Statistics ·Information Theory Optimization in Machine Learning Advanced Optimization (Fall 2023) Lecture 1.Mathematical Background 2
Advanced Optimization (Fall 2023) Lecture 1. Mathematical Background 2 Outline • Calculus • Linear Algebra • Probability & Statistics • Information Theory • Optimization in Machine Learning
Notational Convention -[m={1,..,n} -x,y,v:vectors A.B:matrices -,K:domain d,m,n:dimensions -I:identity matrix X,Y:random variables p,q:probability distributions Advanced Optimization (Fall 2023) Lecture 1.Mathematical Background 3
Advanced Optimization (Fall 2023) Lecture 1. Mathematical Background 3 Notational Convention
Function ·Function mapping f:domf R"→yC Rm Definition 1(Continuous Function).A function f:R"->Rm is continuous at x E dom f if for all e>0 there exists a >0 with y E dom f,such that Ily-x2≤6→If(y)-f(x)2≤e. Advanced Optimization (Fall 2023) Lecture 1.Mathematical Background 4
Advanced Optimization (Fall 2023) Lecture 1. Mathematical Background 4 Function • Function mapping
Part 1.Calculus Gradient and Derivatives ·essian ·Chain Rule Advanced Optimization (Fall 2023) Lecture 1.Mathematical Background 5
Advanced Optimization (Fall 2023) Lecture 1. Mathematical Background 5 Part 1. Calculus • Gradient and Derivatives • Hessian • Chain Rule