版像 NJUAT 南京大学 人工智能学院 SCHODL OF ARTIFICIAL INTELUGENCE,NANJING UNIVERSITY Lecture 5.Online Convex Optimization Advanced Optimization(Fall 2023) Peng Zhao zhaop@lamda.nju.edu.cn Nanjing University
Lecture 5. Online Convex Optimization Peng Zhao zhaop@lamda.nju.edu.cn Nanjing University Advanced Optimization (Fall 2023)
Outline Online Learning Online Convex Optimization ·Convex Functions Strongly Convex Functions Exp-concave Functions Advanced Optimization(Fall 2023) Lecture 5.Online Convex Optimization 2
Advanced Optimization (Fall 2023) Lecture 5. Online Convex Optimization 2 Outline • Online Learning • Online Convex Optimization • Convex Functions • Strongly Convex Functions • Exp-concave Functions
A Brief Review of Statistical Learning The fundamental goal of(supervised)learning:Risk Minimization(RM), minE(h(x),y)], h∈H where -h denotes the hypothesis(model)from the hypothesis space Ht. -(x,y)is an instance chosen from a unknown distribution D. -e(h(x),y)denotes the loss of using hypothesis h on the instance(x,y). Advanced Optimization(Fall 2023) Lecture 5.Online Convex Optimization 3
Advanced Optimization (Fall 2023) Lecture 5. Online Convex Optimization 3 A Brief Review of Statistical Learning
a Brief Review of Statistical Learning Given a loss function f and distribution D,the expected risk of predictor h is R(h)=E(xD[(h(x),y)]. In practice,we can only access to a sample set S={(x1,1),...,(xm,Um)} Thus,the following empirical risk is naturally defined: Rsm=工x.8 77 i=1 Advanced Optimization(Fall 2023) Lecture 5.Online Convex Optimization 4
Advanced Optimization (Fall 2023) Lecture 5. Online Convex Optimization 4 A Brief Review of Statistical Learning
A Brief Review of Statistical Learning A successful story characterization of sample complexity ▣excess risk bound -恶so(a) generalization error bound R,a-≤o(元) Advanced Optimization(Fall 2023) Lecture 5.Online Convex Optimization 5
Advanced Optimization (Fall 2023) Lecture 5. Online Convex Optimization 5 A Brief Review of Statistical Learning • A successful story : characterization of sample complexity p excess risk bound p generalization error bound