Statistical Learning Theory and Applications Lecture 9 Data Representation-Non-Parametric Model Instructor:Quan Wen SCSE@UESTC Fall,2021
Statistical Learning Theory and Applications Lecture 9 Data Representation — Non-Parametric Model Instructor: Quan Wen SCSE@UESTC Fall, 2021
Outline (Level 1) ①Density Estimation 2 Histogram Method 3 Parzen Window 4 k-nearest neighbors 5Summary of nonparametric models 1/45
Outline (Level 1) 1 Density Estimation 2 Histogram Method 3 Parzen Window 4 k-nearest neighbors 5 Summary of nonparametric models 1 / 45
Outline (Level 1) ○Density Estimation ②Histogram Method Parzen Window k-nearest neighbors Summary of nonparametric models 2/45
Outline (Level 1) 1 Density Estimation 2 Histogram Method 3 Parzen Window 4 k-nearest neighbors 5 Summary of nonparametric models 2 / 45
1.Density Estimation Basic concepts 1 Density estimation:estimating the probability density function p(x)based on a given set of training samples D={x1,x2,...,xN}. 2 Estimated density:denoted by p(x). 3 Training samples are i.i.d.and distributed according to p(x). 4 Parametric estimation:parameter vector 0 ofp(x;0) 5 Non-parametric estimation:a function p:X->R 6 Finite number of training samples meaning that there will be some errors in the function(density)estimation. 3/45
1. Density Estimation Basic concepts 1 Density estimation: estimating the probability density function p(x) based on a given set of training samples D = {x1, x2, ..., xN}. 2 Estimated density: denoted by pˆ(x). 3 Training samples are i.i.d. and distributed according to p(x). 4 Parametric estimation: parameter vector θ of p(x; θ) 5 Non-parametric estimation: a function p : X → R 6 Finite number of training samples meaning that there will be some errors in the function (density) estimation. 3 / 45
The parametric model probability estimation has a known global distribution form,i.e.function form. But in fact,we know nothing about distribution. The non-parametric model can be applied to any case of probability distribution without assuming that the form of probability distribution is known. 4/45
▶ The parametric model probability estimation has a known global distribution form, i.e. function form. ▶ But in fact, we know nothing about distribution. ▶ The non-parametric model can be applied to any case of probability distribution without assuming that the form of probability distribution is known. 4 / 45