Statistical Learning Theory and Applications Lecture 8 Data Representation-Parametric Model Instructor:Quan Wen SCSE@UESTC Fal,2021
Statistical Learning Theory and Applications Lecture 8 Data Representation - Parametric Model Instructor: Quan Wen SCSE@UESTC Fall, 2021
Outline (Level 1) Probability Density Estimation 2 Maximum Likelihood Estimation MAP estimation 4 Bayesian estimation 5 Expectation maximization 1/88
Outline (Level 1) 1 Probability Density Estimation 2 Maximum Likelihood Estimation 3 MAP estimation 4 Bayesian estimation 5 Expectation maximization 1 / 88
Outline (Level 1) Probability Density Estimation 2 Maximum Likelihood Estimation MAP estimation Bavesian estimation Expectation maximization 2/88
Outline (Level 1) 1 Probability Density Estimation 2 Maximum Likelihood Estimation 3 MAP estimation 4 Bayesian estimation 5 Expectation maximization 2 / 88
1.Probability Density Estimation Basic Concept 1 Density estimation:estimating the probability density function p(x)based on a given set of training samples D={x1,x2,...,xN}. 2 Estimated density:denoted by p(x). 3 Training samples are i.i.d.and distributed according to p(x). 4 Parametric estimation:parameter vector 0 ofp(x;0) 5 Non-parametric estimation:a function p:X->R 6 Finite number of training samples meaning that there will be some errors in the function(density)estimation. 3/88
1. Probability Density Estimation Basic Concept 1 Density estimation: estimating the probability density function p(x) based on a given set of training samples D = {x1, x2, ..., xN}. 2 Estimated density: denoted by pˆ(x). 3 Training samples are i.i.d. and distributed according to p(x). 4 Parametric estimation: parameter vector θ of p(x; θ) 5 Non-parametric estimation: a function p : X → R 6 Finite number of training samples meaning that there will be some errors in the function (density) estimation. 3 / 88
The parametric model probability estimation has a known global distribution form,i.e.function form. But in fact,we know nothing about distribution. The non-parametric model can be applied to any case of probability distribution without assuming that the form of probability distribution is known. 4/88
▶ The parametric model probability estimation has a known global distribution form, i.e. function form. • But in fact, we know nothing about distribution. ▶ The non-parametric model can be applied to any case of probability distribution without assuming that the form of probability distribution is known. 4 / 88