Chapter 2 Linear Regression
Chapter 2 Linear Regression
Contents 。2.1 Introduction 2.2 Polynomial Curve Fitting:A Toy 。2.3Over-fitting 2.4 Cross-Validation ·2.5 Regularization 2.6 Linear Basis Function Models 2.7 Least Squares Revisited 2.8 Bayesian Linear Regression
Contents • 2.1 Introduction • 2.2 Polynomial Curve Fitting: A Toy • 2.3 Over-fitting • 2.4 Cross-Validation • 2.5 Regularization • 2.6 Linear Basis Function Models • 2.7 Least Squares Revisited • 2.8 Bayesian Linear Regression
2.1 Introduction Supervised learning:regression and classification.The goal of regression is to predict the value of one or more continuous target variable given the value of a p-dimensional vector of input variables. In linear regression,the model for such a relationship is a linear model. Alinear regression example:given a ata set linear regression model assumes that the relationship between the target variable yi and the p-dimensional vector of input variables xi is linear,i.e. yi=Bixu++Bpxip +s=xB+ i=1,.,n
2.1 Introduction • Supervised learning: regression and classification. The goal of regression is to predict the value of one or more continuous target variable given the value of a p-dimensional vector of input variables. • In linear regression, the model for such a relationship is a linear model. • A linear regression example: given a data set , a linear regression model assumes that the relationship between the target variable and the p-dimensional vector of input variables is linear, i.e
2.1.1 History for Regression The earliest form of regression was the method of least squares,which was published by Legendre in 1805,and by Gauss in 1809. ·The term"regression(倒退，退步)"was coined by Francis Galton in the 19th century to describe Gauss Francis Galton a biological phenomenon.The phenomenon was that the heights of descendants of tall ancestors tend to regress down towards a normal average. Galton's work was later extended by Udny Yule and Karl Pearson to a more general statistical context. Udny Yule Karl Pearson
2.1.1 History for Regression • The earliest form of regression was the method of least squares, which was published by Legendre in 1805, and by Gauss in 1809. • The term "regression (倒退，退步)" was coined by Francis Galton in the 19th century to describe a biological phenomenon. The phenomenon was that the heights of descendants of tall ancestors tend to regress down towards a normal average. • Galton’s work was later extended by Udny Yule and Karl Pearson to a more general statistical context. Gauss Francis Galton Udny Yule Karl Pearson
2.1.2 Development for Regression Regression methods continue to be an active research area. New methods have been developed for √robust regression regression involving correlated responses such as time series and growth curves regression in which the predictor (independent variable)or response variables are curves, images,graphs,or other complex data objects regression methods accommodating various types of missing data,nonparametric regression Bayesian methods for regression regression in which the predictor variables are measured with error regression with more predictor variables than observations causal inference with regression
2.1.2 Development for Regression • Regression methods continue to be an active research area . • New methods have been developed for robust regression regression involving correlated responses such as time series and growth curves regression in which the predictor (independent variable) or response variables are curves, images, graphs, or other complex data objects regression methods accommodating various types of missing data, nonparametric regression Bayesian methods for regression regression in which the predictor variables are measured with error regression with more predictor variables than observations causal inference with regression