Ordinary Least Squares (OLS)Estimation It follows that (X'X)B=X'Y. By Assumption 3.3,X'X is nonsingular.Thus, B=(X'X)-1X'Y. Checking the SOC,we have the K x K Hessian matrix ∂2SSR(B) ∂3∂B' 2 [(Yi-X B)Xi] t=1 2X'X is positive definite given Amin(X'X)>0.Thus,B is a global minimizer. ● Suppose Z=Yi,X}',t =1,...,n,is an IID random sample of size n. Consider the sum of squared residual scaled by n-: SsR(0=1∑Y:-XB2 m t=1 To be continued ADVANCED ECONOMETRICS Classical Linear Regression Model May11,2021 21
ADVANCED ECONOMETRICS Classical Linear Regression Model May 11, 2021 21 Ordinary Least Squares (OLS) Estimation To be continued
Ordinary Least Squares (OLS)Estimation and its minimizer -(2x)gn 十1 These are the sample analogs of the population MSE criterion MSE(B)=E(Yt-XB) and its minimizer B*=[E(XX)】E(XY): That is,SSR(B),after scaled by n-1,is the sample analogue of MSE(B), and the OLS B is the sample analogue of the best least squares approxi- mation coefficient B*. ●Put 立=X3. This is called the fitted value (or predicted value)for observation Yi,and et≡Y-Ya To be continued ADVANCED ECONOMETRICS Classical Linear Regression Model May11,2021 22
ADVANCED ECONOMETRICS Classical Linear Regression Model May 11, 2021 22 Ordinary Least Squares (OLS) Estimation To be continued
Ordinary Least Squares(OLS)Estimation is called the estimated residual (or prediction error)for observation Yi Note that et= Yi-Y (XBO+Et)-XiB =et-X4(8-B), where et is the unavoidable true disturbance et,and X(B-B)is an estimation error,which is smaller when a larger data set is available (so B becomes closer to Bo). The FOC implies that the estimated residual e =Y-XB is orthogonal to regressors X in the sense that X'e=∑Xet=0. t=1 This is the consequence of the very nature of OLS,as implied by the FOC of minBeRK SSR(B).It always holds no matter whether E(et X)=0 (note that we do not impose Assumption 3.2).Note that if Xt contains the intercept,then X'e=0 implies e=0. ADVANCED ECONOMETRICS Classical Linear Regression Model May11,2021 23
ADVANCED ECONOMETRICS Classical Linear Regression Model May 11, 2021 23 Ordinary Least Squares (OLS) Estimation
CONTENTS 3.1 Framework and Assumptions 3.2 Ordinary Least Squares (OLS)Estimation 3.3 Goodness of Fit and Model Selection Criteria 3.4 Consistency and Efficiency of OLS 3.5 Sampling Distribution of OLS 3.6 Variance Estimation for OLS 3.7 Hypothesis Testing ADVANCED ECONOMETRICS Classical Linear Regression Model May11,2021 24
ADVANCED ECONOMETRICS Classical Linear Regression Model May 11, 2021 24 3.1 Framework and Assumptions 3.2 Ordinary Least Squares (OLS) Estimation 3.3 Goodness of Fit and Model Selection Criteria 3.4 Consistency and Efficiency of OLS 3.5 Sampling Distribution of OLS 3.6 Variance Estimation for OLS 3.7 Hypothesis Testing CONTENTS
Goodness of Fit and Model Selection Criteria Questions: How well does the linear regression model fit the data?That is,how well does a linear regression model explain the variations of the observed data of {Y}8=1? Definition 2 [Uncentered R2]: The uncentered squared multi-correlation coefficient is defined as yy e'e =-1- where the second equality follows from the first order condition of the OLS esti- mation. Rhas a nice interpretation:the proportion of the uncentered sample quadratic variation in the dependent variables {Y}that can be attributed to the uncentered sample quadratic variation of the predicted values {Y. ·We always have0≤R2c≤l. ADVANCED ECONOMETRICS Classical Linear Regression Model May11,2021 25
ADVANCED ECONOMETRICS Classical Linear Regression Model May 11, 2021 25 Goodness of Fit and Model Selection Criteria Questions: Definition 2