Parameter Estimation and Evaluation Maximum Likelihood Estimation Maximum Likelihood Estimation 2.0e-57 a plot for MLE when 0 is a scalar parameter. Remarks: 1.8e571 1.5e-57 likelihood function 1.3e57 1.0e-571 7.5e-58 5.0e58 2.5058 0. 0.6 0.7 0.8 0.9 61.0 1.1 1.2 1.3 1.4 1.5 MLE is the parameter estimate which makes the ob- served data x"most likely to occur.In other words,by choosing a parameter estimate O(x"),MLE maximizes the probability that X=x",that is,the probability that Xm takes the value of the observed data x". Parameter Estimation and Evaluation Introduction to Statistics and Econometrics April 21,2020 21/207
Parameter Estimation and Evaluation Parameter Estimation and Evaluation Introduction to Statistics and Econometrics April 21, 2020 21/207 Maximum Likelihood Estimation Maximum Likelihood Estimation Remarks:
Parameter Estimation and Evaluation Maximum Likelihood Estimation Maximum Likelihood Estimation Theorem 1(8.1).[Existence of MLE] Suppose,with probability one,L(X")is a continuous func- tion of0∈曰，and曰is a compact set.Then there exists a global maximizer 0 that solves the problem, =0(X")=arg maxL(x"). -Q Remarks: It is often convenient to solve maxeee In L(X"),where In L(X")is called the log-likelihood function,which is a strictly monotonic increasing function of L(X"). Parameter Estimation and Evaluation Introduction to Statistics and Econometrics April 21,2020 22207
Parameter Estimation and Evaluation Parameter Estimation and Evaluation Introduction to Statistics and Econometrics April 21, 2020 22/207 Maximum Likelihood Estimation Maximum Likelihood Estimation Theorem 1 (8.1). [Existence of MLE] Remarks:
Parameter Estimation and Evaluation Maximum Likelihood Estimation Maximum Likelihood Estimation MLE may not be unique.Given an observed data set x",MLE may be obtained at more than one point in Thus,multiple solutions for MLE are possible. ●The maximum is obtained over parameter space曰，where may be subject to some restriction.For example, when estimating a Generalized AutoRegressive Condi- tional Heteroskedasticity (GARCH)model (Bollerslev 1986),0 may be subject to some restrictions to ensure that the conditional variance is always nonnegative. Parameter Estimation and Evaluation Introduction to Statistics and Econometrics April 21,2020 23/207
Parameter Estimation and Evaluation Parameter Estimation and Evaluation Introduction to Statistics and Econometrics April 21, 2020 23/207 Maximum Likelihood Estimation Maximum Likelihood Estimation
Parameter Estimation and Evaluation Maximum Likelihood Estimation Maximum Likelihood Estimation When In L(X")is twice continuously differentiable with respect to0∈Θ，MLE solution will be easy to find.A necessary condition for MLE is that a must satisfy the first order conditions (FOC): aln L(X") 00 =0=0, which consists of p equations if 0 is a p x 1 parameter vector.From this set of first order conditions,we can solve for 0.Graphically,MLE 0 responds to a zero slope for the likelihood function;see Figure 8.2. Parameter Estimation and Evaluation Introduction to Statistics and Econometrics April 21,2020 24/207
Parameter Estimation and Evaluation Parameter Estimation and Evaluation Introduction to Statistics and Econometrics April 21, 2020 24/207 Maximum Likelihood Estimation Maximum Likelihood Estimation
Parameter Estimation and Evaluation Maximum Likelihood Estimation Maximum Likelihood Estimation To find a global maximum,we need to check the second order condition(SOC):If the pxp sample Hessian matrix 立(0)= ∂2lni(0Xn) 80∂0' is negative definite for all0∈Θ，then0 is the global maximum. In many cases,it may be difficult to verify that H(O)is negative definite for all 0∈Θ. It is relatively straightforward to verify that H()is negative definite,which will imply that 0 is a local maximizer. Parameter Estimation and Evaluation Introduction to Statistics and Econometrics April 21,2020 25/207
Parameter Estimation and Evaluation Parameter Estimation and Evaluation Introduction to Statistics and Econometrics April 21, 2020 25/207 Maximum Likelihood Estimation Maximum Likelihood Estimation