Notation Our notation will generally indicate the dependence of probabilities and expected values on e For example, we will denote by egl(x)i the expected value of a random variable h(X)as a function of e Similarly, we will use the notation Pe(A)to denote the probability of an event A
Notation ◼ Our notation will generally indicate the dependence of probabilities and expected values on 𝜃. ◼ For example, we will denote by 𝐸𝜃 ℎ 𝑋 the expected value of a random variable ℎ 𝑋 as a function of 𝜃. ◼ Similarly, we will use the notation 𝑃𝜃 𝐴 to denote the probability of an event 𝐴
Content a Classical parameter estimation Linear regression Binary Hypothesis Testing Significance Testing
Content ◼ Classical Parameter Estimation ◼ Linear Regression ◼ Binary Hypothesis Testing ◼ Significance Testing
Given observationsⅩ=(1,…,!n),an estimator is a random variable of the form 0=g(X), for some function g Note that since the distribution of X depends on 0 the same is true for the distribution of o We use the term estimate to refer to an actual realized value of e
◼ Given observations 𝑋 = 𝑋1,… ,𝑋𝑛 , an estimator is a random variable of the form Θ = 𝑔 𝑋 , for some function 𝑔. ◼ Note that since the distribution of 𝑋 depends on 𝜃, the same is true for the distribution of Θ . ◼ We use the term estimate to refer to an actual realized value of Θ
Sometimes, particularly when we are interested in the role of the number of observations n, we use the notation o for an estimator a It is then also appropriate to view On as a sequence of estimators o One for each value of n a The mean and variance of e are denoted Een and vare on respectively a We sometimes drop this subscript 0 when the context is clear
◼ Sometimes, particularly when we are interested in the role of the number of observations 𝑛, we use the notation Θ𝑛 for an estimator. ◼ It is then also appropriate to view Θ𝑛 as a sequence of estimators. ❑ One for each value of 𝑛. ◼ The mean and variance of Θ 𝑛 are denoted 𝐸𝜃 Θ𝑛 and 𝑣𝑎𝑟𝜃 Θ𝑛 , respectively. ❑ We sometimes drop this subscript 𝜃 when the context is clear
Terminology regarding estimators Estimator:o a function of n observations for an(X1,., Xn)whose distribution depends on日. Estimation error: 0n=0n -8 Bias of the estimator: be(0n=Ee0n-0,is the expected value of the estimation error
Terminology regarding estimators ◼ Estimator: Θ 𝑛, a function of 𝑛 observations for an 𝑋1,… , 𝑋𝑛 whose distribution depends on 𝜃. ◼ Estimation error: Θ෩𝑛 = Θ 𝑛 − 𝜃. ◼ Bias of the estimator: 𝑏𝜃 Θ 𝑛 = 𝐸𝜃 Θ 𝑛 − 𝜃, is the expected value of the estimation error