16.322 Stochastic Estimation and Control, Fall 2004 Prof vander velde Lecture 7 Last time: Moments of the Poisson distribution from its generating function G(S) =e u(s-1) ds d-G r= aG u +A =+H-p Example: Using telescope to measure intensity of an object Photon flux photoelectron flux. The number of photoelectrons are poisson distributed. During an observation we cause N photoelectron emissions. Nis the measure of the signal s=N=at u=it 1(S For signal-to-noise ratio of 10, require N=100 photoelectrons. All this follows from the property that the variance is equal to the mean. This is an unbounded experiment, whereas the binomial distribution is for n number of trials 9/30/2004955AM Page 1 of 10
16.322 Stochastic Estimation and Control, Fall 2004 Prof. Vander Velde 9/30/2004 9:55 AM Page 1 of 10 Lecture 7 Last time: Moments of the Poisson distribution from its generating function. ( 1) ( 1) 2 2 ( 1) 2 1 2 2 2 1 1 2 2 22 2 2 ( ) s s s s s s Gs e dG e ds d G e ds dG X ds d G dG X ds ds X X X µ µ µ µ µ µ µ µ σ µ µµ µ − − − = = = = = = = = = + = + = − = +− = = Example: Using telescope to measure intensity of an object Photon flux Î photoelectron flux. The number of photoelectrons are Poisson distributed. During an observation we cause N photoelectron emissions. N is the measure of the signal. 2 2 1 N N N SN t t S t t t S t λ σ µλ λ λ σ λ λ σ = = = = = = ⎛ ⎞ = ⎜ ⎟ ⎝ ⎠ For signal-to-noise ratio of 10, require N = 100 photoelectrons. All this follows from the property that the variance is equal to the mean. This is an unbounded experiment, whereas the binomial distribution is for n number of trials
16.322 Stochastic Estimation and Control, Fall 2004 Prof vander velde 3. The Poisson approximation to the binomial distribution The binomial distribution, like the Poisson, is that of a random variable taking only positive integral values. Since it involves factorials, the binomial distribution is not very convenient for numerical application We shall show under what conditions the poisson expression serves as a good approximation to the binomial expression-and thus may be used for g00 convenience b(k)= p(1-p k!(n-k Consider a large number of trials, n, with small probability of success in each, p, such that the mean of the distribution, np, is of moderate magnitude. Define u= np with n large and p small P Recalling 2n 2en Stirlings formula b(k)= k!(n-k)! n 2丌n2e A2T(n-k)2e-+k n 2e k! as n becomes large relative to k ue The relative error in this approximation is of order of magnitude Rel. Error≈ 9/30/2004955AM Page 2 of 10
16.322 Stochastic Estimation and Control, Fall 2004 Prof. Vander Velde 9/30/2004 9:55 AM Page 2 of 10 3. The Poisson Approximation to the Binomial Distribution The binomial distribution, like the Poisson, is that of a random variable taking only positive integral values. Since it involves factorials, the binomial distribution is not very convenient for numerical application. We shall show under what conditions the Poisson expression serves as a good approximation to the binomial expression – and thus may be used for convenience. ( ) ! ( ) (1 ) ! ! n k nk bk p p knk − = − − Consider a large number of trials, n, with small probability of success in each, p, such that the mean of the distribution, np, is of moderate magnitude. 1 2 1 2 1 2 1 2 1 2 Define with large and small Recalling: ! ~ 2 Stirling's formula lim 1 ! () 1 !( )! 2 1 ! 2( ) ! 1 n n n n n k k k n n k k n n nk k n k n n np n p p n n ne e n n b k kn k n n n e k n nk e n n e k n µ µ µ π µ µ µ µπ µ π µ + − − →∞ − + − − + − + + − + ≡ = ⎛ ⎞ = − ⎜ ⎟ ⎝ ⎠ ⎛ ⎞ = − ⎜ ⎟ − ⎝ ⎠ ⎛ ⎞ ≈ − ⎜ ⎟ ⎝ ⎠ − = 1 2 1 as becomes large relative to ! 1 ! n k n k k k k k k n k e n e n k kee e k µ µ µ µ µ − − + − − − ⎛ ⎞ ⎜ ⎟ − ⎝ ⎠ ⎛ ⎞ ⎜ ⎟ − ⎝ ⎠ ≈ = The relative error in this approximation is of order of magnitude 2 ( ) Rel. Error ~ k n − µ
16.322 Stochastic Estimation and Control, Fall 2004 Prof vander Velde However, for values of k much smaller or larger than u, the probability becomes all sma The normal distribution Outline 1. Describe the common use of the normal distribution 2. The practical employment of the Central Limit Therorem 3. Relation to tabulated functions Normal distribution function Normal error function Complementary error function 1. Describe the common use of the normal distribution Normally distributed variables appear repeatedly in physical situations Voltage across the plate of a vacuum tub angle trac Atmospheric gust velocity Wave height in the open sea 2. The practical employment of the Central limit Therorem X(i=1, 2,, n)are independent random variables Define the sum of these Xi as ∑ s=∑X Then under the condition lim -A=0 9/30/2004955AM Page 3 of 10
16.322 Stochastic Estimation and Control, Fall 2004 Prof. Vander Velde 9/30/2004 9:55 AM Page 3 of 10 However, for values of k much smaller or larger than µ , the probability becomes small. The Normal Distribution Outline: 1. Describe the common use of the normal distribution 2. The practical employment of the Central Limit Therorem 3. Relation to tabulated functions Normal distribution function Normal error function Complementary error function 1. Describe the common use of the normal distribution Normally distributed variables appear repeatedly in physical situations. • Voltage across the plate of a vacuum tube • Radar angle tracking noise • Atmospheric gust velocity • Wave height in the open sea 2. The practical employment of the Central Limit Therorem ( 1,2,..., ) Xi i n = are independent random variables. Define the sum of these Xi as 2 1 1 2 1 i n i i n i i n S X i S X S X σ σ = = = = = = ∑ ∑ ∑ Then under the condition ( ) 3 1 3 lim 0 i i n S n X i X ii X X β σ β β β →∞ = = = = − ∑
16.322 Stochastic Estimation and Control, Fall 2004 Prof vander Velde he limiting distribution of S is the normal distribution Note that this is true for any distributions of the Xi These are sufficient conditions under which the theorem can be proved. It is not lear that they ar Notice that each of the noises mentioned earlier depend on the accumulated effect of a great many small causes e.g., voltage across plate: electrons traveling from cathode to plate It is convenient to work with the characteristic function since we are dealing with the sum of independent variables Normal probability density function f(x) /2 Normal probability distribution function 1。smt 2丌 √2z Where X This integral with the integrand normalized is tabulated. It is called the normal probability function and symbolized with p 9/30/2004955AM Page 4 of 10
16.322 Stochastic Estimation and Control, Fall 2004 Prof. Vander Velde 9/30/2004 9:55 AM Page 4 of 10 the limiting distribution of S is the normal distribution. Note that this is true for any distributions of the Xi. These are sufficient conditions under which the theorem can be proved. It is not clear that they are necessary. Notice that each of the noises mentioned earlier depend on the accumulated effect of a great many small causes e.g., voltage across plate: electrons traveling from cathode to plate. It is convenient to work with the characteristic function since we are dealing with the sum of independent variables. Normal probability density function: 2 2 ( ) 2 1 ( ) 2 x m fx e σ π − − = Normal probability distribution function: 2 2 2 ( ) 2 2 1 ( ) 2 1 2 Where: 1 x u m x m v F x e du e dv m X u m v dv du σ σ πσ πσ σ σ − − −∞ − − −∞ = = = − = = ∫ ∫ This integral with the integrand normalized is tabulated. It is called the normal probability function and symbolized with Φ
16.322 Stochastic Estimation and Control, Fall 2004 Prof vander Velde o (x) d This is a different x. Note the relationship between this and the quantity x previous defined. We use x again here as this is how a is usually written Not only this function but also its first several derivatives which appear in analytic work are tabulated 3. Relation to tabulated functions Even more generally available are the closely related functions Error function: erf(x) Complementary error function cerf(x)=2 Φ(x)=1 p(1) 1 e/m(cos(to y)+jsin(toy)e Differentiation of this form will yield correctly the first 2 moments of the distribution 9/30/2004955AM Page 5 of 10
16.322 Stochastic Estimation and Control, Fall 2004 Prof. Vander Velde 9/30/2004 9:55 AM Page 5 of 10 2 2 1 ( ) 2 x v x e dv π − −∞ Φ = ∫ This is a different x. Note the relationship between this and the quantity x previous defined. We use x again here as this is how Φ is usually written. Not only this function but also its first several derivatives which appear in analytic work are tabulated. 3. Relation to tabulated functions Even more generally available are the closely related functions: Error function: 2 0 2 ( ) x u erf x e du π − = ∫ Complementary error function: 2 2 ( ) u x cerf x e du π ∞ − = ∫ 1 () 1 2 2 x x erf ⎡ ⎤ ⎛ ⎞ Φ= + ⎢ ⎥ ⎜ ⎟ ⎣ ⎦ ⎝ ⎠ () () ( ) 2 2 2 2 2 2 2 2 2 ( ) 2 ( ) 2 2 2 2 2 1 ( ) 2 1 , where 2 1 (cos sin ) 2 2 cos 2 2 2 2 x m jtx y jt m y y jtm y jtm t jtm t jtm t e e dx x m e e dy y e t y j t y e dy e t y e dy e e e σ σ σ σ φ πσ π σ σ σ π σ π π π ∞ − − −∞ ∞ − + −∞ ∞ − −∞ ∞ − −∞ − ⎛ ⎞ ⎜ ⎟ − ⎝ ⎠ = − = = = + = = = ∫ ∫ ∫ ∫ Differentiation of this form will yield correctly the first 2 moments of the distribution