since(TH), (TT)are no longer possible. Secondly, the a-field taken to become FA={SA,,{(HT)},{(HH)}} Thirdly the probability set function become PA(SA)=1,P4(∞)=0,PA({(HT)})=,P4({(HH)} Thus, knowing that event A-one h has occurred in the first trial transformed the original probability space(S, F, p) to the conditional probability space (SA, FA, PA). The question that naturally arises is to what extent we can de- rive the above conditional probabilities without having to transform the original probability space. The following formula provides us with a way to calculate the conditional probability P(41)=P(A14)=P(41n4 xaMl Let A1=I(HT)) and A=((HT), (HH)J, then since P(A1)=4, P(a)=3 P(A1∩4)=P({(HT)})=士 /41 Using the above rule of conditional probability we can deduce that P(A1∩A2)=P(A1|42)P(A2) P(A2A1)·P(A1)forA1,A2∈F This is called the multiplication rule. Moreover, when knowing that A2 has occurred does not change the original probability of A1, i.e P(A1|A2)=P(A1) we say that Al and A2 are independent
since (T H),(TT) are no longer possible. Secondly, the σ-field taken to become FA = {SA, ∅, {(HT)}, {(HH)}}. Thirdly the probability set function become PA(SA) = 1, PA(∅) = 0, PA({(HT)}) = 1 2 , PA({(HH)} = 1 2 . Thus, knowing that event A-one H has occurred in the first trial transformed the original probability space (S, F,P) to the conditional probability space (SA, FA,PA). The question that naturally arises is to what extent we can derive the above conditional probabilities without having to transform the original probability space. The following formula provides us with a way to calculate the conditional probability. PA(A1) = P(A1|A) = P(A1 ∩ A) P(A) . Example: Let A1 = {(HT)} and A = {(HT),(HH)}, then since P(A1) = 1 4 , P(A) = 1 2 , P(A1 ∩ A) = P({(HT)}) = 1 4 , PA(A1) = P(A1|A) = 1/4 1/2 = 1 2 , as above. Using the above rule of conditional probability we can deduce that P(A1 ∩ A2) = P(A1|A2) · P(A2) = P(A2|A1) · P(A1) for A1, A2 ∈ F. This is called the multiplication rule. Moreover, when knowing that A2 has occurred does not change the original probability of A1, i.e. P(A1|A2) = P(A1), we say that A1 and A2 are independent. 11
Independence is very different from mutual exclusiveness in the sense that A1nA2=0 but P(A, A2)+ P(A1) and vice versa can both arise. Independence is a probabilistic statement which ensures that the occurrence of one event does not influence the occurrence (or non-occurrence) of the other event. On the other hand, mutual exclusiveness is a statement which refers to the events(set) themselves not the associated probability. Two events are said to be mutually exclusive when they cannot occur together
Independence is very different from mutual exclusiveness in the sense that A1 ∩A2 = ∅ but P(A1|A2) 6= P(A1) and vice versa can both arise. Independence is a probabilistic statement which ensures that the occurrence of one event does not influence the occurrence (or non-occurrence) of the other event. On the other hand, mutual exclusiveness is a statement which refers to the events (set) themselves not the associated probability. Two events are said to be mutually exclusive when they cannot occur together. 12
3 Random variables and Probability Distribu tions The model based on(S, F, p) does not provide us with a flexible enough frame- work. The basic idea underlying the construction of (S, F, p) was to set up a framework for studying probability of events as a prelude to analyzing problem involving uncertainty. One facet of E which can help us suggest a more flexible probabilities space is the fact when the experiment is performed the outcome is often considered in relation to some quantifiable attribute; i.e. an attribute which can be repressed in numbers. It turns out that assigning numbers to qual itative outcome make possible a much more flexible formulation of probability theory. This suggests that if we could find a consistent way to assign numbers to outcomes we might be able to change(, F, p)to something more easily handled The concept of a random variable is designed to just that without changing the underlying probabilistic structure of(S, F, p) 3.1 The Concept of a random variable Let us consider the possibility of defining a function X( which maps s directly into the real line R. that is assigning a real number ai to each s1 in S by 1=X(s1, al ER, S1 E S. The question arises as to whether every function from s to R will provided us with a consistent way of attaching numbers to elementary events; consistent in the sense of preserving the event structure of the probability space(S, F, P). The answer, unsurprisingly, is not. This is because, although X is a function defined on S, probabilities are assigned to events in F and the issue we have to face is how to define the value taken by x for the different elements of s in a way which preserve the event structures of F. What we require from X-or (X) is to provide us with a correspondence between Rr and S which reflects the event structure of that is, it preserves union, intersections and complement n other word for each subset N of Rx, the inverse image X-(N) must be an event in F. This prompts us to define a random variable X to be any function satisfying this event preserving condition in relation to some a-field defined on Rr: for generality we always take the borel field B on R
3 Random Variables and Probability Distributions The model based on (S, F,P) does not provide us with a flexible enough framework. The basic idea underlying the construction of (S, F,P) was to set up a framework for studying probability of events as a prelude to analyzing problem involving uncertainty. One facet of E which can help us suggest a more flexible probabilities space is the fact when the experiment is performed the outcome is often considered in relation to some quantifiable attribute; i.e. an attribute which can be repressed in numbers. It turns out that assigning numbers to qualitative outcome make possible a much more flexible formulation of probability theory. This suggests that if we could find a consistent way to assign numbers to outcomes we might be able to change (S, F,P) to something more easily handled. The concept of a random variable is designed to just that without changing the underlying probabilistic structure of (S, F,P). 3.1 The Concept of a Random Variable Let us consider the possibility of defining a function X(·) which maps S directly into the real line R, that is, X(·) : S → Rx, assigning a real number x1 to each s1 in S by x1 = X(s1), x1 ∈ R, s1 ∈ S. The question arises as to whether every function from S to Rx will provided us with a consistent way of attaching numbers to elementary events; consistent in the sense of preserving the event structure of the probability space (S, F,P). The answer, unsurprisingly, is not. This is because, although X is a function defined on S, probabilities are assigned to events in F and the issue we have to face is how to define the value taken by X for the different elements of S in a way which preserve the event structures of F. What we require from X −1 (·) or (X) is to provide us with a correspondence between Rx and S which reflects the event structure of F, that is, it preserves union, intersections and complements. In other word for each subset N of Rx, the inverse image X−1 (N) must be an event in F. This prompts us to define a random variable X to be any function satisfying this event preserving condition in relation to some σ-field defined on Rx; for generality we always take the Borel field B on R. 13
Definition 7 A random variable X is a real valued function from s to R which satisfies the condition that for each Borel set B E B on R, the set X-(B)=s: X(sE B,s∈S} is an event in J Example Define the function X-the number of heads", then X(HH=2, X(THD) 1, X(HTI)=l, and X(TT))=0. Further we see that X-(2)=I(HH)I X-(1)=I(TH), (HT)) and X-(0)=I(TT). In fact, it can be shown that the a-field related to the random variables, X. so defined is 3x={S,0,{(HH)},{(TT)},{(TH),(HT)},{(HH),(TT)} {(HT),(TH),(HH)},{(HT),(TH),(TT)} We can verify that X-(OU(1)=((HT), (TH), ( TT)E Fx, X-GOU {2})={(HH),(TT)}∈ x and X-1({1})u{2})={(HT),(TH),(HH)}∈Fx Ex Consider the random variable Y-number of Head in the first trial". then Y(HHD=Y(HTI=l, and Y(TT)=Y(TH)=0. However Y does not preserve the event structure of Fx since Y-(0=i(TH), (TT)) is not an event in Fx and so does Y-(1)=I(HH), (HT)I From the two examples above, we see that the question"X(: S-+Rx is a random variable "does not make any sense unless some a-field F is also speci- fied. In the case of the function X-number of heads, in the coin-tossing example we see that it is a random variable relative to the a-field Fx. This. however does not preclude y from being a random variable with respect to some other a-field FY; for instance FY=S, 0, (HH), (HT)J, (TH), (TT). Intuition gests that for al value function X(:S-R we should be able define a a-field x on s such that X is a random variable. The concept of a g-field generated by a random variable enables us to concentrate on particular aspects of an experiment without having to consider everything associated with the experiment at the same time. Hence when we choose to define a random variable and the associated a-field we make an implicit choice about the features of the random experiment we are interested in
Definition 7: A random variable X is a real valued function from S to R which satisfies the condition that for each Borel set B ∈ B on R, the set X −1 (B) = {s : X(s) ∈ B, s ∈ S} is an event in F. Example: Define the function X—”the number of heads”, then X({HH}) = 2, X({T H}) = 1, X({HT}) = 1, and X({TT}) = 0. Further we see that X −1 (2) = {(HH)}, X−1 (1) = {(T H),(HT)} and X−1 (0) = {(TT)}. In fact, it can be shown that the σ-field related to the random variables, X, so defined is FX = {S, ∅, {(HH)}, {(TT)}, {(T H),(HT)}, {(HH),(TT)}, {(HT),(T H),(HH)}, {(HT),(T H),(TT)}}. We can verify that X−1 ({0}) ∪ {1}) = {(HT),(T H),(TT)} ∈ FX, X−1 ({0}) ∪ {2}) = {(HH),(TT)} ∈ FX and X−1 ({1})∪{2}) = {(HT),(T H),(HH)} ∈ FX. Example: Consider the random variable Y —”number of Head in the first trial”, then Y ({HH}) = Y ({HT}) = 1, and Y ({TT}) = Y ({T H}) = 0. However Y does not preserve the event structure of FX since Y −1 ({0}) = {(T H),(TT)} is not an event in FX and so does Y −1 ({1}) = {(HH),(HT)} From the two examples above, we see that the question ”X(·) : S → RX is a random variable ?” does not make any sense unless some σ-field F is also speci- fied. In the case of the function X–number of heads, in the coin-tossing example we see that it is a random variable relative to the σ-field FX. This, however, does not preclude Y from being a random variable with respect to some other σ-field FY ; for instance FY = {S, ∅, {(HH),(HT)}, {(T H),(TT)}}. Intuition suggests that for any real value function X(·) : S → R we should be able to define a σ-field FX on S such that X is a random variable. The concept of a σ-field generated by a random variable enables us to concentrate on particular aspects of an experiment without having to consider everything associated with the experiment at the same time. Hence, when we choose to define a random variable and the associated σ-field we make an implicit choice about the features of the random experiment we are interested in. 14
How do we decide that some function X(:S-R is a random variables relative to a given a-field F? From the discussion of the a-field o()generated by the set J=Br: a E R where B2=(oo, c we know that B= a(J) and if X( is such that X-1(-∞,x])={s:X(s)∈(-∞,x,s∈S}∈ F for all(-∞,x∈B then X-(B)={s:X(s)∈B,s∈S}∈ F for all B∈B In other words. when we want to establish that x is a random variables or define Pr( we have to look no further than the half-closed interval(oo, a],and the a-field o() they generate, whatever the range Rr. Let us use the shorthand notation{X(s)≤} instead of{s:X(s)∈(-∞,r,s∈S} to the above examp X-1(-∞,x])={s:X(s)≤x} {(TT)} X≤0( That is r=0), I(TT)(TH(HT)I X<IThat is T=1) I(TT)(TH(HT(HH) X≤2( That is z=2) we can see that X-(oo, d)E Fx for all a E R and thus X( is a random variables with respect to Fx A random variable X relative to F maps S into a subset of the real line, and the borel field B on R plays now the role of F. In order to complete the model we need to assign probabilities to the elements b of B. Common sense suggests that the assignment of the probabilities to the event b E B must be consistent with the probabilities assigned to the corresponding events in F. Formally, we need to define a set function Px(: B-0, 1 such that Px(B)=P(X-1(B)≡P(s:X(s)∈B,s∈S) for all B∈B. For example, in the above example, P2({0})=1/4,P2({1}=1/2,P2({2}=1/4andP2({0}U{1})=3/4
How do we decide that some function X(·) : S → R is a random variables relative to a given σ-field F ? From the discussion of the σ-field σ(J) generated by the set J = {Bx : x ∈ R} where Bx = (−∞, x] we know that B = σ(J) and if X(·) is such that X −1 ((−∞, x]) = {s : X(s) ∈ (−∞, x], s ∈ S} ∈ F for all (−∞, x] ∈ B, then X −1 (B) = {s : X(s) ∈ B, s ∈ S} ∈ F for all B ∈ B. In other words, when we want to establish that X is a random variables or define Px(·) we have to look no further than the half-closed interval (−∞, x], and the σ-field σ(J) they generate, whatever the range Rx. Let us use the shorthand notation {X(s) ≤ x} instead of {s : X(s) ∈ (−∞, x], s ∈ S} to the above example, X −1 ((−∞, x]) = {s : X(s) ≤ x} = ∅ X < 0, {(TT)} X ≤ 0 (That is x = 0), {(TT)(T H)(HT)} X ≤ 1 (That is x = 1), {(TT)(T H)(HT)(HH)} X ≤ 2 (That is x = 2), we can see that X−1 ((−∞, x]) ∈ FX for all x ∈ R and thus X(·) is a random variables with respect to FX. A random variable X relative to F maps S into a subset of the real line, and the Borel field B on R plays now the role of F. In order to complete the model we need to assign probabilities to the elements B of B. Common sense suggests that the assignment of the probabilities to the event B ∈ B must be consistent with the probabilities assigned to the corresponding events in F. Formally, we need to define a set function PX(·) : B → [0, 1] such that PX (B) = P(X−1 (B)) ≡ P(s : X(s) ∈ B, s ∈ S) for all B ∈ B. For example, in the above example, Px({0}) = 1/4, Px({1}) = 1/2, Px({2}) = 1/4 and Px({0} ∪ {1}) = 3/4. 15