CHAPTER 2 Axioms of Probability 2.1 INTRODUCTION 2.2 SAMPLE SPACE AND EVENTS 2.3 AXIOMS OF PROBABILITY 2.4 SOME SIMPLE PROPOSITIONS 2.5 SAMPLE SPACES HAVING EQUALLY LIKELY OUTCOMES 2.6 PROBABILITY AS A CONTINUOUS SET FUNCTION 2.7 PROBABILITY AS A MEASURE OF BELIEF 2.1 INTRODUCTION hapter we introduce the concept of the probability of an event and how prob computed in certain we need the concept of the sample space and the events of an experiment. 2.2 SAMPLE SPACE AND EVENTS y.However of the e that the set of all r an exneriment is k own as the sample space of the experiment and isdenoted by5. Following are some examples: 1.If the outcome of an experiment consists in the determination of the sex of a newborn child,then S=(8,b) where the outcome g means that the child is a girl and b that it is a boy. 2.If the outcome of an experiment is the order of finish in a race among the 7 horses having post positions 1,2,3,4,5,6,and 7,then S=(all 7!permutations of (1.2.3,4.5,6,7)) The outc 3.If the experiment consists of flipping two coins,then the sample space consists of the following four points: S={(H,H,(H,T,(T,),(T,T)1 e s
CHAPTER 2 Axioms of Probability 2.1 INTRODUCTION 2.2 SAMPLE SPACE AND EVENTS 2.3 AXIOMS OF PROBABILITY 2.4 SOME SIMPLE PROPOSITIONS 2.5 SAMPLE SPACES HAVING EQUALLY LIKELY OUTCOMES 2.6 PROBABILITY AS A CONTINUOUS SET FUNCTION 2.7 PROBABILITY AS A MEASURE OF BELIEF 2.1 INTRODUCTION In this chapter, we introduce the concept of the probability of an event and then show how probabilities can be computed in certain situations. As a preliminary, however, we need the concept of the sample space and the events of an experiment. 2.2 SAMPLE SPACE AND EVENTS Consider an experiment whose outcome is not predictable with certainty. However, although the outcome of the experiment will not be known in advance, let us suppose that the set of all possible outcomes is known. This set of all possible outcomes of an experiment is known as the sample space of the experiment and is denoted by S. Following are some examples: 1. If the outcome of an experiment consists in the determination of the sex of a newborn child, then S = {g, b} where the outcome g means that the child is a girl and b that it is a boy. 2. If the outcome of an experiment is the order of finish in a race among the 7 horses having post positions 1, 2, 3, 4, 5, 6, and 7, then S = {all 7! permutations of (1, 2, 3, 4, 5, 6, 7)} The outcome (2, 3, 1, 6, 5, 4, 7) means, for instance, that the number 2 horse comes in first, then the number 3 horse, then the number 1 horse, and so on. 3. If the experiment consists of flipping two coins, then the sample space consists of the following four points: S = {(H, H),(H, T),(T, H),(T, T)} The outcome will be (H, H) if both coins are heads, (H, T) if the first coin is heads and the second tails, (T, H) if the first is tails and the second heads, and (T, T) if both coins are tails. 22
Section 2.2 Sample Space and Events 23 4 If the S=,:ij=1,2,3.4,5,6 where the 5.If the experiment consists of measuring (in hours)the life of a transistor, then the sample space consists of all nonnegative real numbers;that is. S=x:0≤x<oo Any subset E of the sample space is known as an event.In other words,an event is a set consisting of possible outcomes of the experiment.If the outcome of the experi- ment is contained in E,then we say that E has occurred.Following are some examples of events. In the preceding Example 1,if E=(g),then E is the event that the child is a girl. Similarly,if F=(b),then F is the event that the child is a boy. In Example 2,if E=(all o omes in S starting with a 3) then E is the event that horse 3 wins the race. In Example 3,if E=((H,H),(H,T)),then E is the event that a head appears on the first coin. In Example 4,if E=((1,6),(2,5),(3,4),(4,3),(5,2),(6,1)),then E is the event that the sum of the dice equals7 In Example 5,if E =(x:0sxs 5),then E is the event that the transistor does not last longer than 5 hours e new ev ent E U F outcomes that are eith or or in both nd is,the ccur if either E or F occurs.For instance,in Example 1,if event E U F=g.bl s the whole sample space S.In Example 3,if E=((H,H).(H,T))and E U F=((H,H).(H,T).(T.H)) Thus,EU Fwould occur if a head appeared on either coin Sieven o the event E and the cvent may also det alled to cons ome t are n and i event nstance,.in only both Ea For 1 occurs and Fp (T. the event t I tail occurs,then H),(T.T)is the event that at least EF=((H.T),(T.H)) is the that exactly 1 head and 1 tail occur.In example 4,if E=((1.6).(2.5) is the e sum is e event
Section 2.2 Sample Space and Events 23 4. If the experiment consists of tossing two dice, then the sample space consists of the 36 points S = {(i, j): i, j = 1, 2, 3, 4, 5, 6} where the outcome (i, j) is said to occur if i appears on the leftmost die and j on the other die. 5. If the experiment consists of measuring (in hours) the lifetime of a transistor, then the sample space consists of all nonnegative real numbers; that is, S = {x: 0 … x < q} Any subset E of the sample space is known as an event. In other words, an event is a set consisting of possible outcomes of the experiment. If the outcome of the experiment is contained in E, then we say that E has occurred. Following are some examples of events. In the preceding Example 1, if E = {g}, then E is the event that the child is a girl. Similarly, if F = {b}, then F is the event that the child is a boy. In Example 2, if E = {all outcomes in S starting with a 3} then E is the event that horse 3 wins the race. In Example 3, if E = {(H, H),(H, T)}, then E is the event that a head appears on the first coin. In Example 4, if E = {(1, 6),(2, 5),(3, 4),(4, 3),(5, 2),(6, 1)}, then E is the event that the sum of the dice equals 7. In Example 5, if E = {x: 0 … x … 5}, then E is the event that the transistor does not last longer than 5 hours. For any two events E and F of a sample space S, we define the new event E ∪ F to consist of all outcomes that are either in E or in F or in both E and F. That is, the event E ∪ F will occur if either E or F occurs. For instance, in Example 1, if event E = {g} and F = {b}, then E ∪ F = {g, b} That is, E ∪ F is the whole sample space S. In Example 3, if E = {(H, H),(H, T)} and F = {(T, H)}, then E ∪ F = {(H, H),(H, T),(T, H)} Thus, E ∪ F would occur if a head appeared on either coin. The event E ∪ F is called the union of the event E and the event F. Similarly, for any two events E and F, we may also define the new event EF, called the intersection of E and F, to consist of all outcomes that are both in E and in F. That is, the event EF (sometimes written E ∩ F) will occur only if both E and F occur. For instance, in Example 3, if E = {(H, H),(H, T),(T, H)} is the event that at least 1 head occurs and F = {(H, T),(T, H),(T, T)} is the event that at least 1 tail occurs, then EF = {(H, T),(T, H)} is the event that exactly 1 head and 1 tail occur. In example 4, if E = {(1, 6),(2, 5), (3, 4),(4, 3),(5, 2),(6, 1)} is the event that the sum of the dice is 7 and F = {(1, 5),(2, 4), (3, 3),(4, 2),(5, 1)} is the event that the sum is 6, then the event EF does not contain
24 Chapter 2 Axioms of Probability any outcomes and hence could not occur.To give such an event a name,we shall refer to it as the nu 8m二 then E and are said If E1,E2....are events,then the union of these events,denoted by En,is defined to be that event which consists of all outcomes that are in En for at least one value of n =1.2.....Similarly,the intersection of the events En,denoted by En,is defined to be the event consisting of those outcomes which are in all of the events F.n=12 Finally,for any event E,we define the new event E,referred to as the com- plement of E,to consist of all outcomes in the sample space S that are not in E. That is,E will occur if and only if E does not occur.In Example 4,if event E ((1,6),(2,5),(3,4),(4,3),(5,2),(6,1)),then Ee will occur when the sum of the dice does not equal 7.Note that because the experiment must result in some outcome,it follows that Se=. For any two events E and F,if all of the outcomes in E are also in F,then we say that Eis contained in F,or Eisa subset of F,and write ently.F occurrence of FitecF and F C E.we say that E and Farurreno ctimes say as en th occurrence sentation that is useful for illustrating logical relations amon s the diagram ge rcct of all the and nle space as cons circles uithin the of inte can then he indicated by shadin gions of the diao ram for inst the three venn diagra ms shown in Fi re 2.1.the shaded areas repre ent r tively,the events EU F.EF.and E.The Venn diagram in Figure 2.2 indicates that E C F. (a)Shaded region:EUF (b)Shaded region:E (c)Shaded region:E. FIGURE 2.1:Venn Diagrams
24 Chapter 2 Axioms of Probability any outcomes and hence could not occur. To give such an event a name, we shall refer to it as the null event and denote it by Ø. (That is, Ø refers to the event consisting of no outcomes.) If EF = Ø, then E and F are said to be mutually exclusive. We define unions and intersections of more than two events in a similar manner. If E1, E2, ... are events, then the union of these events, denoted by q n=1 En, is defined to be that event which consists of all outcomes that are in En for at least one value of n = 1, 2, .... Similarly, the intersection of the events En, denoted by q n=1 En, is defined to be the event consisting of those outcomes which are in all of the events En, n = 1, 2, .... Finally, for any event E, we define the new event Ec, referred to as the complement of E, to consist of all outcomes in the sample space S that are not in E. That is, Ec will occur if and only if E does not occur. In Example 4, if event E = {(1, 6),(2, 5),(3, 4),(4, 3),(5, 2),(6, 1)}, then Ec will occur when the sum of the dice does not equal 7. Note that because the experiment must result in some outcome, it follows that Sc = Ø. For any two events E and F, if all of the outcomes in E are also in F, then we say that E is contained in F, or E is a subset of F, and write E ( F (or equivalently, F ) E, which we sometimes say as F is a superset of E). Thus, if E ( F, then the occurrence of E implies the occurrence of F. If E ( F and F ( E, we say that E and F are equal and write E = F. A graphical representation that is useful for illustrating logical relations among events is the Venn diagram. The sample space S is represented as consisting of all the outcomes in a large rectangle, and the events E, F, G, ... are represented as consisting of all the outcomes in given circles within the rectangle. Events of interest can then be indicated by shading appropriate regions of the diagram. For instance, in the three Venn diagrams shown in Figure 2.1, the shaded areas represent, respectively, the events E ∪ F, EF, and Ec. The Venn diagram in Figure 2.2 indicates that E ( F. E F E F S S (a) Shaded region: E F. (b) Shaded region: EF. S (c) Shaded region: Ec . E FIGURE 2.1: Venn Diagrams
Section 2.2 Sample Space and Events 25 FIGURE 2.2:E C F The operations of forming unions inter nts of events obey certain rules similar to the rules of algebra.We list a few of these rules: Commutative laws EUF-FUE EF=FE Associative laws (EUF)UG=EU(FUG)(EF)G=E(FG) Distributive laws (EUF)G=EGUFG EFUG=(EUG)(FUG) These relations verified by showing that any outcome that is contained in the event on the left side of the equality s en is also contained in the event on the right grams.For Figure 2.3. (a)Shaded region:EG (b)Shaded region:FG (c)Shaded region:(EUF)G FIGURE 2.3:(EUF)G=EGUFG
Section 2.2 Sample Space and Events 25 S F E FIGURE 2.2: E ( F The operations of forming unions, intersections, and complements of events obey certain rules similar to the rules of algebra. We list a few of these rules: Commutative laws E ∪ F = F ∪ E EF = FE Associative laws (E ∪F) ∪ G = E ∪(F ∪ G) (EF)G = E(FG) Distributive laws (E ∪F)G = EG ∪FG EF ∪ G = (E ∪ G)(F ∪ G) These relations are verified by showing that any outcome that is contained in the event on the left side of the equality sign is also contained in the event on the right side, and vice versa. One way of showing this is by means of Venn diagrams. For instance, the distributive law may be verified by the sequence of diagrams in Figure 2.3. E F (a) Shaded region: EG. G E F (b) Shaded region: FG. G E F (c) Shaded region: (E F )G. G FIGURE 2.3: (E ∪F)G = EG ∪ FG
26 Chapter 2 Axioms of Probability The following useful relationships between the three basic operations of forming unions,intersections,and complements are known as DeMorgan's laws: (g-s j-gs To prove DeMorga'pe huom ofThen x is not contained in E.which means that x is not contained in any of the events Ei=1,2.....n,implying that x is contained in Ef for alli=1,2.....n and thus is contained inE.To go the other way,suppose that x is an outcome ofE.Then x is contained in E for all i=1.2.....n,which means that x is not contained in E:for anyi=1,2....n,implying that x is not contained in E,in turn implying that x is contained in .This proves the first of DeMorgan's laws. To prove the second of DeMorgan's laws,we use the first law to obtain （0-0 which,since(E)=E,is equivalent to )-0 Taking complements of both sides of the preceding equation yields the result we seek. namely, 05-0j 2.3 AXIOMS OF PROBABILITY One way of defining the probability of an event is in terms of its relative frequency. Such a definition usually goes as follows:We suppose that an experiment,whose sam- ple space is S,is repeatedly performed under exactly the same conditions.For each event E of the sample space S.we define n(E)to be the number of times in the first n repetitions of the experiment that the event Eoccurs.Then P(E),the probability of the event E,is defined as
26 Chapter 2 Axioms of Probability The following useful relationships between the three basic operations of forming unions, intersections, and complements are known as DeMorgan’s laws: ⎛ ⎝n i=1 Ei ⎞ ⎠ c = n i=1 Ec i ⎛ ⎝n i=1 Ei ⎞ ⎠ c = n i=1 Ec i To prove DeMorgan’s laws, suppose first that x is an outcome of n i=1 Ei c . Then x is not contained in n i=1 Ei, which means that x is not contained in any of the events Ei, i = 1, 2, ... , n, implying that x is contained in Ec i for all i = 1, 2, ... , n and thus is contained in n i=1 Ec i . To go the other way, suppose that x is an outcome of n i=1 Ec i . Then x is contained in Ec i for all i = 1, 2, ... , n, which means that x is not contained in Ei for any i = 1, 2, ... , n, implying that x is not contained in n i Ei, in turn implying that x is contained in n 1 Ei c . This proves the first of DeMorgan’s laws. To prove the second of DeMorgan’s laws, we use the first law to obtain ⎛ ⎝n i=1 Ec i ⎞ ⎠ c = n i=1 (Ec i) c which, since (Ec)c = E, is equivalent to ⎛ ⎝n 1 Ec i ⎞ ⎠ c = n 1 Ei Taking complements of both sides of the preceding equation yields the result we seek, namely, n 1 Ec i = ⎛ ⎝n 1 Ei ⎞ ⎠ c 2.3 AXIOMS OF PROBABILITY One way of defining the probability of an event is in terms of its relative frequency. Such a definition usually goes as follows: We suppose that an experiment, whose sample space is S, is repeatedly performed under exactly the same conditions. For each event E of the sample space S, we define n(E) to be the number of times in the first n repetitions of the experiment that the event E occurs. Then P(E), the probability of the event E, is defined as P(E) = limn→q n(E) n