Data Mining with Bayesian Networks(1)Instructor: Qiang YangHong Kong University of Science and TechnologyQyang@cs.ust.hkThanks:DanWeld,EibeFrank
Data Mining with Bayesian Networks (I) Instructor: Qiang Yang Hong Kong University of Science and Technology Qyang@cs.ust.hk Thanks: Dan Weld, Eibe Frank
Goal: Mining ProbabilityModelsProbability BasicsOur state s in world W, is distributed according toprobability distribution 0 <= Pr(s) <= 1 for all sES Pr(s)= 1For subsets S1 and S2Pr(S1 U S2) = Pr(s1) + Pr(s2) - Pr(s1 S2)Bayes Rule:
Goal: Mining Probability Models n Probability Basics n Our state s in world W, is distributed according to probability distribution n 0 <= Pr(s) <= 1 for all s n S Pr(s) = 1 n For subsets S1 and S2, Pr(S1 S2) = Pr(s1) + Pr(s2) - Pr(s1 S2) n Bayes Rule:
Weather data setPlayOutlookHumidityWindyTemperaturehothighFALSEnosunnyhothighTRUEnosunnyhighhotFALSEovercastyeshighmildrainyFALSEyescoolnormalrainyFALSEyescoolrainynormalTRUEnocoolnormalTRUEyesovercasthighmildFALSEnosunnycoolnormalFALSEyessunnymildrainynormalFALSEyesmildnormalTRUEyessunnyhighmildTRUEovercastyeshotnormalFALSEovercastyesmildhighTRUErainyno
Weather data set Outlook Temperature Humidity Windy Play sunny hot high FALSE no sunny hot high TRUE no overcast hot high FALSE yes rainy mild high FALSE yes rainy cool normal FALSE yes rainy cool normal TRUE no overcast cool normal TRUE yes sunny mild high FALSE no sunny cool normal FALSE yes rainy mild normal FALSE yes sunny mild normal TRUE yes overcast mild high TRUE yes overcast hot normal FALSE yes rainy mild high TRUE no
BasicsUnconditional or Prior ProbabilityPr(Play=yes) + Pr(Play=no)=1: Pr(Play=yes) is sometimes written as Pr(Play) Table has 9 yes, 5 no Pr(Play=yes)=9/(9+5)=9/14Thus, Pr(Play=no)=5/14JointProbabilityof Playand Windy Pr(Play=x,Windy=y) for all values x and y, should be 1Windy=True/Windy=False3/146/14Play=yes?3/14Play=no
Basics n Unconditional or Prior Probability n Pr(Play=yes) + Pr(Play=no)=1 n Pr(Play=yes) is sometimes written as Pr(Play) n Table has 9 yes, 5 no n Pr(Play=yes)=9/(9+5)=9/14 n Thus, Pr(Play=no)=5/14 n Joint Probability of Play and Windy: n Pr(Play=x,Windy=y) for all values x and y, should be 1 Play=yes Play=no Windy=True Windy=False 3/14 3/14 ? 6/14
Probability BasicsPlayWindyConditional Probability*FALSEnoPr(A/B)TRUEno# (Windy=False)=8*yes*FALSEWithin the 8,*yes*FALSE#(Play=yes)=6Pr(Play=yes / Windy=False)*yes*FALSE=6/8TRUEnoPr(Windy=False)=8/14TRUEyesPr(Play=Yes)=9/14ApplyingBayesRule*FALSEnoPr(B|A) = Pr(A/B)Pr(B) / Pr(A)*yes*FALSEPr(Windy=False|Play=yes)=*yes*FALSE6/8*8/14/(9/14)=6/9TRUEyesTRUEyes*yes*FALSETRUEno
Probability Basics n Conditional Probability n Pr(A|B) n # (Windy=False)=8 n Within the 8, n #(Play=yes)=6 n Pr(Play=yes | Windy=False) =6/8 n Pr(Windy=False)=8/14 n Pr(Play=Yes)=9/14 n Applying Bayes Rule n Pr(B|A) = Pr(A|B)Pr(B) / Pr(A) n Pr(Windy=False|Play=yes)= 6/8*8/14/(9/14)=6/9 Windy Play *FALSE no TRUE no *FALSE *yes *FALSE *yes *FALSE *yes TRUE no TRUE yes *FALSE no *FALSE *yes *FALSE *yes TRUE yes TRUE yes *FALSE *yes TRUE no