Estimating A-Posteriori Probabilities w How do we compute P(CX) 罐 Bayes theorem: P(C|Ⅹ)=P(ⅩC)P(C)/P w P(X is constant for all classes MP(C)= relative freq of class c samples NC such that P(cX) is maximum C such that P(XIC)P(C is maximum N Problem: computing P(XC) is not feasible
16 Estimating A-Posteriori Probabilities How do we compute P(C|X). Bayes theorem: P(C|X) = P(X|C)·P(C) / P(X) P(X) is constant for all classes. P(C) = relative freq of class C samples C such that P(C|X) is maximum = C such that P(X|C)·P(C) is maximum Problem: computing P(X|C) is not feasible!
The Naive Bayesian Approach wa Naive assumption All attributes are mutually conditionally independent P(x1x…,×kC)=P(×1C)∵…P(C) If i-th attribute is categorical P(XIC)is estimated as the relative freq of samples having value x; as i-th attribute in class C w If i-th attribute is continuous PX IC)is estimated thru a Gaussian density function M Computationally easy in both cases
17 The Naïve Bayesian Approach Naïve assumption: – All attributes are mutually conditionally independent P(x1 ,…,xk |C) = P(x1 |C)·…·P(xk |C) If i-th attribute is categorical: – P(xi |C) is estimated as the relative freq of samples having value xi as i-th attribute in class C If i-th attribute is continuous: – P(xi |C) is estimated thru a Gaussian density function Computationally easy in both cases
An Example Using The Naive Bayesian Approach Luk Tang Pong Cheng B/s Buy Sell Buy Buy Buy Sell Buy Sell Hold Sell Buy Buy Sell BuyBuy Buy Sell Hold Sel Buy BBSSSB Sell Hold Sell Sell HoldHold Sell Sell Buy Buy Buy Buy Buy Hold Sell Buy Sell Buy Sell Buy Buy Buy Sell Sell Hold Buy Buy Sell Hold Sell Sell Buy SBSSSSSB Sel‖! BuyBuy Sell
18 An Example Using The Naïve Bayesian Approach Luk Tang Pong Cheng B/S Buy Sell Buy Buy B Buy Sell Buy Sell B Hold Sell Buy Buy S Sell Buy Buy Buy S Sell Hold Sell Buy S Sell Hold Sell Sell B Hold Hold Sell Sell S Buy Buy Buy Buy B Buy Hold Sell Buy S Sell Buy Sell Buy S Buy Buy Sell Sell S Hold Buy Buy Sell S Hold Sell Sell Buy S Sell Buy Buy Sell B
The Example Continued On one particular day, X=<Sell, Sell, Buy, Buy> P( Sell), P(Sell) P(Sell Sell) P(Sell Sell), P(Buy Sell), P(Buy Sell )P(Se)=3/92/93/9:6/99/14=0.010582 P(X Buy)' P(Buy) P(Sell Buy)' P(Sell Buy)' P(Buy Buy)' P(Buy Bu y)P(Buy)=2/52/54/52/5·5/14=0.018286 You should buy
19 The Example Continued On one particular day, X=<Sell,Sell,Buy,Buy> – P(X|Sell)·P(Sell)= P(Sell|Sell)·P(Sell|Sell)·P(Buy|Sell)·P(Buy|Sell )·P(Sell) = 3/9·2/9·3/9·6/9·9/14 = 0.010582 – P(X|Buy)·P(Buy) = P(Sell|Buy)·P(Sell|Buy)·P(Buy|Buy)·P(Buy|Bu y)·P(Buy) = 2/5·2/5·4/5·2/5·5/14 = 0.018286 You should Buy
Advantages of The Bayesian Approach s Probabilistic Calculate explicit probabilities Incremental Additional example can incrementally increase/decrease a class probability. w Probabilistic classification Classify into multiple classes weighted by their probabilities a Standard Though computationally intractable, the approach can provide a standard of optimal decision making 20
20 Advantages of The Bayesian Approach Probabilistic. – Calculate explicit probabilities. Incremental. – Additional example can incrementally increase/decrease a class probability. Probabilistic classification. – Classify into multiple classes weighted by their probabilities. Standard. – Though computationally intractable, the approach can provide a standard of optimal decision making