Choosing the SplittingAttribute At each node, available attributes are evaluated onthe basis of separating the classes of the trainingexamples. A Goodness function is used for thispurpose.Typical goodness functions:informationgain (ID3/C4.5)information gain ratiogini index6
6 Choosing the Splitting Attribute ◼ At each node, available attributes are evaluated on the basis of separating the classes of the training examples. A Goodness function is used for this purpose. ◼ Typical goodness functions: ◼ information gain (ID3/C4.5) ◼ information gain ratio ◼ gini index
Which attribute to select?outlookhumidityhighnormalrainysunnyovercastyesyesyesyesyesyesyesyesyesyesnoyesyesyesyesnononoyesyesnononoyesnoyesnonowindytemperaturefalsetruemildcoolhotyesyesyesyesyesyesyesyesyesyesyesyesyesyesyesnonoyesyesyesnononononononono7
7 Which attribute to select?
A criterion for attributeselectionWhich is the best attribute?Theonewhichwill result inthesmallesttree Heuristic: choose the attribute that produces the "purest"nodesPopular impurity criterion: information gain Information gain increases with the average purity of thesubsets that an attribute producesStrategy: choose attribute that results in greatestinformation gain8
8 A criterion for attribute selection ◼ Which is the best attribute? ◼ The one which will result in the smallest tree ◼ Heuristic: choose the attribute that produces the “purest” nodes ◼ Popular impurity criterion: information gain ◼ Information gain increases with the average purity of the subsets that an attribute produces ◼ Strategy: choose attribute that results in greatest information gain