Topics in Artificial IntelligenceMachine LearningTechniques forData MiningBSemester20002026/9/22Universityof Waikato
2026/9/22 University of Waikato 1 Machine Learning Techniques for Data Mining B Semester 2000 Topics in Artificial Intelligence:
SimplicityfirstSimple algorithms often work surprisingly wellMany different kinds of simple structure exist:One attributemight doall theworkAllattributes might contribute independently withequalimportanceAlinear combination might be sufficientAninstance-based representation mightworkbestSimplelogical structuresmightbeappropriateHowto evaluate theresult?2026/9/22UniversityofWaikato
2026/9/22 University of Waikato 2 Simplicity first ◼ Simple algorithms often work surprisingly well ◼ Many different kinds of simple structure exist: ◆ One attribute might do all the work ◆ All attributes might contribute independently with equal importance ◆ A linear combination might be sufficient ◆ An instance-based representation might work best ◆ Simple logical structures might be appropriate ◼ How to evaluate the result?
Inferring rudimentary rules1R:learns a 1-level decision treeIn other words,generates a set of rulesthat all teston oneparticular attributeBasic version (assumingnominalattributes)Onebranchforeachof theattribute'svaluesEachbranchassignsmostfrequentclassError rate: proportion of instances that don't belongtothemajority class of their corresponding branchChooseattributewithlowest errorrate2026/9/22UniversityofWaikato
2026/9/22 University of Waikato 3 Inferring rudimentary rules ◼ 1R: learns a 1-level decision tree ◆ In other words, generates a set of rules that all test on one particular attribute ◼ Basic version (assuming nominal attributes) ◆ One branch for each of the attribute’s values ◆ Each branch assigns most frequent class ◆ Error rate: proportion of instances that don’t belong to the majority class of their corresponding branch ◆ Choose attribute with lowest error rate
Pseudo-codefor1RFor each attribute,For each value of the attribute, make a rule as follows:count how often each class appearsfind the most freguent classmake the rule assign that class to this attribute-valueCalculate the error rate of the rulesChoose the rules with the smallest error rateNote:“"missing”is always treated as a separateattribute value2026/9/22UniversityofWaikato
2026/9/22 University of Waikato 4 Pseudo-code for 1R For each attribute, For each value of the attribute, make a rule as follows: count how often each class appears find the most frequent class make the rule assign that class to this attribute-value Calculate the error rate of the rules Choose the rules with the smallest error rate ◼ Note: “missing” is always treated as a separate attribute value
Evaluating the weather attributesOutlookWindyTempHumidityPlayAttributeRulesErrorsTotalHotNoSunnyHighFalseerrorsSunnyHotHighTrueNo2/5Outlook4/14Sunny→NoHotHighFalseOvercastYes0/4Overcast→YesRainyMildHighFalseYes2/5Rainy→YesRainyCoolYesNormalFalse2/45/14TemperatureHot → No*RainyNoCoolNormalTrue2/6Mild → YesOvercastCoolNormalTrueYes1/4Cool→ YesMildNoSunnyHighFalse3/7Humidity4/14High → NoSunnyCoolNormalFalseYes1/7Normal→YesRainyMildNormalFalseYes2/8WindyFalse→Yes5/14SunnyMildNormalTrueYes3/6True -→ No*OvercastMildHighTrueYesHotNormalFalseYesOvercastMildNoRainyHighTrue52026/9/22UniversityofWaikato
2026/9/22 University of Waikato 5 Evaluating the weather attributes Attribute Rules Errors Total errors Outlook Sunny → No 2/5 4/14 Overcast → Yes 0/4 Rainy → Yes 2/5 Temperature Hot → No* 2/4 5/14 Mild → Yes 2/6 Cool → Yes 1/4 Humidity High → No 3/7 4/14 Normal → Yes 1/7 Windy False → Yes 2/8 5/14 True → No* 3/6 Outlook Temp. Humidity Windy Play Sunny Hot High False No Sunny Hot High True No Overcast Hot High False Yes Rainy Mild High False Yes Rainy Cool Normal False Yes Rainy Cool Normal True No Overcast Cool Normal True Yes Sunny Mild High False No Sunny Cool Normal False Yes Rainy Mild Normal False Yes Sunny Mild Normal True Yes Overcast Mild High True Yes Overcast Hot Normal False Yes Rainy Mild High True No