Ensemble LearningNOV.2016
Ensemble Learning NO V. 2 0 1 6
Ensemble learningindividual learner1individual learner2combinationoutputmoduleindividual learnerTweakclassifiersstrong classifierHomogeneous。BaselearnersHeterogenousComponentlearners9/22/2026PATTERNRECOGNITION2
Ensemble learning 9/22/2026 PATTERN RECOGNITION 2 individual learner 1 individual learner 2 individual learner T . combination module output weak classifiers strong classifier Homogeneous ◦ Base learners Heterogenous ◦ Component learners
Ensemble learningHowtoachievehigherperformanceusing weaklearners?TestTestTestTestTestTestTestTestTest313213212VVh1VVVh1Xh1xxXvVh2h2h2vvvxxxXVh3Vh3VVh3VXXXXenseenseenseVVvVVXXXXmblemblembleoRelativelyhighaccuracy。Diversity9/22/2026PATTERNRECOGNITION3
Ensemble learning 9/22/2026 PATTERN RECOGNITION 3 How to achieve higher performance using weak learners? ◦ Relatively high accuracy ◦ Diversity Test 1 Test 2 Test 3 h1 √ √ x h2 x √ √ h3 √ x √ ense mble √ √ √ Test 1 Test 2 Test 3 h1 √ √ x h2 √ √ x h3 √ √ x ense mble √ √ x Test 1 Test 2 Test 3 h1 √ x x h2 x √ x h3 x x √ ense mble x x x
Ensemble learningEnsembleclassifiers。BoostingBaggingandRandom Forest9/22/2026PATTERNRECOGNITION
Ensemble learning Ensemble classifiers ◦ Boosting ◦ Bagging and Random Forest 9/22/2026 PATTERN RECOGNITION 4
BoostingHigh-level idea:combinealotof classifiersoSequentiallyconstruct/identifytheseclassifiersoneatatimeoUseweak classifiers toarrive at complexdecision boundaries (strongclassifiers)OurplanoDescribeAdaBoostalgorithmoDerivethealgorithm9/22/2026PATTERNRECOGNITION5
Boosting High-level idea: combine a lot of classifiers ◦ Sequentially construct / identify these classifiers one at a time ◦ Use weak classifiers to arrive at complex decision boundaries (strong classifiers) Our plan ◦ Describe AdaBoost algorithm ◦ Derive the algorithm 9/22/2026 PATTERN RECOGNITION 5