Maximum Margin f est f( w, b=sign w x-b) denotes +1 denotes -1 The maximum margin linear classifier is the linear classifier Support vectors With the, um are those datapoints that maximum margin the margin This is the pushes up against simplest kind of SVM(Called an SVM Linear sⅥM Copyright 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 11
Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 11 Maximum Margin f x a y est denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) The maximum margin linear classifier is the linear classifier with the, um, maximum margin. This is the simplest kind of SVM (Called an LSVM) Support Vectors are those datapoints that the margin pushes up against Linear SVM
Why maximum Margin? 1. Intuitively this feels safest denotes +1 2. Empirically it works very well. denotes -1 3. If we've made a small error in the location of the boundary (it's been jolted in its perpendicular direction) this gives us least chance of causing a misclassification Support vectors are those 4. LooCV is easy since the model is datapoints that immune to removal of any non the margin support-vector datapoints. pushes up 15. There's some theory(using VC against dimension that is related to(but not the same as the proposition that this is a good thing. Copyright 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 12
Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 12 Why Maximum Margin? denotes +1 denotes -1 f(x,w,b) = sign(w. x - b) The maximum margin linear classifier is the linear classifier with the, um, maximum margin. This is the simplest kind of SVM (Called an LSVM) Support Vectors are those datapoints that the margin pushes up against 1. Intuitively this feels safest. 2. Empirically it works very well. 3. If we’ve made a small error in the location of the boundary (it’s been jolted in its perpendicular direction) this gives us least chance of causing a misclassification. 4. LOOCV is easy since the model is immune to removal of any nonsupport-vector datapoints. 5. There’s some theory (using VC dimension) that is related to (but not the same as) the proposition that this is a good thing
Estimate the margin ° denotes+1 denotes-1 wx+b= 0 X X-Vector W-Normal vector b- Scale value What is the distance expression for a point x to a line wx+b= 0? X·W X·W+ Copyright 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 13
Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 13 Estimate the Margin • What is the distance expression for a point x to a line wx+b= 0? denotes +1 denotes -1 x wx +b = 0 2 2 2 1 ( ) d i i b b d w= + + = = x w x w x w X – Vector W – Normal Vector b – Scale Value W
Estimate the margin ° denotes+1 denotes -1 wx+b= 0 Margin What is the expression for margin? x·w+b margin arg mind(x=arg min X∈D X∈D d 2 Copyright 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 14
Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 14 Estimate the Margin • What is the expression for margin? denotes +1 denotes -1 wx +b = 0 2 1 margin arg min ( ) arg min d D D i i b d w= + = x x x w x Margin
Maximize Margin denotes +1 denotes-1 wx+b= 0 Margin「 argmax ma rgin(w,b, D) W argmax arg mind(X) . 6 ∈D 16+x, . w argmax arg min ∈D Copyright 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 15
Copyright © 2001, 2003, Andrew W. Moore Support Vector Machines: Slide 15 Maximize Margin denotes +1 denotes -1 wx +b = 0 , , 2 , 1 argmax margin( , , ) = argmax arg min ( ) argmax arg min i i b i b D i d b D i i b D d b w = + = w w x w x w x x w Margin