Distance from a point to decision boundary The unsigned distance from a point x to a hyperplane h is Iwx+b dux We can remove the absolute value by exploiting the fact that the decision boundary classifies every point in the training dataset correct Namely,(w'x+b) and x's label y must have the same sign,so y(+ b lwl‖l 1/27/2021 PATTERN RECOGNITION
Distance from a point to decision boundary The unsigned distance from a point 𝒙 to a hyperplane ℋ is We can remove the absolute value ⋅ by exploiting the fact that the decision boundary classifies every point in the training dataset correctly Namely, (𝒘𝑇𝒙 + 𝑏) and x’s label y must have the same sign, so: 1/27/2021 PATTERN RECOGNITION 16 𝑑ℋ 𝒙 = 𝒘𝑇𝒙 + 𝑏 𝒘 2 𝑑ℋ 𝒙 = 𝑦(𝒘𝑇𝒙 + 𝑏) 𝒘 2
Intuition: where to put the decision boundary? In the example below there are several separating hyperplanes Each of them is valid as it successfully separates our data set with men on one side and women on the other side o Women 十 Men 8 There can be a lot of separating hyperplanes 170 175 Size(cm) 1/27/2021 PATTERN RECOGNITION
Intuition: where to put the decision boundary? In the example below there are several separating hyperplanes. Each of them is valid as it successfully separates our data set with men on one side and women on the other side. 1/27/2021 PATTERN RECOGNITION 17 There can be a lot of separating hyperplanes
Intuition: where to put the decision boundary? Suppose we select the green hyperplane and use it to classify on real life data o Women t Men This hyperplane does not generalize well 160 175 185 Size(cm) 1/27/2021 PATTERN RECOGNITION
Intuition: where to put the decision boundary? Suppose we select the green hyperplane and use it to classify on real life data 1/27/2021 PATTERN RECOGNITION 18 This hyperplane does not generalize well
Intuition: where to put the decision boundary? So we will try to select an hyperplane as far as possible from data points trom each category: o Women t Men This one looks better 170 180 SIze(cm) 1/27/2021 PATTERN RECOGNITION
Intuition: where to put the decision boundary? So we will try to select an hyperplane as far as possible from data points from each category: 1/27/2021 PATTERN RECOGNITION 19 This one looks better
Intuition: where to put the decision boundary? When we use it with real life data, we can see it still make perfect classification o Women Men The black hyperplane classifies more accurately than the green one 155 170 175 Size(cm) 1/27/2021 PATTERN RECOGNITION
Intuition: where to put the decision boundary? When we use it with real life data, we can see it still make perfect classification. 1/27/2021 PATTERN RECOGNITION 20 The black hyperplane classifies more accurately than the green one