2.3.3.Geometric margin B Given training dataset Tand hyperplane(w,b),geometric margin of(w,b)w.r.t. the sample (i,yi)is defined as: =(+) Geometric margin is signed distance from sample to (w,b). When sample is correctly classified by (w,b),geometric margin becomes distance to (w,b). 25/263
2.3.3. Geometric margin Given training dataset T and hyperplane (w, b), geometric margin of (w, b) w.r.t. the sample (xi , yi) is defined as: γi = yi w kwk · xi + b kwk ▶ Geometric margin is signed distance from sample to (w, b). ▶ When sample is correctly classified by (w, b), geometric margin becomes distance to (w, b). 25 / 263
Geometric margin of(w,b)w.r.t.training dataset Tis: min Y i=1,,W Relationship between functional margin and geometric margin: Ifw=1,then functional margin is equal to geometric margin. If hyperplane parameters w,b change proportionally (the hyperplane does not change),functional margin also changes proportionally,and geometric margin does not change. 26/263
▶ Geometric margin of (w, b) w.r.t. training dataset T is: γ = min i=1,...,N γi ▶ Relationship between functional margin and geometric margin: γi = γˆi kwk γ = γˆ kwk ▶ If kwk = 1, then functional margin is equal to geometric margin. ▶ If hyperplane parameters w, b change proportionally (the hyperplane does not change), functional margin also changes proportionally, and geometric margin does not change. 26 / 263
Outline (Level 1-2) SVM Linear SVM in LSC o Convex optimization o Linear separability and classifier o Linear SVM in LSC Maximum Margin Types of SVM SV and margin boundary Three Types of Margin Lagrange Duality Modeling of SVM Optimization o Dual method of linear SVM in o Primal optimization problem for LSC 27/263
Outline (Level 1-2) 2 Linear SVM in LSC Linear separability and classifier Types of SVM Three Types of Margin Modeling of SVM Optimization Primal optimization problem for SVM Convex optimization Linear SVM in LSC - Maximum Margin SV and margin boundary Lagrange Duality Dual method of linear SVM in LSC 27 / 263
Outline (Level 2-3) Modeling of SVM Optimization o Geometric Margin Maximization Modeling o Distance Maximization Modelling 28/263
Outline (Level 2-3) Modeling of SVM Optimization Geometric Margin Maximization Modeling Distance Maximization Modelling 28 / 263
2.4.Modeling of SVM Optimization 2.4.1.Geometric Margin Maximization Modeling For linearly separable training datasets,there are infinitely number of linear separation hyperplanes(perceptrons). Separation hyperplane with the largest geometric margin is unique,which is also called hard margin maximization(vs soft margin maximization when training dataset is approximately linearly separable). Intuition of maximum margin:hyperplane with largest geometric margin for training datasets means classifying training data with sufficient confidence. That is to say,not only positive and negative samples are separated,but also the most difficult to separate samples(samples closest to hyperplane)are also sufficiently confident to be separated.Such a hyperplane should have good classification and prediction capabilities for unknown new samples. 29/263
2.4. Modeling of SVM Optimization 2.4.1. Geometric Margin Maximization Modeling ▶ For linearly separable training datasets, there are infinitely number of linear separation hyperplanes (perceptrons). ▶ Separation hyperplane with the largest geometric margin is unique, which is also called hard margin maximization (vs soft margin maximization when training dataset is approximately linearly separable). ▶ Intuition of maximum margin: hyperplane with largest geometric margin for training datasets means classifying training data with sufficient confidence. That is to say, not only positive and negative samples are separated, but also the most difficult to separate samples (samples closest to hyperplane) are also sufficiently confident to be separated. Such a hyperplane should have good classification and prediction capabilities for unknown new samples. 29 / 263