Data -Target 一Fit c E 'good fit' overfit out-of-sample error Strategy model complexity √Increase Sample √Remove Outliers in-sample error Decrease model complexity VC dimension,dve Train-Validation-Test Regularization Source:http://jasonding1354.github.io/
Strategy Increase Sample Remove Outliers Decrease model complexity Train-Validation-Test Regularization Source：http://jasonding1354.github.io/
Model Selection and Regularization Structural risk -Empirical risk regularization （出，rCx》+( i1 .(F)measures model complexity,aiming at selecting a model that can fit the current data as simple as possible .A is the trade-off between model fitness and model complexity
Structural risk – Empirical risk + regularization • 𝜙 𝐹 measures model complexity, aiming at selecting a model that can fit the current data as simple as possible • 𝜆 is the trade-off between model fitness and model complexity Model Selection and Regularization 1 N 𝑖=1 𝑁 𝐿 𝑌𝑖 , 𝐹 𝑋𝑖 + 𝝀𝝓 𝑭
Choice ofΦ(F) L2 norm: ()F)a LI norm: L0)-F)+会B, Lp norm,nuclear norm
L2 norm: L1 norm: Lp norm, nuclear norm… Choice of 𝝓 𝑭 𝐿 𝛽 = 1 N 𝑖=1 𝑁 𝐿 𝑌𝑖 , 𝐹 𝑋𝑖 |𝛽 + 𝝀 𝟐 𝜷 𝟐 𝐿 𝛽 = 1 N 𝑖=1 𝑁 𝐿 𝑌𝑖 , 𝐹 𝑋𝑖 |𝛽 + 𝝀 𝟐 𝜷 𝟏
Classification vs.Prediction ■Classification predicts categorical class labels(discrete or nominal) classifies data(constructs a model)based on the training set and the values (class labels)in a classifying attribute and uses it in classifying new data ■Prediction models continuous-valued functions,i.e.,predicts unknown or missing values Typical applications √Credit approval √Target marketing √Medical diagnosis √fraud detection
Classification predicts categorical class labels (discrete or nominal) classifies data (constructs a model) based on the training set and the values (class labels) in a classifying attribute and uses it in classifying new data Prediction models continuous-valued functions, i.e., predicts unknown or missing values Typical applications Credit approval Target marketing Medical diagnosis Fraud detection Classification vs. Prediction
Classification
Classification