Big Data Analysis and Mining Evaluation other classifiers Qinpei Zhao赵钦佩 qinpeizhao@tongji.edu.cn 2015 Fall 2021/2/8
2021/2/8 1 Big Data Analysis and Mining Qinpei Zhao 赵钦佩 qinpeizhao@tongji.edu.cn 2015 Fall Evaluation & other classifiers
ROC curve Receiver Operating Characteristics(Roc) graphs have long been used in signal detection theory to depict the tradeoff between hit rates and false alarm rates over noisy channel a Recent years have seen an increase in the use of roc graphs in the machine learning community A useful technique for organizing classifiers and visualizing their performance a Especially useful for domains with skewed class distribution and unequal classification error cost
ROC curve ◼ Receiver Operating Characteristics (ROC) graphs have long been used in signal detection theory to depict the tradeoff between hit rates and false alarm rates over noisy channel ◼ Recent years have seen an increase in the use of ROC graphs in the machine learning community ◼ A useful technique for organizing classifiers and visualizing their performance ◼ Especially useful for domains with skewed class distribution and unequal classification error cost
True condition Total population Condition positive Condition negative ∑ Condition positive Positive predictive value(PPv). Predicted condition False positive Precision False discovery rate(FDR) positive 2 True posit posit Test outcome postive condition 2Test outcome positive Predicted condition False negative False omission rate(FOR) Negative predictive value(NPV) (ype II error) True negative 2False negative E True negat ∑ Test outcome negative 2 Test outcome negative True positive rate(IPR) False positive rate(FPR). Positive likelihood ratio (LR+ Sensitivity, Recall FAlse positive curacy (Acc= ∑ Condition negative Diagnostic odds ratio(DOR) 2 True positive+ ETrue negative otal population False negative rate(FNR), True negative rate(TNR), Negative likelihood ratio (LR- ∑ False negative ∑ True negative TNR ∑ Condition negati 2021/2/8
2021/2/8 3
ROC curve ROC curve is a plot of TPR(sensitivity) against FPR(specificity) which depicts relative trade-offs between benefits(true positives) and costs(false positives) number of true positives Sensitivit number of true positives+ number of false negatives number of true negatives Specificity= number of true negatives number of false positives
ROC curve ROC curve is a plot of TPR(sensitivity) against FPR(specificity) which depicts relative trade-offs between benefits (true positives) and costs (false positives)
Exampl e ROC space TP=63FP=2891 P=76‖FP=1288 rfect FN=37TN=72109 0.8 N=24 TN=88112 100 10020(07 100 100200 A Sensitivity 8 o5 better Specificity 0.3 02 c worse 0 0.2 0.4 0.6 0.8 FPR or(1-specificity)
Example