How Far We Have Progressed in the Journey? An Examination of Cross-Project Defect Prediction

How Far We Have Progressed in the Journey?An Examination of CPDP 1:11 暖 销 用 E 9 wurTco 5+3 围 员 围 是 1 是园 ACM Transactions on Software Engineering and Methodology,Vol.27.No.1.Article 1.Pub.date:April 2018

How Far We Have Progressed in the Journey? An Examination of CPDP 1:11 Table 2. Literature Overview of Cross-Project Defect Prediction Project characteristics Key modeling components (challenges) covered Performance evaluation context Study Year Topic #Source projects (releases) #Target projects (releases) Languages in source and target projects Privatize data Homogenize features Filter instances Balance classes Transform distributions Select features Target training data Application scenario Main performance indicators Test data avail? Against SSM? Select for comp.? Zhang et al. [122] 2017 Distribution transforma- tion 18 18 Java Yes Yes Yes Classification F1, AUC All Yes Stuckman et al. [101] 2017 Vulnerability prediction 6 3 Php Yes Yes Yes Yes Classification F1 All Yes Aarti et al. [1] 2017 Utility analysis in prediction 17(35) 17(35) C/C++/Java Yes Yes Yes Classification Precision, recall Partial Yes Herbold et al. [37] 2017 Local prediction models 48(79) 48(79) C/C++/Java Yes Yes Yes Classification F1, AUC All Yes Ryu et al. [95] 2017 Combination of TL and CIL 15 15 Java Yes Yes Yes Classification G3, Balance Not No Jing et al. [42] 2017 Class imbalance learning 16 16 C/C++/Java Yes Yes Yes Classification F2, AUC All Yes Krishna et al. [52] 2016 Bellwether effect 23(51) 23(51) C/C++/Java Yes Yes Classification ED All Yes Xia et al. [116] 2016 Compositional model 10(29) 10(29) Java Yes Yes Both F1, PofB20 All Yes Zhang et al. [123] 2016 Universal model 1385 5 C/Pascal/C++/ Java/C# Yes Classification AUC All Yes Kamei et al. [44] 2016 Just-in-time prediction 11 11 Java/JS/C/C++/ Perl/Ruby Yes Yes Yes Yes Classification AUC Partial No Wang et al. [109] 2016 Learning semantic features 10(26) 10(11) Java Yes Yes Classification F1 All Yes Ryu et al. [93] 2016 Class imbalance learning 10 10 C/C++/Java Yes Yes Classification AUC Partial Yes (Continued) ACM Transactions on Software Engineering and Methodology, Vol. 27, No. 1, Article 1. Pub. date: April 2018

1:12 Y.Zhou et al. 自 麦 居菲 至 用 g 兰 兰 n 美 令 多 且 围 多 ACM Transactions on Software Engineering and Methodology,Vol.27,No.1,Article 1.Pub.date:April 2018

1:12 Y. Zhou et al. Table 2. Continued Project characteristics Key modeling components (challenges) covered Performance evaluation context Study Year Topic #Source projects (releases) #Target projects (releases) Languages in source and target projects Privatize data Homogenize features Filter instances Balance classes Transform distributions Select features Target training data Application scenario Main performance indicators Test data avail? Against SSM? Select for comp.? Ryu et al. [92] 2016 Multi￾objective optimization 13 13 Java Yes Yes Classification Balance, AUC All Yes Hosseini et al. [39] 2016 Training data selection 13 13 Java Yes Classification F1, G2 All Yes Moshtari et al. [71] 2016 Vulnerability predictors 5 5 C/C++/Java Classification AUC, F2 Not No You et al. [118] 2016 Rank-oriented model 15(39) 15(39) Java Yes Yes Ranking Prec@10 All Yes Cheng et al. [17] 2016 Heterogeneous prediction 14 14 C/Java Yes Yes Yes Classification F1, AUC All Yes Yu et al. [121] 2016 Transfer defect learning 15 15 Java Yes Yes Yes Classification G1 Not No Kaur et al. [46] 2016 Text-based prediction N/A N/A N/A N/A N/A N/A N/A N/A N/A N/A Classification N/A N/A No Kaur et al. [45] 2016 Value of academic projects 17 4(16) Java Yes Classification AUC All Yes Catal et al. [14] 2016 Threshold￾based prediction 6 6 C/C++/Java Yes Classification AUC All Yes Yu et al. [120] 2016 Filter vs. feature selection 6 6 C/Java Yes Yes Classification AUC All No Porto et al. [87] 2016 Filter with feature selection 11(36) 11(36) Java Yes Yes Both G1, FPA Not No He et al. [29] 2016 New relevancy filter method 15 15 Java Yes Yes Classification AUC All Yes (Continued) ACM Transactions on Software Engineering and Methodology, Vol. 27, No. 1, Article 1. Pub. date: April 2018

How Far We Have Progressed in the Journey?An Examination of CPDP 1:13 影 是销 号 尊 ONo 尊 兰 型 且 昌 且 围 (FF)PI 翡 1 11 ACM Transactions on Software Engineering and Methodology,Vol.27.No.1.Article 1.Pub.date:April 2018

How Far We Have Progressed in the Journey? An Examination of CPDP 1:13 Table 2. Continued Project characteristics Key modeling components (challenges) covered Performance evaluation context Study Year Topic #Source projects (releases) #Target projects (releases) Languages in source and target projects Privatize data Homogenize features Filter instances Balance classes Transform distributions Select features Target training data Application scenario Main performance indicators Test data avail? Against SSM? Select for comp.? Ryu et al. [94] 2015 Hybrid instance selection 7 7 C/C++/Java Yes Yes Yes Classification Balance Partial Yes Kawata et al. [47] 2015 New relevancy filter method 12(35) 18(21) Java Yes Classification F1, G1, AUC All Yes He et al. [26] 2015 Simplified metric set 10(34) 10(34) Java Yes Yes Classification F1 All Yes Chen et al. [15] 2015 Negative samples reduction 15 15 Java Yes Yes Yes Yes Classification G1, MCC Not No Canfora et al. [12] 2015 Multi-objective optimization 10 10 Java Yes Ranking AUCEC All Yes Yes Peters et al. [85] 2015 Privacy￾preserving data sharing 7 10 Java Yes Yes Yes Classification G1 All Yes Jing et al. [41] 2015 Heterogeneous prediction 14 14 C/Java Yes Yes Yes Classification F1 All Yes Nam et al. [79] 2015 Heterogeneous prediction 28 28 C/Java Yes Yes Classification AUC All Yes Qin et al. [89] 2015 Aging related defect prediction 7 7 C/C++ Yes Yes Classification Balance Not No Amasaki et al. [5] 2015 Data simplification 14(44) 14(44) Java Yes Yes Yes Classification F1, AUC All Yes Singh et al. [99] 2015 Prediction at design phase 7 7 C/C++/Java Yes Classification AUC All Yes Zhang et al. [126] 2015 Classifier ensemble 10 10 Java Yes Both NofB20, F1 All Yes (Continued) ACM Transactions on Software Engineering and Methodology, Vol. 27, No. 1, Article 1. Pub. date: April 2018

1:14 Y.Zhou et al. 司 即 兰 1 美1 勇 e 每 01 N ACM Transactions on Software Engineering and Methodology,Vol.27,No.1,Article 1.Pub.date:April 2018

1:14 Y. Zhou et al. Table 2. Continued Project characteristics Key modeling components (challenges) covered Performance evaluation context Study Year Topic #Source projects (releases) #Target projects (releases) Languages in source and target projects Privatize data Homogenize features Filter instances Balance classes Transform distributions Select features Target training data Application scenario Main performance indicators Test data avail? Against SSM? Select for comp.? Satin et al. [96] 2015 Impact of classifier 1270 1270 C/Pascal/C++/ Java/C# Yes Yes Classification AUC Not No Cao et al. [13] 2015 Transfer defect learning 8 8 C/Java Yes Yes Yes Classifcation F1 All Yes Altinger et al. [3] 2015 Adoption in automotive industry 2 2 C Yes Yes Yes Classifcation F1 Not No He et al. [30] 2015 Transfer defect learning 11 11 Java Yes Yes Yes Yes Classifcation F1 Not No Panichella et al. [82] 2014 Classifier ensemble 10 10 Java Yes Both AUC, AUCEC All Yes Mizuno et al. [66] 2014 Text-based prediction 8(28) 8(28) Java Yes Classification F1 All Yes He et al. [27] 2014 Imbalanced feature sets 11 11 Java Yes Yes Yes Classification F1 All Yes He et al. [28] 2014 Training data simplification 10(34) 10(34) Java Yes Yes Classification F1, G1 All Yes Ma et al. [60] 2014 Associative classification 9 9 C/C++/Java Yes Yes Yes Classification AUC Not No Peters et al. [84] 2013 Balancing privacy and utility 10 10 Java Yes Yes Yes Classification G1 All Yes Peters et al. [86] 2013 Training data selection 12(35) 18(21) Java Yes Classification G1 All Yes Turhan et al. [107] 2013 Mixed project data 41(73) 41(73) C/C++/Java Yes Yes Classification Balance All Yes Singh et al. [100] 2013 Utility of OO metrics 2 2 C++/Java Classification F1 Not No Moshtari et al. [72] 2013 Vulnerability predictors 5 5 C/C++/Java Classification F2, AUC Not No (Continued) ACM Transactions on Software Engineering and Methodology, Vol. 27, No. 1, Article 1. Pub. date: April 2018

How Far We Have Progressed in the Journey?An Examination of CPDP 1:15 9 9 暖 门 量 巢 莫 是 1 朝 ( l 到 ACM Transactions on Software Engineering and Methodology.Vol.27.No.1.Article 1.Pub.date:April 2018

How Far We Have Progressed in the Journey? An Examination of CPDP 1:15 Table 2. Continued Project characteristics Key modeling components (challenges) covered Performance evaluation context Study Year Topic #Source projects (releases) #Target projects (releases) Languages in source and target projects Privatize data Homogenize features Filter instances Balance classes Transform distributions Select features Target training data Application scenario Main performance indicators Test data avail? Against SSM? Select for comp.? Nam et al. [80] 2013 Transfer defect learning 8 8 Java Yes Yes Classification F1 All Yes He et al. [31] 2013 Cross proprietary projects 10(34) 10(34) Java Yes Yes Yes Yes Classification G1 All No Ma et al. [61] 2012 Transfer defect learning 7 3 C/C++/Java Yes Yes Classification F1, AUC, All Yes Rahman et al. [90] 2012 Effort-aware evaluation 9(38) 9(38) Java Yes Both F1, AUC, AUCEC Not No Uchigaki et al. [108] 2012 Classifier ensemble 12 12 C/C++/Java Yes Yes Classification AUC Partial Yes He et al. [32] 2012 Feasibility of cross projects 10(34) 10(34) Java Yes Classification F1 All Yes Babic [7] 2012 Adaptive defect prediction 8 8 Java Yes Yes Yes Yes Classification F1 Not No Yu et al. [119] 2012 Utility of complexity metrics 5 4 C Yes Classification Recall, Precision Yes Yes Rremraj et al. [88] 2011 Utility of network metrics 3 3 Java Yes Yes Yes Classification F1 Not No Liu et al. [59] 2010 Classifier ensemble 7 7 C/C++/Java Yes Classification NECM Partial Yes Jureczko et al. [43] 2010 Training data selection 38(92) 38(92) Java Yes Yes Ranking E1(80) All No Turhan et al. [104] 2010 Regularities in defect prediction 13 13 C/C++/Java Yes Yes Yes Classification Balance Partial No Turhan et al. [106] 2009 Applicability of cross companies 10 10 C/C++/Java Yes Classification Balance Not No Zimmer. et al. [129] 2009 Applicability of cross projects 12(28) 12(28) C/C++/C#/Java Yes Classification Recall, Precision Not No (Continued) ACM Transactions on Software Engineering and Methodology, Vol. 27, No. 1, Article 1. Pub. date: April 2018