《网络与系统安全》教学参考文献：An adaptive trust model based on recommendation filtering algorithm for the Internet of Things systems

Computer Networks 190(2021)107952 Contents lists available at ScienceDirect Computer Networks ELSEVIER journal homepage:www.elsevier.com/locate/comnet An adaptive trust model based on recommendation filtering algorithm for the Internet of Things systems Guozhu Chen",Fanping Zeng2,b,",Jian Zhangd,Tingting Lu",Jingfei Shen",Wenjuan Shua .School of Computer Science and Technology,University of Science and Technology of China,Hefei,Anhui,China Anhui Province Key Lab of Software in Computing and Communication,Hefei,Anhui,China State Key Laboratory of Computer Science,Institute of Software Chinese Academy of Sciences,Beijing,China University of Chinese Academy of Sciences,Beijing,China ARTICLE INFO ABSTRACT Keywords: The Internet of Things (loT)is growing rapidly and brings great convenience to humans.But it also causes Internet of Things some security issues which may have negative impacts on humans.Trust management is an effective method Trust model to solve these problems by establishing trust relationships among interconnected IoT objects.In this paper, we propose an adaptive trust model based on recommendation filtering algorithm for the IoT systems.The utilization of sliding window and time decay function when calculating direct trust can greatly accelerate the convergence rate of trust evaluation. We design a recommendation filtering algorithm to effectively filter out bad recommendations and minimize the impact of malicious objects.An adaptive weight is developed to better combine direct trust and recommendation trust into synthesis trust so as to adapt to the dynamically hostile environment.In the simulation experiments,we compare our adaptive trust model with three related models:TBSM,NRB and NTM. The experimental results indicate that our trust model converges fast and the mean absolute error is always less than 0.05 when the proportion of malicious nodes is from 10%to 70%.The comparative experiments further verify the effectiveness of our trust model in terms of accuracy,convergence rate and resistance to trust related attacks. 1.Introduction objects which have different functions and provide diverse services and applications.Consequently,an IoT trust model should be universal and The concept of Internet of Things (loT)is to connect a large number capable of running on various types of objects.Second,most objects of objects in the real physical world to the Internet based on standard have limited capacities so that the existing trust models in p2P and communication protocols and unique addressing schemes [1].These social networks are no longer applicable.Third,many of the objects will interconnected objects can be service providers offering services and sharing resources and information with each other.For the past few be malicious for their own benefits and then carry out various malicious years,IoT has grown rapidly and a series of relevant services and ap- attacks in order to reduce the trust value of others or improve their plications including smart home,smart city and smart community [2] own trustworthiness.As a result,IoT trust models should be resistant emerged.These services and applications bring great convenience to to those malicious attacks. humans,but they also cause some security issues that may do harm to To meet the challenges discussed above,we propose an adaptive our lives.For example,a misbehaved object can perform various types trust model to establish trust relationships among objects.Our trust of malicious attacks to destroy the integrity and availability of data model based on the recommendation filtering algorithm can effectively and network resources.Trust management is an effective method to resist malicious attacks carried out by misbehaved objects and evaluate solve the above security issues by establishing trust relationships among the trust value of target objects accurately.The major contributions of objects and then excluding malicious objects.It allows multiple objects to share their opinions about the trust value of their companions [3]. our paper are as follows: Although trust management can effectively solve some of the secu- rity problems,there are still some challenges in building trust man. We propose a system architecture based on trust third parties agement systems.First,there are a large number of heterogeneous (TTPs)which provides a secure and reliable trust computing Corresponding author at School of Computer Science and Technology,University of Science and Technology of China,Hefei,Anhui,China. E-mail addresses:chengz18@mail.ustc.edu.cn (G.Chen),billzeng@ustc.edu.cn (F.Zeng). https://doi.org/10.1016/j.comnet.2021.107952 Received 7 September 2020;Received in revised form 27 January 2021;Accepted 17 February 2021 Available online 22 February 2021 1389-1286/@2021 Elsevier B.V.All rights reserved

Computer Networks 190 (2021) 107952 Available online 22 February 2021 1389-1286/© 2021 Elsevier B.V. All rights reserved. Contents lists available at ScienceDirect Computer Networks journal homepage: www.elsevier.com/locate/comnet An adaptive trust model based on recommendation filtering algorithm for the Internet of Things systems Guozhu Chen a , Fanping Zeng a,b,∗ , Jian Zhang c,d , Tingting Lu a , Jingfei Shen a , Wenjuan Shu a a School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, China b Anhui Province Key Lab of Software in Computing and Communication, Hefei, Anhui, China c State Key Laboratory of Computer Science, Institute of Software Chinese Academy of Sciences, Beijing, China d University of Chinese Academy of Sciences, Beijing, China A R T I C L E I N F O Keywords: Internet of Things Trust model A B S T R A C T The Internet of Things (IoT) is growing rapidly and brings great convenience to humans. But it also causes some security issues which may have negative impacts on humans. Trust management is an effective method to solve these problems by establishing trust relationships among interconnected IoT objects. In this paper, we propose an adaptive trust model based on recommendation filtering algorithm for the IoT systems. The utilization of sliding window and time decay function when calculating direct trust can greatly accelerate the convergence rate of trust evaluation. We design a recommendation filtering algorithm to effectively filter out bad recommendations and minimize the impact of malicious objects. An adaptive weight is developed to better combine direct trust and recommendation trust into synthesis trust so as to adapt to the dynamically hostile environment. In the simulation experiments, we compare our adaptive trust model with three related models: TBSM, NRB and NTM. The experimental results indicate that our trust model converges fast and the mean absolute error is always less than 0.05 when the proportion of malicious nodes is from 10% to 70%. The comparative experiments further verify the effectiveness of our trust model in terms of accuracy, convergence rate and resistance to trust related attacks. 1. Introduction The concept of Internet of Things (IoT) is to connect a large number of objects in the real physical world to the Internet based on standard communication protocols and unique addressing schemes [1]. These interconnected objects can be service providers offering services and sharing resources and information with each other. For the past few years, IoT has grown rapidly and a series of relevant services and ap￾plications including smart home, smart city and smart community [2] emerged. These services and applications bring great convenience to humans, but they also cause some security issues that may do harm to our lives. For example, a misbehaved object can perform various types of malicious attacks to destroy the integrity and availability of data and network resources. Trust management is an effective method to solve the above security issues by establishing trust relationships among objects and then excluding malicious objects. It allows multiple objects to share their opinions about the trust value of their companions [3]. Although trust management can effectively solve some of the secu￾rity problems, there are still some challenges in building trust man￾agement systems. First, there are a large number of heterogeneous ∗ Corresponding author at: School of Computer Science and Technology, University of Science and Technology of China, Hefei, Anhui, China. E-mail addresses: chengz18@mail.ustc.edu.cn (G. Chen), billzeng@ustc.edu.cn (F. Zeng). objects which have different functions and provide diverse services and applications. Consequently, an IoT trust model should be universal and capable of running on various types of objects. Second, most objects have limited capacities so that the existing trust models in P2P and social networks are no longer applicable. Third, many of the objects will be malicious for their own benefits and then carry out various malicious attacks in order to reduce the trust value of others or improve their own trustworthiness. As a result, IoT trust models should be resistant to those malicious attacks. To meet the challenges discussed above, we propose an adaptive trust model to establish trust relationships among objects. Our trust model based on the recommendation filtering algorithm can effectively resist malicious attacks carried out by misbehaved objects and evaluate the trust value of target objects accurately. The major contributions of our paper are as follows: • We propose a system architecture based on trust third parties (TTPs) which provides a secure and reliable trust computing https://doi.org/10.1016/j.comnet.2021.107952 Received 7 September 2020; Received in revised form 27 January 2021; Accepted 17 February 2021

G.Chen et al. Computer Networks 190(2021)107952 environment and hence saves storage and computing resources evaluated by the trustor.If the trustor is satisfied with the service of IoT objects.Although in some previous work [4-7]and [8], provided by the trustee,it will give the trustee a high trust rating. the authors proposed hybrid architectures which are similar to However,the trustor cannot interact with all trustees directly all the ours,they did not specify what components are included in their time.In this situation,the trustor needs recommendations from other proposed architecture and they did not explain how to apply their objects that have interaction histories with trustees.Those objects who trust models to the architectures they proposed.Instead,we clar- give recommendations to the trustor are called recommenders. ify the components included in our architecture and the functions According to the above descriptions,we know that there are two of these components.Meanwhile,we explain the process of trust types of trust relationships including direct trust and recommendation evaluation and the interaction process of these components in the trust between a trustor and a trustee.The type of a given trust relation- architecture we proposed. ship depends on the way the trustor communicates with the trustee.If Considering that the impact of past feedback will decrease over the trustor communicates with the trustee directly,this trust relation- time,we introduce a sliding window to store feedback and use ship is considered direct trust.Otherwise,we call the trust relationship a time decay function to reduce the weight of the previous recommendation trust.In our trust model,the trustor evaluates the feedback.The differences from [5]and [6]are that we not only trustee's trust value by synthesis trust that combines direct trust and use the decay function to reduce the impact of previous feedback, recommendation trust by adaptive weight. we also propose a sliding window to save the feedback of the most recent period of time.The use of the sliding window can reflect 2.2.Attack model the changes in the trust value of the IoT objects more quickly because of the fact that recent behaviors can better reflect the A malicious object is dishonest and can perform malicious attacks current trust status of IoT objects. such as providing bad service or recommending adverse trust informa- We design a recommendation filtering algorithm based on k- tion about trustees to the trustor.We call these attacks trust related means to filter out bad recommendations provided by malicious attacks.The trust related attacks are summarized as follows: recommenders.Although a similar filtering algorithm was pro- On-off attacks:A malicious object behaves well for a period of posed in [5],we also introduce three important factors on the time and badly at other times.For example,a trustee can provide basis of our filtering algorithm.Even if the filtering algorithm a trustor with good service that does not need many resources and cannot completely filter out the bad recommendations,the use prefers not to serve the trustor when the trustor needs too many of these three important factors can reduce the negative impact resources. of the bad recommendations on the calculation of the recommen- Self promoting attacks:A malicious object can promote its dation trust as much as possible. reputation by offering good recommendations about itself so We introduce an adaptive weight that can adjust automatically ac- that it can be selected as a service provider and then provides cording to the dynamic environment to combine direct trust and poor service.A service requester can hardly select good service recommendation trust.The experimental results indicate that our providers under these attacks if the trust model does not ignore adaptive trust model enables fast and accurate trust evaluation bad recommendations about the malicious object itself. and resists malicious attacks in the dynamically hostile environ- Bad mouthing attacks:A malicious recommender can slander ment.Compared with the fixed weight used in [9]and [10],our the reputation of a well-behaved trustee by providing the trustor adaptive weight enables fast and accurate trust evaluation and with bad recommendations about that trustee.As a result,the resists malicious attacks in the dynamically hostile environment. trustee that is evaluated by the trustor with a low trust rating cannot be selected as a service provider. The remainder of this paper is organized as follows.In Section 2,we introduce the concept of trust and attack model in loT.In Section 3,we Ballot stuffing attacks:These attacks are similar to bad survey the related work of IoT trust models.In Section 4,we propose mouthing attacks.A badly-behaved trustee that cannot offer the system architecture and the process of trust evaluation.In Section 5, satisfying service will be highly rated by malicious recommenders we elaborate on our adaptive trust model and we give the experimental that give opposite recommendations to the trustor.When multi- results and relevant analysis in Section 6.Finally,we summarize the ple recommenders collaborate with each other to perform these paper and outline the future work in Section 7. attacks at the same time,they can boost the reputation of a bad trustee quickly. 2.Background Selective misbehavior attacks:A malicious recommender pro vides the trustor with bad recommendations about some trustees In this section,we first introduce the concept of trust in IoT,the and gives correct recommendations about others.In such a case, the trustor can hardly judge if the recommender is malicious main participants in the trust model and the types of trust.Then,we because of its intermittent malicious behavior. list some trust related attacks that can break the trust management system.Finally,we introduce some common outlier detection methods From the above description of trust related attacks,we know that which can be used to detect bad recommendations caused by those trust models are under many security threats that can break the func- trust related attacks and filter out them from all the recommendations tionality of trust management systems.Therefore,trust models should received by the trustor. consider multiple trust factors in order to evaluate trustees accurately. They should also take more defensive measures to avoid the negative 2.1.Trust in internet of things effect of bad recommendations so as to improve the stability of trust evaluation in the dynamically hostile environment. In human society,trust usually indicates the degree of subjective belief between people.People are more likely to communicate with 2.3.Outlier detection methods people they trust.Similarly,IoT objects are more willing to use services provided by trusted objects.Objects can evaluate the trust value of In Section 2.2,we have already introduced that malicious rec- others through trust models before using their service. ommenders which perform some trust related attacks such as bad There are three main participants in a trust model:trustor,trustee mouthing attacks and ballot stuffing attacks will provide bad recom- and recommender.A trustor is an object who wants to evaluate the mendations to the trustor.If the trustor uses these bad recommen- trust value of others.Correspondingly,a trustee is an object who is dations,the accuracy of the recommendation trust evaluation will be 2

Computer Networks 190 (2021) 107952 2 G. Chen et al. environment and hence saves storage and computing resources of IoT objects. Although in some previous work [4–7] and [8], the authors proposed hybrid architectures which are similar to ours, they did not specify what components are included in their proposed architecture and they did not explain how to apply their trust models to the architectures they proposed. Instead, we clar￾ify the components included in our architecture and the functions of these components. Meanwhile, we explain the process of trust evaluation and the interaction process of these components in the architecture we proposed. • Considering that the impact of past feedback will decrease over time, we introduce a sliding window to store feedback and use a time decay function to reduce the weight of the previous feedback. The differences from [5] and [6] are that we not only use the decay function to reduce the impact of previous feedback, we also propose a sliding window to save the feedback of the most recent period of time. The use of the sliding window can reflect the changes in the trust value of the IoT objects more quickly because of the fact that recent behaviors can better reflect the current trust status of IoT objects. • We design a recommendation filtering algorithm based on 𝑘- means to filter out bad recommendations provided by malicious recommenders. Although a similar filtering algorithm was pro￾posed in [5], we also introduce three important factors on the basis of our filtering algorithm. Even if the filtering algorithm cannot completely filter out the bad recommendations, the use of these three important factors can reduce the negative impact of the bad recommendations on the calculation of the recommen￾dation trust as much as possible. • We introduce an adaptive weight that can adjust automatically ac￾cording to the dynamic environment to combine direct trust and recommendation trust. The experimental results indicate that our adaptive trust model enables fast and accurate trust evaluation and resists malicious attacks in the dynamically hostile environ￾ment. Compared with the fixed weight used in [9] and [10], our adaptive weight enables fast and accurate trust evaluation and resists malicious attacks in the dynamically hostile environment. The remainder of this paper is organized as follows. In Section 2, we introduce the concept of trust and attack model in IoT. In Section 3, we survey the related work of IoT trust models. In Section 4, we propose the system architecture and the process of trust evaluation. In Section 5, we elaborate on our adaptive trust model and we give the experimental results and relevant analysis in Section 6. Finally, we summarize the paper and outline the future work in Section 7. 2. Background In this section, we first introduce the concept of trust in IoT, the main participants in the trust model and the types of trust. Then, we list some trust related attacks that can break the trust management system. Finally, we introduce some common outlier detection methods which can be used to detect bad recommendations caused by those trust related attacks and filter out them from all the recommendations received by the trustor. 2.1. Trust in internet of things In human society, trust usually indicates the degree of subjective belief between people. People are more likely to communicate with people they trust. Similarly, IoT objects are more willing to use services provided by trusted objects. Objects can evaluate the trust value of others through trust models before using their service. There are three main participants in a trust model: trustor, trustee and recommender. A trustor is an object who wants to evaluate the trust value of others. Correspondingly, a trustee is an object who is evaluated by the trustor. If the trustor is satisfied with the service provided by the trustee, it will give the trustee a high trust rating. However, the trustor cannot interact with all trustees directly all the time. In this situation, the trustor needs recommendations from other objects that have interaction histories with trustees. Those objects who give recommendations to the trustor are called recommenders. According to the above descriptions, we know that there are two types of trust relationships including direct trust and recommendation trust between a trustor and a trustee. The type of a given trust relation￾ship depends on the way the trustor communicates with the trustee. If the trustor communicates with the trustee directly, this trust relation￾ship is considered direct trust. Otherwise, we call the trust relationship recommendation trust. In our trust model, the trustor evaluates the trustee’s trust value by synthesis trust that combines direct trust and recommendation trust by adaptive weight. 2.2. Attack model A malicious object is dishonest and can perform malicious attacks such as providing bad service or recommending adverse trust informa￾tion about trustees to the trustor. We call these attacks trust related attacks. The trust related attacks are summarized as follows: • On–off attacks: A malicious object behaves well for a period of time and badly at other times. For example, a trustee can provide a trustor with good service that does not need many resources and prefers not to serve the trustor when the trustor needs too many resources. • Self promoting attacks: A malicious object can promote its reputation by offering good recommendations about itself so that it can be selected as a service provider and then provides poor service. A service requester can hardly select good service providers under these attacks if the trust model does not ignore bad recommendations about the malicious object itself. • Bad mouthing attacks: A malicious recommender can slander the reputation of a well-behaved trustee by providing the trustor with bad recommendations about that trustee. As a result, the trustee that is evaluated by the trustor with a low trust rating cannot be selected as a service provider. • Ballot stuffing attacks: These attacks are similar to bad mouthing attacks. A badly-behaved trustee that cannot offer satisfying service will be highly rated by malicious recommenders that give opposite recommendations to the trustor. When multi￾ple recommenders collaborate with each other to perform these attacks at the same time, they can boost the reputation of a bad trustee quickly. • Selective misbehavior attacks: A malicious recommender pro￾vides the trustor with bad recommendations about some trustees and gives correct recommendations about others. In such a case, the trustor can hardly judge if the recommender is malicious because of its intermittent malicious behavior. From the above description of trust related attacks, we know that trust models are under many security threats that can break the func￾tionality of trust management systems. Therefore, trust models should consider multiple trust factors in order to evaluate trustees accurately. They should also take more defensive measures to avoid the negative effect of bad recommendations so as to improve the stability of trust evaluation in the dynamically hostile environment. 2.3. Outlier detection methods In Section 2.2, we have already introduced that malicious rec￾ommenders which perform some trust related attacks such as bad mouthing attacks and ballot stuffing attacks will provide bad recom￾mendations to the trustor. If the trustor uses these bad recommen￾dations, the accuracy of the recommendation trust evaluation will be

G.Chen et al. Computer Networks 190 (2021)107952 reduced.In order to effectively avoid the negative impact of these trust mouthing attacks,ballot stuffing attacks,selective misbehavior attacks related attacks,these bad recommendations can be regarded as outliers and on-off attacks.We also explain these attacks in detail in Section 2.2 and detected by outlier detection methods.Therefore,the trustor can and our trust model can resist these trust related attacks effectively.In use a recommendation filtering algorithm based on outlier detection the next paragraph,we introduce some specific trust models and their methods to eliminate these bad recommendations when evaluating the advantages and limitations. recommendation trust of trustees.In this subsection,we introduce some Chen et al.[18]clarified the concept of trust and reputation in common outlier detection methods and then we will compare and IoT and proposed an IoT trust management model based on fuzzy analyze these methods in Section 5.2.1 so as to explain why we choose theory.But in their model,a trustor cannot evaluate trustees without k-means to filter out bad recommendations. direct interactions.To solve this problem,our trust model adopts the recommendation trust evaluation to help the trustor calculate the trust Grubbs'test:Grubbs'test which was proposed by Grubbs et al. [11]is a statistically based outlier detection method.It is used to value of trustees indirectly.Nitti et al.[19]proposed two types of trust models:subjective model and objective model.In the subjective detect outliers in one-dimensional data under the assumption that model,each trustor calculates and stores the trust value of trustees the data is generated by a Gaussian distribution.It calculates the itself.In the objective model,a distributed hash table is designed for z score of each data instance and compares the z score with the storing the information of each node.But these two trust models are threshold.The z score is calculated by dividing the absolute value susceptible to malicious nodes in the network.Considering that the of the difference between the data instance and the average value trust evaluation is sensitive to context,Saied et al.[20]designed a of the data by the standard deviation of the data.A data instance context-aware and multi-service approach to trust management.The whose z score greater than the threshold will be regarded as an model selects a certain number of historical trust values to calculate outlier. the current trust value.But it is difficult to quickly evaluate the Box plot:Box plot [12]is a simple statistical technique to detect trustworthiness when there is not enough trust related information.To outliers in one-dimensional and multi-dimensional data.It first solve this problem,Xia et al.[21]designed a kernel-based nonlinear calculates the Inter Quartile Range(/OR)which is the difference multivariate gray prediction model to predict the direct trust which between the first quartile(O)and the third quartile(O3).Then, needs a small amount of historical information.Experimental results data instances greater than 3+1.5 /OR or less than 01-1.5 indicate the accuracy and convergence rate of the trust model.But,the IOR will be regarded as outliers. proportion of malicious nodes is only 30%in their experiments.Our Isolation forest:Isolation forest was brought by Liu et al.[13] trust model is still accurate when the proportion of malicious nodes is and can be viewed as the unsupervised counterpart of decision as high as 70%. trees.An isolation tree is generated with a given sample set by Some work brings social attributes to the IoT.A comprehensive recursively choosing one random attribute and one random split model was proposed in [22]and used the social relations of users on value of the data on every tree node until the height limit is the real social platform to establish the social relationship among nodes reached or the terminal leaf contains one distinct data instance. so as to make the experimental results more persuasive.Chen et al.[9] The principle is that outliers have a higher chance of being divided trust into three types based on social attributes:honesty,coop- isolated on an earlier stage than normal data instances.Hence, eration and community-interest.The trust model separately calculates outliers are expected to have a shorter height in the isolation the three types of trust and combines them according to the actual trees. scenario.However,it needs a large number of experiments to determine Local outlier factor(LOF):LOF [14]is a well-known approach the best weight.When the trustor and the trustee do not interact with that first introduced the concept of local outliers.The LOF score each other directly,recommendations are important to trust evaluation. for a data instance is based on the average ratio of the instance's Xia et al.[23]proposed a trust model that divides recommendations neighbors'density to the instance's density.For a normal instance into direct recommendations and indirect recommendations and uses lying in a dense region,its local density will be similar to that of direct trust and similarity value to calculate the weight of the two its neighbors,while for an outlier,its local density will be lower types of recommendations.But their work lacked security analysis of than that of its neighbors.Hence,LOF scores of normal instances their model.To avoid the impact of bad recommendations,a trust are close to 1 while outliers'LOF scores are much greater than 1. model with clustering technique was proposed in [5]to dynamically DBSCAN:DBSCAN [15]is a density-based clustering algorithm filter out attacks related to bad recommendations.Similarly,Chen and can be used as an outlier detection method.It has two user- et al.[6]developed a trust management system that adopts distributed specified parameters that determine the density of the data and collaborative filtering to select feedback and uses social contacts as it autonomously determines the number of clusters.Users can filters.However,they did not illustrate how to establish social contacts determine which clusters of data instances are outliers according between nodes.Same as above related work,our model adopts a to the rules set in advance by themselves. recommendation filtering algorithm to filter out bad recommendations k-Means:k-Means [16]is another clustering algorithm and can provided by malicious recommenders.Besides,our model considers also be used for outlier detection.k is the number of clusters and three important factors:direct trust,similarity value and confidence needs to be specified by users in advance.Similar to DBSCAN, level to further reduce the impact of bad recommendations. users can determine which clusters of data instances are outliers Machine learning based trust models have been proposed in re- according to their own rules. cent years.A trust model based on SVM and k-means was presented in [24]to classify the extracted trust features and combine them to 3.Related work produce a final trust value,whereas it is only valid in some situations. Caminha et al.[25]proposed a smart trust management method that In this section,we survey recently proposed trust models for en- can detect on-off attacks.However,this method cannot resist collusion hancing the security of IoT systems.Guo et al.[17]published a survey attacks such as bad mouthing attacks.A trust evaluation method based and presented a classification of trust models for IoT and this classi. on usage scenarios was presented in [26].The authors believed that fication contains eight classes based on five trust design dimensions: the trustworthiness of the service provided by the target node varies trust composition,trust propagation,trust aggregation,trust update according to the scenario in which the service is used and they used and trust formation.The trust model we propose also involves these neural network training to obtain the trustworthiness of the service. five dimensions.Furthermore,they presented trust related attacks that Alshehri et al.[7]proposed a clustering-driven intelligent method can perturb the trust computation models:self promoting attacks,bad that can filter out dishonest recommenders.In addition,Boudagdigue

Computer Networks 190 (2021) 107952 3 G. Chen et al. reduced. In order to effectively avoid the negative impact of these trust related attacks, these bad recommendations can be regarded as outliers and detected by outlier detection methods. Therefore, the trustor can use a recommendation filtering algorithm based on outlier detection methods to eliminate these bad recommendations when evaluating the recommendation trust of trustees. In this subsection, we introduce some common outlier detection methods and then we will compare and analyze these methods in Section 5.2.1 so as to explain why we choose 𝑘-means to filter out bad recommendations. • Grubbs’ test: Grubbs’ test which was proposed by Grubbs et al. [11] is a statistically based outlier detection method. It is used to detect outliers in one-dimensional data under the assumption that the data is generated by a Gaussian distribution. It calculates the 𝑧 score of each data instance and compares the 𝑧 score with the threshold. The 𝑧 score is calculated by dividing the absolute value of the difference between the data instance and the average value of the data by the standard deviation of the data. A data instance whose 𝑧 score greater than the threshold will be regarded as an outlier. • Box plot: Box plot [12] is a simple statistical technique to detect outliers in one-dimensional and multi-dimensional data. It first calculates the Inter Quartile Range(𝐼𝑄𝑅) which is the difference between the first quartile(𝑄1 ) and the third quartile(𝑄3 ). Then, data instances greater than 𝑄3 + 1.5 ∗ 𝐼𝑄𝑅 or less than 𝑄1 − 1.5 ∗ 𝐼𝑄𝑅 will be regarded as outliers. • Isolation forest: Isolation forest was brought by Liu et al. [13] and can be viewed as the unsupervised counterpart of decision trees. An isolation tree is generated with a given sample set by recursively choosing one random attribute and one random split value of the data on every tree node until the height limit is reached or the terminal leaf contains one distinct data instance. The principle is that outliers have a higher chance of being isolated on an earlier stage than normal data instances. Hence, outliers are expected to have a shorter height in the isolation trees. • Local outlier factor(LOF): LOF [14] is a well-known approach that first introduced the concept of local outliers. The LOF score for a data instance is based on the average ratio of the instance’s neighbors’ density to the instance’s density. For a normal instance lying in a dense region, its local density will be similar to that of its neighbors, while for an outlier, its local density will be lower than that of its neighbors. Hence, LOF scores of normal instances are close to 1 while outliers’ LOF scores are much greater than 1. • DBSCAN: DBSCAN [15] is a density-based clustering algorithm and can be used as an outlier detection method. It has two user￾specified parameters that determine the density of the data and it autonomously determines the number of clusters. Users can determine which clusters of data instances are outliers according to the rules set in advance by themselves. • 𝑘-Means: 𝑘-Means [16] is another clustering algorithm and can also be used for outlier detection. 𝑘 is the number of clusters and needs to be specified by users in advance. Similar to DBSCAN, users can determine which clusters of data instances are outliers according to their own rules. 3. Related work In this section, we survey recently proposed trust models for en￾hancing the security of IoT systems. Guo et al. [17] published a survey and presented a classification of trust models for IoT and this classi￾fication contains eight classes based on five trust design dimensions: trust composition, trust propagation, trust aggregation, trust update and trust formation. The trust model we propose also involves these five dimensions. Furthermore, they presented trust related attacks that can perturb the trust computation models: self promoting attacks, bad mouthing attacks, ballot stuffing attacks, selective misbehavior attacks and on–off attacks. We also explain these attacks in detail in Section 2.2 and our trust model can resist these trust related attacks effectively. In the next paragraph, we introduce some specific trust models and their advantages and limitations. Chen et al. [18] clarified the concept of trust and reputation in IoT and proposed an IoT trust management model based on fuzzy theory. But in their model, a trustor cannot evaluate trustees without direct interactions. To solve this problem, our trust model adopts the recommendation trust evaluation to help the trustor calculate the trust value of trustees indirectly. Nitti et al. [19] proposed two types of trust models: subjective model and objective model. In the subjective model, each trustor calculates and stores the trust value of trustees itself. In the objective model, a distributed hash table is designed for storing the information of each node. But these two trust models are susceptible to malicious nodes in the network. Considering that the trust evaluation is sensitive to context, Saied et al. [20] designed a context-aware and multi-service approach to trust management. The model selects a certain number of historical trust values to calculate the current trust value. But it is difficult to quickly evaluate the trustworthiness when there is not enough trust related information. To solve this problem, Xia et al. [21] designed a kernel-based nonlinear multivariate gray prediction model to predict the direct trust which needs a small amount of historical information. Experimental results indicate the accuracy and convergence rate of the trust model. But, the proportion of malicious nodes is only 30% in their experiments. Our trust model is still accurate when the proportion of malicious nodes is as high as 70%. Some work brings social attributes to the IoT. A comprehensive model was proposed in [22] and used the social relations of users on the real social platform to establish the social relationship among nodes so as to make the experimental results more persuasive. Chen et al. [9] divided trust into three types based on social attributes: honesty, coop￾eration and community-interest. The trust model separately calculates the three types of trust and combines them according to the actual scenario. However, it needs a large number of experiments to determine the best weight. When the trustor and the trustee do not interact with each other directly, recommendations are important to trust evaluation. Xia et al. [23] proposed a trust model that divides recommendations into direct recommendations and indirect recommendations and uses direct trust and similarity value to calculate the weight of the two types of recommendations. But their work lacked security analysis of their model. To avoid the impact of bad recommendations, a trust model with clustering technique was proposed in [5] to dynamically filter out attacks related to bad recommendations. Similarly, Chen et al. [6] developed a trust management system that adopts distributed collaborative filtering to select feedback and uses social contacts as filters. However, they did not illustrate how to establish social contacts between nodes. Same as above related work, our model adopts a recommendation filtering algorithm to filter out bad recommendations provided by malicious recommenders. Besides, our model considers three important factors: direct trust, similarity value and confidence level to further reduce the impact of bad recommendations. Machine learning based trust models have been proposed in re￾cent years. A trust model based on SVM and 𝑘-means was presented in [24] to classify the extracted trust features and combine them to produce a final trust value, whereas it is only valid in some situations. Caminha et al. [25] proposed a smart trust management method that can detect on–off attacks. However, this method cannot resist collusion attacks such as bad mouthing attacks. A trust evaluation method based on usage scenarios was presented in [26]. The authors believed that the trustworthiness of the service provided by the target node varies according to the scenario in which the service is used and they used neural network training to obtain the trustworthiness of the service. Alshehri et al. [7] proposed a clustering-driven intelligent method that can filter out dishonest recommenders. In addition, Boudagdigue

G.Chen et al. Computer Networks 190(2021)107952 Node TTP Feedback Feedback Feedback sender Recerver RKepository Trust Evaluation Request Request Recommendation Sender Receiver Direct Trust Trust Trust Value Trust Value Receiver Sender Synthesis Trust Fig.1.Architecture of the trust model. trustee and recommender are all nodes.Meanwhile,a node can play Node Closest TTP Other TTPs different roles according to different requirements.Nodes are usually Send feedback IoT objects with limited capabilities and resources so that they can Store feedback hardly perform complex computing all the time.To solve such prob- lems,TTPs that provide safe and reliable trust computing environment are introduced into our trust model.We divide the nodes into multiple groups and each group has a TTP responsible for assisting the node in trust evaluation.Each node sends feedback about the services it has Trust evaluation request received from service providers to its closest TTP in the process of Search feedback trust evaluation.Hence,our system architecture is a hybrid architecture and TTPs in our architecture play supporting roles.The nodes in the Feedback request trustor roles really need to perform trust evaluation to evaluate the trust Send feedback back value of trustees.With the help of TTPs to evaluate the trust value of trustees,nodes can save energy as much as possible and thus extend Trust evaluation their lifetime. There are three components in a node,which are feedback sender, request sender and trust value receiver.The details of these three Send trust value components are as follows. Feedback sender:It sends feedback that is provided by nodes to TTPs.If a node is satisfied with the service it has received from the service provider,it will give positive feedback to its closest Fig.2.Process of trust evaluation. TTP through the feedback sender. Request sender:If a node wants to learn about the trust value of others,it will send a request for trust evaluation to its closest TTP et al.[27]proposed a distributed advanced analytical trust model through the request sender. based on a Markov chain which can effectively resist bad mouthing Trust value receiver:It receives the trust value of trustees that is attacks and ballot stuffing attacks.But they do not explain how to sent by TTPs. select suitable nodes as recommenders.Wang et al.[4]proposed a There are five components in a TTP,which are feedback receiver, novel trust mechanism based on a multilayer structure that solves energy consumption problems.Trust models based on machine learning feedback repository,request receiver,trust evaluation module and trust value sender.The details of these five components are as follows. may require large amounts of data to ensure the performance of trust evaluation.On the contrary,our model uses adaptive weight to com- Feedback receiver:It is a component that receives feedback from bine direct trust and recommendation trust according to the current nodes and then sends the feedback to the feedback repository. environment and only requires some necessary information to rapidly Feedback repository:It is a place where stores feedback from evaluate trustees.In addition,the introduction of TTPs can reduce the nodes.The feedback in the feedback repository will be used to energy consumption of IoT objects and extend their lifetime. evaluate the trust value of trustees later. Request receiver:It receives the request for trust evaluation from 4.System overview the trustor and then notifies trust evaluation module to evaluate the specific trustee's trust value In this section,we first present the architecture of our trust model Trust evaluation module:This module computes the direct trust, and specify the role of each component in the architecture.Then,we recommendation trust and synthesis trust of trustees through give the process of trust evaluation in our trust model so as to explain feedback from feedback repository and the trust model we pro- how the components work together to establish trust relationships posed. .Trust value sender:It sends trust value that is evaluated by trust among objects in a dynamically hostile IoT environment. evaluation module to the trustor sending trust request before. 4.1.The proposed system architecture 4.2.Process of trust evaluation Fig.1 illustrates the system architecture of our trust model.There In this subsection,we elaborate on how the components mentioned are two main entities in it:nodes and trust third parties (TTPs).Trustor, above cooperate with each other in the trust management system in

Computer Networks 190 (2021) 107952 4 G. Chen et al. Fig. 1. Architecture of the trust model. Fig. 2. Process of trust evaluation. et al. [27] proposed a distributed advanced analytical trust model based on a Markov chain which can effectively resist bad mouthing attacks and ballot stuffing attacks. But they do not explain how to select suitable nodes as recommenders. Wang et al. [4] proposed a novel trust mechanism based on a multilayer structure that solves energy consumption problems. Trust models based on machine learning may require large amounts of data to ensure the performance of trust evaluation. On the contrary, our model uses adaptive weight to com￾bine direct trust and recommendation trust according to the current environment and only requires some necessary information to rapidly evaluate trustees. In addition, the introduction of TTPs can reduce the energy consumption of IoT objects and extend their lifetime. 4. System overview In this section, we first present the architecture of our trust model and specify the role of each component in the architecture. Then, we give the process of trust evaluation in our trust model so as to explain how the components work together to establish trust relationships among objects in a dynamically hostile IoT environment. 4.1. The proposed system architecture Fig. 1 illustrates the system architecture of our trust model. There are two main entities in it: nodes and trust third parties (TTPs). Trustor, trustee and recommender are all nodes. Meanwhile, a node can play different roles according to different requirements. Nodes are usually IoT objects with limited capabilities and resources so that they can hardly perform complex computing all the time. To solve such prob￾lems, TTPs that provide safe and reliable trust computing environment are introduced into our trust model. We divide the nodes into multiple groups and each group has a TTP responsible for assisting the node in trust evaluation. Each node sends feedback about the services it has received from service providers to its closest TTP in the process of trust evaluation. Hence, our system architecture is a hybrid architecture and TTPs in our architecture play supporting roles. The nodes in the trustor roles really need to perform trust evaluation to evaluate the trust value of trustees. With the help of TTPs to evaluate the trust value of trustees, nodes can save energy as much as possible and thus extend their lifetime. There are three components in a node, which are feedback sender, request sender and trust value receiver. The details of these three components are as follows. • Feedback sender: It sends feedback that is provided by nodes to TTPs. If a node is satisfied with the service it has received from the service provider, it will give positive feedback to its closest TTP through the feedback sender. • Request sender: If a node wants to learn about the trust value of others, it will send a request for trust evaluation to its closest TTP through the request sender. • Trust value receiver: It receives the trust value of trustees that is sent by TTPs. There are five components in a TTP, which are feedback receiver, feedback repository, request receiver, trust evaluation module and trust value sender. The details of these five components are as follows. • Feedback receiver: It is a component that receives feedback from nodes and then sends the feedback to the feedback repository. • Feedback repository: It is a place where stores feedback from nodes. The feedback in the feedback repository will be used to evaluate the trust value of trustees later. • Request receiver: It receives the request for trust evaluation from the trustor and then notifies trust evaluation module to evaluate the specific trustee’s trust value. • Trust evaluation module: This module computes the direct trust, recommendation trust and synthesis trust of trustees through feedback from feedback repository and the trust model we pro￾posed. • Trust value sender: It sends trust value that is evaluated by trust evaluation module to the trustor sending trust request before. 4.2. Process of trust evaluation In this subsection, we elaborate on how the components mentioned above cooperate with each other in the trust management system in

G.Chen et al. Computer Networks 190(2021)107952 order to implement trust evaluation.Fig.2 illustrates the process of In Eq.(2),denotes the amount of positive feedback provided trust evaluation and the detailed description is as follows: by trustoriabout trusteeat timeanddenotes the amount of negative feedback.e is a time decay function and is a decay (1)Each node periodically sends feedback about the services it has factor that affects the decay rate of the time decay function.m is the received from service providers to its closest TTP via its feedback sender. size of the sliding window.pf and nf are the amount of positive and negative feedback at time t,respectively. (2)Each feedback receiver of the TTP receives feedback from nodes and uploads the feedback to its feedback repository. Another problem we need to solve in the direct trust evaluation is to migrate the risk of on-off attacks.We use a penalty factor to amplify (3)A trustor will use request sender to a send trust evaluation the influence of negative feedback and the trust value of the trustee request to its closest TTP when it wants to obtain the trust value will decrease faster if it provides the trustor with bad service.Trustor of the target trustee. will give negative feedback about the trustee and the weight of negative (4)When a TTP receives a trust evaluation request from the trustor, feedback will be greater with the influence of the penalty factor.Eq.(3) it first searches whether there is feedback about the target is the final formula to evaluate the direct trust. trustee in its feedback repository.If not,it will request feedback about that trustee from other TTPs.The TTP which stores the DT= 唱+1 required feedback will send them back. (3) 号+唱*PF+2 (5)The TTP utilizes the feedback and its trust evaluation module to evaluate the direct trust,recommendation trust and synthesis In Eq.(3),PF is the penalty factor.The calculation of and trust of the target trustee can be found in Eq.(2). (6)After the work of the trust evaluation module,the TTP sends the target trustee's trust value to the trustor through the trust value 5.2.Recommendation trust sender. (7)Finally,the trustor receives the trust value of the trustee and When the trustor does not interact with the trustee directly,it then decides whether to receive services provided by the trustee. lacks essential information to evaluate the trustee's direct trust.At this time,the trustor needs to request recommendations from recom- 5.The proposed trust model menders who have interacted with the trustee before and then uses these recommendations to calculate the recommendation trust of the In this section,we propose the concrete methods used in the trust trustee.Under the trust related attacks,the trustor may receive some model that can evaluate the trust value accurately and steadily in the bad recommendations.To avoid the influence of these attacks,we dynamically hostile environment. propose a recommendation filtering algorithm based on k-means to filter out malicious recommenders.For the recommendations provided 5.1.Direct trust by remaining recommenders after filtering,some important factors are applied to ensure the accuracy of the recommendation trust. We adopt a Bayesian inference model [28]based on beta probability density function to evaluate the direct trust of the trustee.Eq.(1)shows 5.2.1.The choice of k-means the direct trust of trustor i about trustee j. We have already discussed why we need a recommendation filtering a9+1 algorithm based on outlier detection methods in Section 2.3.Now we (1) analyze the applicability of these outlier detection methods according 0+唱+2 to the characteristics of bad recommendations and explain why we finally propose a recommendation filtering algorithm based on k-means In Eq.(1),DT represents the direct trust of trustor i about trustee instead of other outlier detection methods.The bad recommendations j at time t.It is a real number in the range of [0,1]where 1 indicates about the trustee provided by malicious recommenders are often op- complete trust,0.5 indicates uncertainty and 0 indicates complete posite to the ground truth of the trustee.For example,if the ground distrust.denotes the total number of positive feedback given by truth of a well-behaved trustee is 1,malicious recommenders are likely trustor i about trustee j from the beginning of trust evaluation to to give recommendations less than 0.5 to reduce the recommendation current time 1.Similarly,is the total number of negative feedback. trust of the trustee.These behaviors performed by malicious recom- If the services provided by the trustee can meet the requirements,the menders are called bad mouthing attacks.Ballot stuffing attacks are trustor will give positive feedback to the trustee.On the contrary,the just the opposite of these behaviors. trustor will give negative feedback. When the proportion of malicious recommenders is relatively small, We consider the influence of feedback is blunted over time because most of the recommendations received by the trustor are close to feedback from past interactions cannot accurately reflect the current the ground truth of the trustee.The six outlier detection methods status of the trustee.So the weight of previous feedback should be re- introduced above can all effectively detect bad recommendations in duced.To achieve this,we introduce a time decay function whose value such a case.Then,the trustor can filter out these outliers based on the will decrease constantly over time,and adopt a sliding window which detection results.However,when the proportion of malicious recom- only stores and updates the feedback from recent interactions.The menders increases,the proportion of bad recommendations will also sliding window has m time slots in order from its left side to the right increase.Not all of these outlier detection methods are effective in this side.Each time slot stores the amount of positive and negative feedback situation.The average value of all recommendations is no longer close during an interaction and the corresponding time when this interaction to the average value of good recommendations,but a value between happened.The rightmost time slot stores the latest feedback that has good recommendations and bad recommendations.The z scores of the most important influence to the direct trust evaluation.Eq.(2) all recommendations will be less than the fixed threshold and thus shows the calculation of positive feedback and negative feedback. grubbs'test cannot detect bad recommendations as outliers.Similarly, the first quartile will fall among bad recommendations instead of good 9-∑e*+pj recommendations,resulting in all recommendations being within the (2) specified range of the box plot.Therefore,box plot cannot detect =∑e-+n时 bad recommendations either.Both isolation forest and LOF treat data instances in the sparse area as outliers.The difference is that isolation

Computer Networks 190 (2021) 107952 5 G. Chen et al. order to implement trust evaluation. Fig. 2 illustrates the process of trust evaluation and the detailed description is as follows: (1) Each node periodically sends feedback about the services it has received from service providers to its closest TTP via its feedback sender. (2) Each feedback receiver of the TTP receives feedback from nodes and uploads the feedback to its feedback repository. (3) A trustor will use request sender to a send trust evaluation request to its closest TTP when it wants to obtain the trust value of the target trustee. (4) When a TTP receives a trust evaluation request from the trustor, it first searches whether there is feedback about the target trustee in its feedback repository. If not, it will request feedback about that trustee from other TTPs. The TTP which stores the required feedback will send them back. (5) The TTP utilizes the feedback and its trust evaluation module to evaluate the direct trust, recommendation trust and synthesis trust of the target trustee. (6) After the work of the trust evaluation module, the TTP sends the target trustee’s trust value to the trustor through the trust value sender. (7) Finally, the trustor receives the trust value of the trustee and then decides whether to receive services provided by the trustee. 5. The proposed trust model In this section, we propose the concrete methods used in the trust model that can evaluate the trust value accurately and steadily in the dynamically hostile environment. 5.1. Direct trust We adopt a Bayesian inference model [28] based on beta probability density function to evaluate the direct trust of the trustee. Eq. (1) shows the direct trust of trustor 𝑖 about trustee 𝑗. 𝐷𝑇 (𝑡) 𝑖𝑗 = 𝛼 (𝑡) 𝑖𝑗 + 1 𝛼 (𝑡) 𝑖𝑗 + 𝛽 (𝑡) 𝑖𝑗 + 2 (1) In Eq. (1), 𝐷𝑇 (𝑡) 𝑖𝑗 represents the direct trust of trustor 𝑖 about trustee 𝑗 at time 𝑡. It is a real number in the range of [0, 1] where 1 indicates complete trust, 0.5 indicates uncertainty and 0 indicates complete distrust. 𝛼 (𝑡) 𝑖𝑗 denotes the total number of positive feedback given by trustor 𝑖 about trustee 𝑗 from the beginning of trust evaluation to current time 𝑡. Similarly, 𝛽 (𝑡) 𝑖𝑗 is the total number of negative feedback. If the services provided by the trustee can meet the requirements, the trustor will give positive feedback to the trustee. On the contrary, the trustor will give negative feedback. We consider the influence of feedback is blunted over time because feedback from past interactions cannot accurately reflect the current status of the trustee. So the weight of previous feedback should be re￾duced. To achieve this, we introduce a time decay function whose value will decrease constantly over time, and adopt a sliding window which only stores and updates the feedback from recent interactions. The sliding window has 𝑚 time slots in order from its left side to the right side. Each time slot stores the amount of positive and negative feedback during an interaction and the corresponding time when this interaction happened. The rightmost time slot stores the latest feedback that has the most important influence to the direct trust evaluation. Eq. (2) shows the calculation of positive feedback and negative feedback. 𝛼 (𝑡) 𝑖𝑗 = ∑𝑚 𝑖=1 𝑒 −𝜆(𝑡−𝑡 𝑖 ) ∗ 𝛼 (𝑡 𝑖 ) 𝑖𝑗 + 𝑝𝑓 𝛽 (𝑡) 𝑖𝑗 = ∑𝑚 𝑖=1 𝑒 −𝜆(𝑡−𝑡 𝑖 ) ∗ 𝛽 (𝑡 𝑖 ) 𝑖𝑗 + 𝑛𝑓 (2) In Eq. (2), 𝛼 (𝑡 𝑖 ) 𝑖𝑗 denotes the amount of positive feedback provided by trustor 𝑖 about trustee 𝑗 at time 𝑡 𝑖 and 𝛽 (𝑡 𝑖 ) 𝑖𝑗 denotes the amount of negative feedback. 𝑒 −𝜆(𝑡−𝑡 𝑖 ) is a time decay function and 𝜆 is a decay factor that affects the decay rate of the time decay function. 𝑚 is the size of the sliding window. 𝑝𝑓 and 𝑛𝑓 are the amount of positive and negative feedback at time 𝑡, respectively. Another problem we need to solve in the direct trust evaluation is to migrate the risk of on–off attacks. We use a penalty factor to amplify the influence of negative feedback and the trust value of the trustee will decrease faster if it provides the trustor with bad service. Trustor will give negative feedback about the trustee and the weight of negative feedback will be greater with the influence of the penalty factor. Eq. (3) is the final formula to evaluate the direct trust. 𝐷𝑇 (𝑡) 𝑖𝑗 = 𝛼 (𝑡) 𝑖𝑗 + 1 𝛼 (𝑡) 𝑖𝑗 + 𝛽 (𝑡) 𝑖𝑗 ∗ 𝑃 𝐹 + 2 (3) In Eq. (3), 𝑃 𝐹 is the penalty factor. The calculation of 𝛼 (𝑡) 𝑖𝑗 and 𝛽 (𝑡) 𝑖𝑗 can be found in Eq. (2). 5.2. Recommendation trust When the trustor does not interact with the trustee directly, it lacks essential information to evaluate the trustee’s direct trust. At this time, the trustor needs to request recommendations from recom￾menders who have interacted with the trustee before and then uses these recommendations to calculate the recommendation trust of the trustee. Under the trust related attacks, the trustor may receive some bad recommendations. To avoid the influence of these attacks, we propose a recommendation filtering algorithm based on 𝑘-means to filter out malicious recommenders. For the recommendations provided by remaining recommenders after filtering, some important factors are applied to ensure the accuracy of the recommendation trust. 5.2.1. The choice of 𝑘-means We have already discussed why we need a recommendation filtering algorithm based on outlier detection methods in Section 2.3. Now we analyze the applicability of these outlier detection methods according to the characteristics of bad recommendations and explain why we finally propose a recommendation filtering algorithm based on 𝑘-means instead of other outlier detection methods. The bad recommendations about the trustee provided by malicious recommenders are often op￾posite to the ground truth of the trustee. For example, if the ground truth of a well-behaved trustee is 1, malicious recommenders are likely to give recommendations less than 0.5 to reduce the recommendation trust of the trustee. These behaviors performed by malicious recom￾menders are called bad mouthing attacks. Ballot stuffing attacks are just the opposite of these behaviors. When the proportion of malicious recommenders is relatively small, most of the recommendations received by the trustor are close to the ground truth of the trustee. The six outlier detection methods introduced above can all effectively detect bad recommendations in such a case. Then, the trustor can filter out these outliers based on the detection results. However, when the proportion of malicious recom￾menders increases, the proportion of bad recommendations will also increase. Not all of these outlier detection methods are effective in this situation. The average value of all recommendations is no longer close to the average value of good recommendations, but a value between good recommendations and bad recommendations. The 𝑧 scores of all recommendations will be less than the fixed threshold and thus grubbs’ test cannot detect bad recommendations as outliers. Similarly, the first quartile will fall among bad recommendations instead of good recommendations, resulting in all recommendations being within the specified range of the box plot. Therefore, box plot cannot detect bad recommendations either. Both isolation forest and LOF treat data instances in the sparse area as outliers. The difference is that isolation