Artificial Intelligence Review(2005)24: 179-19 C Springer 2005 DOI10.1007/s10462-005-4612-x Explanation in Recommender Systems DAVID MCSHERRY School of Computing and Information Engineering, University of Ulster, Coleraine BT52 ISA, Northern Ireland, UK(e- mail: dmg mcsherry @ulster: ac uk) Abstract. There is increasing awareness in recommender systems research of the need to make the recommendation process more transparent to users. Following a brief review of existing approaches to explanation in recommender systems, we focus in this paper on a case-based reasoning(CBR) approach to product recommendation that offers important benefits in terms of the ease with which the recommendation process can be explained and the system s recommendations can be justified. For example, rec- ommendations based on incomplete queries can be justified on the grounds that the er's preferences with respect to attributes not mentioned in her query cannot affect the outcome. We also show how the relevance of any question the user is asked can be explained in terms of its ability to discriminate between competing cases, thus giving users a unique insight into the recommendation process. Keywords: attribute-selection strategy, case-based reasoning, explanation, recommender 1. Introduction The importance of intelligent systems having the ability to explain their reasoning is well recognised in domains such as medical decision aking and intelligent tutoring (e.g. Armengol et al., 2001; Sormo and Aamodt, 2002; Evans-Romaine and Marling, 2003). In an intelli gent tutoring system, for example, communicating the reasoning pro- cess to students may be as important as finding the right solution. Until recently, explanation in recommender systems appears to have been a relatively neglected issue. However, recent research has high lighted the importance of making the recommendation process more transparent to users and the potential role of explanation in achiev ing this objective(Herlocker et al., 2000; Shimazu, 2002; McSherry 2002b,2003b; Reilly et al,2005) Herlocker et al. (2000) suggest that the black box image of recom- mender systems may be one of the reasons why they have gained much less acceptance in high-risk domains such as holiday packages or invest- ment portfolios than in low-risk domains such as CDs or movies. They
DOI 10.1007/s10462-005-4612-x Artificial Intelligence Review (2005) 24: 179–197 © Springer 2005 Explanation in Recommender Systems DAVID MCSHERRY School of Computing and Information Engineering, University of Ulster, Coleraine BT52 1SA, Northern Ireland, UK (e-mail: dmg.mcsherry@ulster.ac.uk) Abstract. There is increasing awareness in recommender systems research of the need to make the recommendation process more transparent to users. Following a brief review of existing approaches to explanation in recommender systems, we focus in this paper on a case-based reasoning (CBR) approach to product recommendation that offers important benefits in terms of the ease with which the recommendation process can be explained and the system’s recommendations can be justified. For example, recommendations based on incomplete queries can be justified on the grounds that the user’s preferences with respect to attributes not mentioned in her query cannot affect the outcome. We also show how the relevance of any question the user is asked can be explained in terms of its ability to discriminate between competing cases, thus giving users a unique insight into the recommendation process. Keywords: attribute-selection strategy, case-based reasoning, explanation, recommender systems 1. Introduction The importance of intelligent systems having the ability to explain their reasoning is well recognised in domains such as medical decision making and intelligent tutoring (e.g. Armengol et al., 2001; Sørmo and Aamodt, 2002; Evans-Romaine and Marling, 2003). In an intelligent tutoring system, for example, communicating the reasoning process to students may be as important as finding the right solution. Until recently, explanation in recommender systems appears to have been a relatively neglected issue. However, recent research has highlighted the importance of making the recommendation process more transparent to users and the potential role of explanation in achieving this objective (Herlocker et al., 2000; Shimazu, 2002; McSherry, 2002b, 2003b; Reilly et al., 2005). Herlocker et al. (2000) suggest that the black box image of recommender systems may be one of the reasons why they have gained much less acceptance in high-risk domains such as holiday packages or investment portfolios than in low-risk domains such as CDs or movies. They
180 D MCSHERRY argue that extracting meaningful explanations from the computational models on which recommendations are based is a challenge that must e addressed to enable the development of recommender systems that are more understandable, more effective, and more acceptable. It is an argument that seems equally compelling in collaborative and content based approaches to product recommendation McSherry (2003a)proposes a case-base reasoning(CBR)approach to product recommendation that combines an effective strategy for reducing the length of recommendation dialogues with a mechanism for ensuring that the dialogue is terminated only when it is certain that the recommendation will be the same no matter how the user chooses to extend her query. Referring to the approach as incremen- tal nearest neighbour (iNN), we focus here on the benefits it offers terms of making the recommendation process more transparent to users. One advantage is that recommendations based on incomplete queries can be justified on the grounds that the user's preferences with respect to attributes not mentioned in her query cannot affect the out come. We also show how the relevance of any question the user is asked can be explained in terms of its ability to discriminate between competing cases, thus giving users a unique insight into the recom ommender systems and some of the pproaches to explanation in rec- In Section 2. we examine existing essons learned from this research In Section 3, we present a detailed account of the recommendation process in inN and the important role played by the concept of case dominance in the approach. In Section 4, we present an approach to explanation in which there is no requirement for domain knowledge other than the similarity knowledge and cases already available to the system. We demonstrate the approach in a mixed-initiative recom- mender system called Top Case which can explain the relevance of any question the user is asked in strategic terms, recognise when the dia- logue can be safely terminated, and justify its recommendations on the grounds that any un-elicited preferences of the user cannot affect the outcome. Related work is discussed in Section 5 and our conclusions are presented in Section 6 2. Existing Approaches Herlocker et al.(2000)evaluated several approaches to explanation in the collaborative movie recommender movie Lens in terms of their
180 D. MCSHERRY argue that extracting meaningful explanations from the computational models on which recommendations are based is a challenge that must be addressed to enable the development of recommender systems that are more understandable, more effective, and more acceptable. It is an argument that seems equally compelling in collaborative and contentbased approaches to product recommendation. McSherry (2003a) proposes a case-base reasoning (CBR) approach to product recommendation that combines an effective strategy for reducing the length of recommendation dialogues with a mechanism for ensuring that the dialogue is terminated only when it is certain that the recommendation will be the same no matter how the user chooses to extend her query. Referring to the approach as incremental nearest neighbour (iNN), we focus here on the benefits it offers in terms of making the recommendation process more transparent to users. One advantage is that recommendations based on incomplete queries can be justified on the grounds that the user’s preferences with respect to attributes not mentioned in her query cannot affect the outcome. We also show how the relevance of any question the user is asked can be explained in terms of its ability to discriminate between competing cases, thus giving users a unique insight into the recommendation process. In Section 2, we examine existing approaches to explanation in recommender systems and some of the lessons learned from this research. In Section 3, we present a detailed account of the recommendation process in iNN and the important role played by the concept of case dominance in the approach. In Section 4, we present an approach to explanation in which there is no requirement for domain knowledge other than the similarity knowledge and cases already available to the system. We demonstrate the approach in a mixed-initiative recommender system called Top Case which can explain the relevance of any question the user is asked in strategic terms, recognise when the dialogue can be safely terminated, and justify its recommendations on the grounds that any un-elicited preferences of the user cannot affect the outcome. Related work is discussed in Section 5 and our conclusions are presented in Section 6. 2. Existing Approaches Herlocker et al. (2000) evaluated several approaches to explanation in the collaborative movie recommender MovieLens in terms of their
EXPLANATION IN RECOMMENDER SYSTEMS 181 effects on user acceptance of the system's recommendations. The most onvincing explanation of why a movie was recommended was one in which users were shown a histogram of the ratings of the same movie by similar users. Moreover, grouping together of good ratings (4 or and bad ratings(I or 2)and separation of ambivalent ratings(3)was found to increase the effectiveness of the histogram approach. Inter estingly, the second most convincing explanation was a simple state ment of the system's performance in the past e.g Movie Lens has predicted correctly for you 80% of the time in the past Another important finding was that some of the explanations eval uated had a negative impact on acceptance, the goal of explanation in this instance, showing that no explanation may be better than one that is poorly designed CBR recommender systems that can explain their recommenda- tions include Shimazu's(2002) Expert Clerk and McSherry's(2003b) First Case. Expert Clerk can explain why it is proposing two contrast ing products in terms of the trade-offs between their positive and neg ative features e.g This blouse is more expensive but the material is silk. That one is cheaper but the material is polyester Its explanations are based on assumed preferences with respect to attributes not mentioned in the user's query. For example, a blouse made of silk is assumed to be preferred to one made of polyester In a similar way, First Case can explain why one case is more highly recommended than another by highlighting the benefits it offers (McSherry, 2003b). As the following example illustrates, it can also explain why a given product, such as a personal computer, is recom- mended in terms of the compromises it involves with respect to the Case 38 differs from your query only in processor speed and mon- itor size. It is better than Case 50 in terms of memory and price However. the potentia of explanation in recommender sys- tems is not limited to explaining why a particular item is recom- mended. In this paper, we present a CBr recommender system that can also explain the relevance of any question the user is asked in terms of its ability to discriminate between competing cases. Reilly et al. s(2005)dynamic approach to critiquing in recommender sys tems differs from traditional critiquing approaches (e.g. Burke, 2002)
EXPLANATION IN RECOMMENDER SYSTEMS 181 effects on user acceptance of the system’s recommendations. The most convincing explanation of why a movie was recommended was one in which users were shown a histogram of the ratings of the same movie by similar users. Moreover, grouping together of good ratings (4 or 5) and bad ratings (1 or 2) and separation of ambivalent ratings (3) was found to increase the effectiveness of the histogram approach. Interestingly, the second most convincing explanation was a simple statement of the system’s performance in the past e.g. MovieLens has predicted correctly for you 80% of the time in the past. Another important finding was that some of the explanations evaluated had a negative impact on acceptance, the goal of explanation in this instance, showing that no explanation may be better than one that is poorly designed. CBR recommender systems that can explain their recommendations include Shimazu’s (2002) ExpertClerk and McSherry’s (2003b) First Case. ExpertClerk can explain why it is proposing two contrasting products in terms of the trade-offs between their positive and negative features e.g. This blouse is more expensive but the material is silk. That one is cheaper but the material is polyester. Its explanations are based on assumed preferences with respect to attributes not mentioned in the user’s query. For example, a blouse made of silk is assumed to be preferred to one made of polyester. In a similar way, First Case can explain why one case is more highly recommended than another by highlighting the benefits it offers (McSherry, 2003b). As the following example illustrates, it can also explain why a given product, such as a personal computer, is recommended in terms of the compromises it involves with respect to the user’s preferences. Case 38 differs from your query only in processor speed and monitor size. It is better than Case 50 in terms of memory and price. However, the potential role of explanation in recommender systems is not limited to explaining why a particular item is recommended. In this paper, we present a CBR recommender system that can also explain the relevance of any question the user is asked in terms of its ability to discriminate between competing cases. Reilly et al.’s (2005) dynamic approach to critiquing in recommender systems differs from traditional critiquing approaches (e.g. Burke, 2002)
182 D MCSHERRY in that critiques are dynamically generated by the system and may involve compromises as well as improvements relative to the currently recommended case. In this way, the user is informed in advance of trade-offs associated with desired improvements. Before selecting a suggested critique, the user can ask to see an explanation of the trade offs involved In recommender systems that treat some or all of the user's require ments as constraints that must be satisfied, explanation can also play an important role in recovery from the retrieval failures that occur when there is no exact match for the user's requirements(Hammond et al., 1996; McSherry, 2004). Hammond et al.s(1996)Car Naviga tor is a recommender system for cars that uses declarative knowl edge to explain trade-offs that are known to be common causes of retrieval failure in the domain, such as that between fuel economy and horsepower. For example, if the user asks for good fuel economy and high horsepower, she is shown a movie explaining the trade-off between these features. The user is also advised that she will need to revise her query if she hopes to find a car that meets her require ments In recent work, we combined a knowledge-light approach to expla nation of retrieval failure with a mixed-initiative approach to recovery from retrieval failure in a CBR recommender system called Show Me (McSherry, 2004). Failure to retrieve a case that exactly matches the user's query triggers an explanation that draws the user's attention t combinations of features in her query for which there are no matching cases e orry, there are no products that match these combinations of fea- tures in your query:(price 500, type laptop),(type laptop screen size = 19) As well as highlighting areas of the product space in which the case library is lacking in coverage, the explanation may reveal misconceptions on the part of the user such as the price she expects to pay for the prod uct she is seeking. Showing the user only the minimally failing sub-queries of her query, a technique we have adapted from research on co-opera- tive responses to failing data base queries( Gasterland et al., 1992), helps to minimise cognitive load in the approach. Explanation of the retrieval failure is followed in Show Me by a mixed-initiative recovery process in which the user is guided in the selection of one or more constraints to be eliminated from her query(Mcsherry, 2004)
182 D. MCSHERRY in that critiques are dynamically generated by the system and may involve compromises as well as improvements relative to the currently recommended case. In this way, the user is informed in advance of trade-offs associated with desired improvements. Before selecting a suggested critique, the user can ask to see an explanation of the tradeoffs involved. In recommender systems that treat some or all of the user’s requirements as constraints that must be satisfied, explanation can also play an important role in recovery from the retrieval failures that occur when there is no exact match for the user’s requirements (Hammond et al., 1996; McSherry, 2004). Hammond et al.’s (1996) Car Navigator is a recommender system for cars that uses declarative knowledge to explain trade-offs that are known to be common causes of retrieval failure in the domain, such as that between fuel economy and horsepower. For example, if the user asks for good fuel economy and high horsepower, she is shown a movie explaining the trade-off between these features. The user is also advised that she will need to revise her query if she hopes to find a car that meets her requirements. In recent work, we combined a knowledge-light approach to explanation of retrieval failure with a mixed-initiative approach to recovery from retrieval failure in a CBR recommender system called ShowMe (McSherry, 2004). Failure to retrieve a case that exactly matches the user’s query triggers an explanation that draws the user’s attention to combinations of features in her query for which there are no matching cases e.g. Sorry, there are no products that match these combinations of features in your query: (price ≤ 500, type = laptop), (type = laptop, screen size = 19) As well as highlighting areas of the product space in which the case library is lacking in coverage, the explanation may reveal misconceptions on the part of the user such as the price she expects to pay for the product she is seeking. Showing the user only the minimally failing sub-queries of her query, a technique we have adapted from research on co-operative responses to failing database queries (Gasterland et al., 1992), helps to minimise cognitive load in the approach. Explanation of the retrieval failure is followed in ShowMe by a mixed-initiative recovery process in which the user is guided in the selection of one or more constraints to be eliminated from her query (McSherry, 2004)
EXPLANATION IN RECOMMENDER SYSTEMS 183 3. Incremental Nearest Neighbour In this section, a brief overview of conversational CBR(Aha et al 2001)in the context of product recommendation is followed by a detailed account of the recommendation process in iNn and the important role played by the concept of case dominance in the approach. One distinguishing feature of our approach is a goal-driven attribute selection strategy that has been shown to be very effective in reducing the length of recommendation dialogues(McSherry, 2003a) Another is a simple mechanism for ensuring that the dialogue is ter- minated only when it is certain that a more similar case will not be found if the dialogue is allowed to continue 3. 1. Conversational CBr In CBr approaches to product recommendation, descriptions of the available products are stored in a case library and retrieved in response to a query representing the preferences of the user. In con- versational CBR(CCBR)approaches like INN, a query is incremer tally (and often incompletely) elicited in an interactive dialogue with the user. We focus here on approaches in which the retrieval of recom- mended cases is based on their similarity to the elicited query, rather than relying on exact matching as in most decision-tree approaches (e.g. Doyle and Cunningham, 2000: McSherry, 2001b) Given a query Q over a subset Ao of the case attributes A, the similarity of any case C to Q is typically defined to be sim(C, 2)=> wa sima(C, 2), where for each a E A, Wa is the importance weight assigned to a and sima(C, @)is a local measure of the similarity of a(C), the value of a in C, to Ia(Q), the preferred value of a. As usual in practice, we assume that0≤sima(x,y)≤ I for all a∈ a and that sim(x,y)=lif nd only if x=y. We also assume that for each a E A, the distance measure I-sima satisfies the triangle inequalit a generic algorithm for CCBR in product recommendation (CCBR- PR)is shown in Figure 1. At each stage of the recommenda- tion dialogue, the system selects the next most useful attribute, asks the user for the preferred value, and retrieves the case (or product)
EXPLANATION IN RECOMMENDER SYSTEMS 183 3. Incremental Nearest Neighbour In this section, a brief overview of conversational CBR (Aha et al., 2001) in the context of product recommendation is followed by a detailed account of the recommendation process in iNN and the important role played by the concept of case dominance in the approach. One distinguishing feature of our approach is a goal-driven attribute selection strategy that has been shown to be very effective in reducing the length of recommendation dialogues (McSherry, 2003a). Another is a simple mechanism for ensuring that the dialogue is terminated only when it is certain that a more similar case will not be found if the dialogue is allowed to continue. 3.1. Conversational CBR In CBR approaches to product recommendation, descriptions of the available products are stored in a case library and retrieved in response to a query representing the preferences of the user. In conversational CBR (CCBR) approaches like iNN, a query is incrementally (and often incompletely) elicited in an interactive dialogue with the user. We focus here on approaches in which the retrieval of recommended cases is based on their similarity to the elicited query, rather than relying on exact matching as in most decision-tree approaches (e.g. Doyle and Cunningham, 2000; McSherry, 2001b). Given a query Q over a subset AQ of the case attributes A, the similarity of any case C to Q is typically defined to be: sim(C, Q)= a∈AQ wa sima(C, Q), where for each a ∈ A,wa is the importance weight assigned to a and sima(C, Q) is a local measure of the similarity of πa(C), the value of a in C, to πa(Q), the preferred value of a. As usual in practice, we assume that 0 ≤ sima(x, y) ≤ 1 for all a ∈ A and that sima(x, y) = 1 if and only if x = y. We also assume that for each a ∈ A, the distance measure 1−sima satisfies the triangle inequality. A generic algorithm for CCBR in product recommendation (CCBR-PR) is shown in Figure 1. At each stage of the recommendation dialogue, the system selects the next most useful attribute, asks the user for the preferred value, and retrieves the case (or product)