various aspects of the interaction that are important to the user. This categorization provides a common language for users and recommenders to communicate intention, thus providing a key linking algorithms to users'needs e Mapping Metrics -Benchmark-+ Recommender oman process on HRi aspects 三 differs among algorithm guarantee full Metres coverage related datasets salient HRI properties matrx guarantee full coverage Figure 1-2: The Human-Recommender Interaction Process Model To connect hRi theory to algorithms, we designed a set of metrics to benchmark wide families of recommender algorithms. These metrics work with existing accuracy metrics to provide a richer understanding of the strengths and weaknesses of recommender algorithms. By linking the meaning of these metrics to the language of HRI, we have a two-way path connecting user needs to recommender algorithm Research Approach and Contributions To provide support for our thesis and its two related points, we focus our approach in three areas. First, we apply recommender systems in the domain of peer-reviewed research papers, a domain where users have external criteria for selecting items to consume. As far as we are aware this is the first work to propose, design, build, and perform a complete experimental evaluation of recommenders in this domain. Second gue that current recommender systems research in not focused on user needs, but rather on algorithm design and performance. To bring users back into focus, we propose Reproduced with permission of the copyright owner. Further reproduction prohibited without permission
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission
to shift perception from" recommenders giving recommendations to users receiving recommendations'by reflecting on how users perceive recommenders and the ecommendation process. Third, to escape the curse of accuracy, we look to different ways of evaluating recommender systems algorithms. To this end, we propose a new set of recommender metrics and perform detailed experimental evaluation of several recommender algorithms using these metrics Therefore, our research approach has three prongs: Recommending Research Papers, Re-examining the recommendation Process, and Understanding recommender Algorithms. For each prong, we will list the specific research contributions this prong provides to our thesis and the relevant chapters in this dissertation ecommending research Papers We propose a novel approach to use collaborative-filtering and machine learning algorithms he domain of peer reviewed research papers by mining the citation ork between papers( Chapter 5) We propose hybrid recommender algorithms combining content-based and collaborative filtering algorithms for use in this domain following Burke's established hierarchy( Chapter 5) We propose two new extensions to collaborative filtering algorithms in this domain: Denser Collaborative Filtering and Symmetric Collaborative Filtering Chapter 3) We demonstrate the value of our new approach, our new algorithm extensions and our hybrid algorithms against other approaches mimicking how users previously searched for research papers through a series of offline simulation experiments and online user studies( Chapter 5, Chapter 8) We present an analysis of users in the digital library domain, presenting a list of user types, a list of user information seeking tasks, and a set of personas, all of which are used to demonstrate the effectiveness of our overall research approach 10 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission
A User-Centric Approach to the Recommendation Process We argue that a recommendation list, not an individual item, should be the important unit of measure when determining the usefulness or the user satisfaction with a recommender system( Chapter 4) We propose a new metric to measure the similarity of a recommendation list b y comparing the pair-wise similarities of all items in the list: the Intra-List Similarity Metric(ILSM)( Chapter 4) Through a series of offline simulation experiments and online user studies, we demonstrate the value of ILSM and the effect of topic diversification on recommendation lists( Chapter 4) We propose Human-Recommender Interaction theory (HRD) as a way to view the recommendation process from an end user's perspective and as a language for describing user expectations and perceptions of recommendations and the recommender system itself Chapter 6) We introduce the HRI Analytical Process Model, a process model to bridge user types and information seeking tasks to recommender algorithms through the language of HRi and a classification of recommender algorithms by a series of metrics based on HRI(Chapter 6) Understanding Recommender Algorithms We argue for the creation of a new set of recommender metrics, stating that the current predictive accuracy and decision support metrics were not designed to, and thus unable to classify recommender algorithms among the dimensions needed for HRI(Chapter 1, Chapter 3) We propose a set of new recommender metrics to categorize recommender algorithms based on the differences in recommendation lists they generate, and use this information as part of the HRI Process Model( Chapter 8) We conduct a series of offline simulation experiments benchmarking the differences across several recommender algorithms, including collaborati filtering, content-based filtering, and hybrid algorithms, using this new set of Reproduced with permission of the copyright owner. Further reproduction prohibited without permission
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission
metrics(including ILSM) in the domain of peer-reviewed research papers Chapter 8) We run an online user study confirming the differences in recommender algorithms found by our set of new recommender metrics( Chapter 9) Dissertation Organization This dissertation is organized as follows: In Chapter 2, we review previous and related work, paying close attention to work from library and information sciences as well as from recommender systems In Chapter 3, we discuss the recommendation process, implementation the recommender algorithms used in this dissertation, including a discussion of their strengths and weaknesses, and a summary of current predictive accuracy and decision support meti In Chapter 4, we argue for the use of recommendation lists as the relevant units of comparison between recommender algorithms, introduce the Intra-List Similarity metric, and report on a series of experiments studying the diversity of recommendation lists In Chapter 5, we discuss how recommender algorithms can be used in the domain of research papers, including a set of hybrid recommender algorithms for this domain and two sets of experiments exploring how well these algorithms perform in this domain and how users reacted to the recommendation they received In Chapter 6, we introduce Human-Recommender Interaction theory(HRI) and the hRi Process Model In Chapter 7, we present a set of user types, a set of user information seeking tasks, and a list of personas for the domain of peer-reviewed research papers in a digital library. In Chapter 8, we present a set of new recommender metrics and a detailed offline simulation experiment using these metrics to catalog the performance of several recommender algorithms from Chapter 3 12 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission
In Chapter 9, we describe the design and review the results of an online user study validating our experimental results from Chapter 8 Finally, in Chapter 10, we conclude with a summary of our research findings, a discussion of their implications and on potential future work. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission
Reproduced with permission of the copyright owner. Further reproduction prohibited without permission