《电子商务 E-business》阅读文献：Expertise recommender a flexible recommendation system and architecture

Expertise Recommender: A Flexible Recommendation System and Architecture David w. McDonald and mark s. Ackerman Department of Information and Computer Science University of California, Irvine Irvine CA 92697-3425 ddmcdonal, ackerman @ics. uci. edu ABSTRACT Recommendation systems are one possible technology that Locating the expertise necessary to solve difficult problems can augment and assist the natural expertise locating behav- a nuanced social and collaborative problem. In organiza- ior in organizations. A recommendation system that su tions, some people assist others in locating expertise by gests people who have some expertise with a problem holds making referrals. People who make referrals fill key organ the promise to provide, in a very small way, a service simi izational roles that have been identified by CSCW and af- lar to that provided by key people. Expertise recommenda filiated research. Expertise locating systems are not de- tion systems can reduce the load on people in these roles signed to replace people who fill these key organizational and provide alternative recommendations when these peo- roles. Instead, expertise locating systems attempt to de crease workload and support people who have no other Th resents an options. Recommendation systems are collaborative soft the Expertise Recommender system(ER). ER offers several ware that can be applied to expertise locating. This work advances over previously existing systems describes a general recommendation architecture that is grounded in a field study of expertise locating. Our exper- The architecture is open and flexible enough to address tise recommendation system details the work necessary to different organizational environments. If CSCW sys fit expertise recommendation to a work setting. The archi tems, and specifically recommender systems, are si tecture and implementation begin to tease apart the techni ated in their activity, organizationally specific compo cal aspects of providing good recommendations from social nents will be necessary within any general-purpose nd collaborative concerns framework. Recommender systems, to date, have been “ one size fits all Keywords Recommendation systems, collaborative filtering, expert .Organizationally specific implementations are more locators, expertise location, expertise finding, information robust by teasing apart technical aspects of making seeking, CSCW, computer-supported cooperative work. good recommendations from the social and collabora tive aspects of matching individuals. We do this based NTRODUCTION veryday, people face difficult problems that they cannot on findings from an earlier field study solve alone. In these situations the right people are the ones The work proposes an alternative approach to the crea who have the expertise to answer a specific question or in ion and maintenance of user profiles than ratings. This some other way move the problem toward resolution approach uses organizationally relevant data sources to In organizational settings, people fill key roles that assist in create profiles that are more suited for automated solving problems or initiating important collaborations pertise location. Managers, senior employees, gurus, information mediators Each of these claims will be covered below. [4], and expertise concierges [13] are sought because of We begin this paper with an overview of prior systems that their ability to solve problems directly or make well in- support expertise locating. Next, we consider what it means ned referrals. The ability to fill this role makes these to seek expertise socially, in order to firmly ground the individuals valuable to the organization and makes it ap- requirements of our system. We follow with a description parent when they are unavailable of our system, Expertise Recommender(Er). A small sce nario demonstrates how a user interacts with our system. Our last sections include the details of the architecture and Permission to make digital or hard are not made or distributed for profit or commercial advantage and that EXPERTISE LOCATING SYSTEMS AND CSCW copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers CSCW and adjacent literatures provide examples of sys ion and/or a fee tems that help people find others with suitable expertise. CSCW00, December 2-6, 2000, Philadelphia, PA Systems that assist with expertise location are similar to a Copyright2000ACM1-58113-222000012S5.00. broad class of systems known as recommendation systems

Expertise Recommender: A Flexible Recommendation System and Architecture David W. McDonald and Mark S. Ackerman Department of Information and Computer Science University of California, Irvine Irvine, CA 92697-3425 {dmcdonal, ackerman}@ics.uci.edu ABSTRACT Locating the expertise necessary to solve difficult problems is a nuanced social and collaborative problem. In organiza￾tions, some people assist others in locating expertise by making referrals. People who make referrals fill key organ￾izational roles that have been identified by CSCW and af￾filiated research. Expertise locating systems are not de￾signed to replace people who fill these key organizational roles. Instead, expertise locating systems attempt to de￾crease workload and support people who have no other options. Recommendation systems are collaborative soft￾ware that can be applied to expertise locating. This work describes a general recommendation architecture that is grounded in a field study of expertise locating. Our exper￾tise recommendation system details the work necessary to fit expertise recommendation to a work setting. The archi￾tecture and implementation begin to tease apart the techni￾cal aspects of providing good recommendations from social and collaborative concerns. Keywords Recommendation systems, collaborative filtering, expert locators, expertise location, expertise finding, information seeking, CSCW, computer-supported cooperative work. INTRODUCTION Everyday, people face difficult problems that they cannot solve alone. In these situations the right people are the ones who have the expertise to answer a specific question or in some other way move the problem toward resolution. In organizational settings, people fill key roles that assist in solving problems or initiating important collaborations. Managers, senior employees, gurus, information mediators [4], and expertise concierges [13] are sought because of their ability to solve problems directly or make well in￾formed referrals. The ability to fill this role makes these individuals valuable to the organization and makes it ap￾parent when they are unavailable. Recommendation systems are one possible technology that can augment and assist the natural expertise locating behav￾ior in organizations. A recommendation system that sug￾gests people who have some expertise with a problem holds the promise to provide, in a very small way, a service simi￾lar to that provided by key people. Expertise recommenda￾tion systems can reduce the load on people in these roles and provide alternative recommendations when these peo￾ple are unavailable. This work presents an architecture and implementation of the Expertise Recommender system (ER). ER offers several advances over previously existing systems: • The architecture is open and flexible enough to address different organizational environments. If CSCW sys￾tems, and specifically recommender systems, are situ￾ated in their activity, organizationally specific compo￾nents will be necessary within any general-purpose framework. Recommender systems, to date, have been “one size fits all”. • Organizationally specific implementations are more robust by teasing apart technical aspects of making good recommendations from the social and collabora￾tive aspects of matching individuals. We do this based on findings from an earlier field study. • The work proposes an alternative approach to the crea￾tion and maintenance of user profiles than ratings. This approach uses organizationally relevant data sources to create profiles that are more suited for automated ex￾pertise location. Each of these claims will be covered below. We begin this paper with an overview of prior systems that support expertise locating. Next, we consider what it means to seek expertise socially, in order to firmly ground the requirements of our system. We follow with a description of our system, Expertise Recommender (ER). A small sce￾nario demonstrates how a user interacts with our system. Our last sections include the details of the architecture and of an implementation. EXPERTISE LOCATING SYSTEMS AND CSCW CSCW and adjacent literatures provide examples of sys￾tems that help people find others with suitable expertise. Systems that assist with expertise location are similar to a broad class of systems known as recommendation systems. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. CSCW’00, December 2-6, 2000, Philadelphia, PA. Copyright 2000 ACM 1-58113-222-0/00/0012…$5.00. 231

Recommendation systems are commonly used to recom- rithms, implement a single collaborative model that can be nend documents based on user preferences [17]. This defi- described as Birds of a Feather Flock Together(BOF) nition distinguishes recommendation systems from systems These systems tightly couple the architecture and the col that are more properly characterized as information re- laborative model. trieval (IR)or information filtering In the case of PHOAKS, Referral Web, and active collabo- There are a wide variety of recommender systems. In that rative filtering [12], different algorithms and different col context, the term"documents"should be broadly con- laborative models were chosen because the BOF model and strued. Recommendation systems have been used to rec- clustering algorithms did not effectively fit the recommen- ommend a variety of things including Usenet news [11], dation situation. In expertise recommendation, a BOF web pages [2, 91, videos [8], movies, music [18] and books. model that clusters profiles may not effectively distinguish Some systems refer a user to other people, which is essen- individuals who have expertise from those who have only a tially making recommendations about people Who Knows small amount of knowledge. Alternatively, a BOF model and Referral Web are typical examples of systems that that was effective at clustering the profiles of individuals make referrals. Who Knows [6, 20] found people whe who have expertise would likely not be effective at distin- knew something about a topic, based on a profile con- guishing in which topics their expertise lay structed from exemplary work documents. In a sense, This previous work suggests two critical improvements documents were seen as surrogates for people's interests in First, any expertise recommendation solution should sup a twist on standard IR models. On the other hand, Referral port a range of collaborative recommendation models and Web [10] used social networks to assist expertise location. behaviors. Below, we will show the social requirements for Referral Web identified relevant individuals by their par- this, but technically, an open architecture would permit a icipation in co-authoring relationships and presented users range of potential solutions. Second, one of the most diffi with a chain of relationships that needed to be traversed cult and perhaps most interesting problems from a CSCW from the person seeking expertise to the person who might perspective is how to begin untying the technical aspects of have the desired knowledge. Yenta [5] is a hybrid of these making recommendations from the social and collaborative two models, using profiles constructed from personal data aspects of making a recommendation to a specific user. s well as routing messages along a network of inferred Current recommendation approaches, especially BOF ap- shared interests proaches, do not provide this flexi In our view, recommendation systems are not completely The architecture that we will present is capable of support synonymous with collaborative filtering (CF) systems. ing a range of collaborative recommendation models. An Many CF systems rely on explicit statements of user opin- organizationally specific implementation will demonstrate ion, such as ratings, to create user profiles By relying on the flexibility in the architecture ratings, CF systems often have difficulty generating the However befor ore turning to the architecture and the subse initial user profile and, as the profiles develop, must deal quent implementation we must describe what people want with a sparseness of ratings relative to the total number of when they seek other people for their expertise. We need to items. These are two active areas of cf research show how the recommendation of expertise is in the nu- A few successful recommendation systems rely on implicit anced details of collaborative interaction. These details are opinions, rather than explicit ratings. Tapestry [7] the basis for our approach to the technical work. GroupLens [16] and PHOAKs [9, 21] have all used indi- rect activity. Tapestry made recommendations based on Our prior work [13] describes a field study to examine ex- who read or responded to a particular bulletin board mes- pertise location. The field study was motivated by two de sage. One version of Group Lens used time-spent-reading a frequency-of-mention in a stream of discussion as a type of ing of the intricacies of expertise and expertise locating in a voting mechanism for web pages. Explicit rating schemes rich work setting. Secondly, we hoped to learn how exper are unlikely to work for expertise recommendation, because augment the natural expertise locating behavior that people people are reticent to explicitly state opinions of co-workers with out an appropriate context. Ratings may not sufficiently informative or have not always been ori- not work, but it might be possible to collect feedback. ented toward the objective of informing a system architec- CF systems have another weakness for expertise recom- ture and implementation. In short, we hoped to firmly situ- mendation. Many recommendation systems have very spe- ate systems development in a complex nuanced social set cific architectures that are tailored to recommend their spe. ting as well as implement something useful to a group of cific type of artifact. As well, these architectures commonly participants implement a single clustering algorithm(e. g. Pearson Cor- relation, Single-Value Decomposition, Bayesian classifier The following briefly recounts our field study and the re- for all participants and all artifacts. These clustering algo- sults [13] rithms, also called neighborhood-based predictive algo- 232

Recommendation systems are commonly used to recom￾mend documents based on user preferences [17]. This defi￾nition distinguishes recommendation systems from systems that are more properly characterized as information re￾trieval (IR) or information filtering. There are a wide variety of recommender systems. In that context, the term “documents” should be broadly con￾strued. Recommendation systems have been used to rec￾ommend a variety of things including Usenet news [11], web pages [2, 9], videos [8], movies, music [18] and books. Some systems refer a user to other people, which is essen￾tially making recommendations about people. Who Knows and Referral Web are typical examples of systems that make referrals. Who Knows [6, 20] found people who knew something about a topic, based on a profile con￾structed from exemplary work documents. In a sense, documents were seen as surrogates for people’s interests in a twist on standard IR models. On the other hand, Referral Web [10] used social networks to assist expertise location. Referral Web identified relevant individuals by their par￾ticipation in co-authoring relationships and presented users with a chain of relationships that needed to be traversed from the person seeking expertise to the person who might have the desired knowledge. Yenta [5] is a hybrid of these two models, using profiles constructed from personal data as well as routing messages along a network of inferred shared interests. In our view, recommendation systems are not completely synonymous with collaborative filtering (CF) systems. Many CF systems rely on explicit statements of user opin￾ion, such as ratings, to create user profiles. By relying on ratings, CF systems often have difficulty generating the initial user profile and, as the profiles develop, must deal with a sparseness of ratings relative to the total number of items. These are two active areas of CF research. A few successful recommendation systems rely on implicit opinions, rather than explicit ratings. Tapestry [7], GroupLens [16] and PHOAKS [9, 21] have all used indi￾rect activity. Tapestry made recommendations based on who read or responded to a particular bulletin board mes￾sage. One version of GroupLens used time-spent-reading a message as an implicit opinion. And PHOAKS relies on frequency-of-mention in a stream of discussion as a type of voting mechanism for web pages. Explicit rating schemes are unlikely to work for expertise recommendation, because people are reticent to explicitly state opinions of co-workers with out an appropriate context. Ratings may not work, but it might be possible to collect feedback. CF systems have another weakness for expertise recom￾mendation. Many recommendation systems have very spe￾cific architectures that are tailored to recommend their spe￾cific type of artifact. As well, these architectures commonly implement a single clustering algorithm (e.g. Pearson Cor￾relation, Single-Value Decomposition, Bayesian classifier) for all participants and all artifacts. These clustering algo￾rithms, also called neighborhood-based predictive algo￾rithms, implement a single collaborative model that can be described as Birds of a Feather Flock Together (BOF). These systems tightly couple the architecture and the col￾laborative model. In the case of PHOAKS, Referral Web, and active collabo￾rative filtering [12], different algorithms and different col￾laborative models were chosen because the BOF model and clustering algorithms did not effectively fit the recommen￾dation situation. In expertise recommendation, a BOF model that clusters profiles may not effectively distinguish individuals who have expertise from those who have only a small amount of knowledge. Alternatively, a BOF model that was effective at clustering the profiles of individuals who have expertise would likely not be effective at distin￾guishing in which topics their expertise lay. This previous work suggests two critical improvements. First, any expertise recommendation solution should sup￾port a range of collaborative recommendation models and behaviors. Below, we will show the social requirements for this, but technically, an open architecture would permit a range of potential solutions. Second, one of the most diffi￾cult and perhaps most interesting problems from a CSCW perspective is how to begin untying the technical aspects of making recommendations from the social and collaborative aspects of making a recommendation to a specific user. Current recommendation approaches, especially BOF ap￾proaches, do not provide this flexibility. The architecture that we will present is capable of support￾ing a range of collaborative recommendation models. An organizationally specific implementation will demonstrate the flexibility in the architecture. However, before turning to the architecture and the subse￾quent implementation we must describe what people want when they seek other people for their expertise. We need to show how the recommendation of expertise is in the nu￾anced details of collaborative interaction. These details are the basis for our approach to the technical work. GROUNDED REQUIREMENTS Our prior work [13] describes a field study to examine ex￾pertise location. The field study was motivated by two de￾sires. First, we hoped to contribute to a better understand￾ing of the intricacies of expertise and expertise locating in a rich work setting. Secondly, we hoped to learn how exper￾tise locating systems could be improved to better assist and augment the natural expertise locating behavior that people exhibit. The issues and findings raised in the literature were not sufficiently informative or have not always been ori￾ented toward the objective of informing a system architec￾ture and implementation. In short, we hoped to firmly situ￾ate systems development in a complex nuanced social set￾ting as well as implement something useful to a group of participants. The following briefly recounts our field study and the re￾sults [13]. 232

The MSC Field Study status and can then constrain who is pursued for expertise. The field site was a medium sized software company that Lastly, Orr's [ 14] work with repair technicians reveals how builds, sells, and supports medical and dental practice man- narrative exchange(war stories)serves to demonstrate ex agement software. Medical Software Corporation(MSC) pertise and notions of competency which then guide re has about 100 employees at its headquarters where the quests for assistance with difficult problems study took place. The participants in the study worked in Escalation is the mechanism that fixes breakdowns in iden- Communications. These three departments are central to the development and support of MSC's products. These identification and selection because it requires fine judge ments often based on incomplete and imperfect informa- departments comprise a little more than one third of the tion. Escalation is an iteration of identification and selec- employees at headquarters tion given additional knowledge and information gained The following findings are based on an initial 5 months in from prior iterations. During escalation the individual who the field. Data was collected using qualitative methods in- initially engaged the problem maintains problem owner cluding participant observation, semi-structured formal ship. Escalation results in a wider, spreading activation interviews and informal interviews. The data was analyzed through the organization that involves more people in the sing standard ethnographic techniques eventual solution to a difficult problem. Our study found that people at MSC commonly engage in The architecture that we describe later in this paper pro- three behaviors that support expertise locating: expertise vides specific support for this model of expertise location identification, expertise selection, and escalation. We note An implementation of this architecture, however, requires that identification, selection, and escalation are analytical specifics of how people perform these behaviors. The next distinctions useful for understanding the fine social and section describes two specific identification heuristics that cognitive decisions which are made during expertise locat were used by participants at MSC ing and that will later be useful for technical design. It is Expertise Locating Heuristics clear that only the most self-reflective of the participants The MSC field study provides a detailed understanding of made much if any distinction among these three behaviors the following two identification heuristics. We found this Expertise identification is the problem of finding a set of understanding essential to building a nuanced system. candidates who are likely to have the desired expertise. These heuristics, Change History and Tech Support, are Expertise identification relies on prior experience with oth behaviors and rules about interpreting implied meaning ers, key people like an expertise concierge, and historica using specific historical artifacts at MSC when identifying artifacts. The expertise concierge routes people to others potential expertise at MSC. These historical artifacts have with the necessary knowledge and expertise. The role has many, potentially crucial, features, but the participants at much in common with Ehrlich and Cash's information me- tend to only a few through specific rules of thumb. The diator [4],Paepke's contact broker [15], and Allen's tech- details are quite important to the behavior of the partici ological gatekeeper [l]. Each of these roles is specialized pants and the eventual behavior of a system to the organizational context. Historical artifacts are prod ucts and byproducts of the work that other MSC employees Change History Heuristic have produced. The composition and form of historical Change History is a single heuristic designed to augment an artifacts range from unstructured to highly structured things expertise locating behavior common to MSC's developers that can exist on-line or off-line Expertise selection is the way participants picked one per The"Line 10 Rule"is a function of the organization that son (or a small number of people)to approach for help MSC and the work practices that are enforced there. The Expertise selection is guided by organizational criteria, the developers follow a common work practice. For each soft ad on the potential source, and how a source reacts and ware change, they check the appropriate module out of the interacts during a request for help. Social, organizational, version control system, make the changes, test the and individual preference assist in narrowing a set of iden and then check the changes back into the version tified candidates to one or a small number of people who system. Each change is annotated with the module name, can be contacted the module version, the developer responsible for the ge, the check-in date, and In some cases, prior research has intertwined elements of the change. Since many changes are the result of a contrac expertise identification and selection. Allens[l] work with tual obligation, an administrative assistant cross-validates engineers demonstrated that information seeking is heavily the changes in the version control system, the work-orders or uenced by social networks. Cicourel [3] observed that that divide up multiple changes, the time spent on changes ganizational and institutional factors reinforce expertise and the total time and accounts for the contract Developers at MsC do not specialize in any part of the sys Names and individual identifiers have been changed to tem. In practice, however, there is some attempt to assign a protect the privacy of the corporation and the participants developer work in an area of the code where she ha

The MSC Field Study The field site was a medium sized software company that builds, sells, and supports medical and dental practice man￾agement software. Medical Software Corporation1 (MSC) has about 100 employees at its headquarters where the study took place. The participants in the study worked in Technical Development, Technical Support and Technical Communications. These three departments are central to the development and support of MSC’s products. These departments comprise a little more than one third of the employees at headquarters. The following findings are based on an initial 5 months in the field. Data was collected using qualitative methods in￾cluding participant observation, semi-structured formal interviews and informal interviews. The data was analyzed using standard ethnographic techniques. Our study found that people at MSC commonly engage in three behaviors that support expertise locating: expertise identification, expertise selection, and escalation. We note that identification, selection, and escalation are analytical distinctions useful for understanding the fine social and cognitive decisions which are made during expertise locat￾ing and that will later be useful for technical design. It is clear that only the most self-reflective of the participants made much if any distinction among these three behaviors. Expertise identification is the problem of finding a set of candidates who are likely to have the desired expertise. Expertise identification relies on prior experience with oth￾ers, key people like an expertise concierge, and historical artifacts. The expertise concierge routes people to others with the necessary knowledge and expertise. The role has much in common with Ehrlich and Cash’s information me￾diator [4], Paepke’s contact broker [15], and Allen’s tech￾nological gatekeeper [1]. Each of these roles is specialized to the organizational context. Historical artifacts are prod￾ucts and byproducts of the work that other MSC employees have produced. The composition and form of historical artifacts range from unstructured to highly structured things that can exist on-line or off-line. Expertise selection is the way participants picked one per￾son (or a small number of people) to approach for help. Expertise selection is guided by organizational criteria, the load on the potential source, and how a source reacts and interacts during a request for help. Social, organizational, and individual preference assist in narrowing a set of iden￾tified candidates to one or a small number of people who can be contacted. In some cases, prior research has intertwined elements of expertise identification and selection. Allen’s [1] work with engineers demonstrated that information seeking is heavily influenced by social networks. Cicourel [3] observed that organizational and institutional factors reinforce expertise 1 Names and individual identifiers have been changed to protect the privacy of the corporation and the participants. status and can then constrain who is pursued for expertise. Lastly, Orr’s [14] work with repair technicians reveals how narrative exchange (war stories) serves to demonstrate ex￾pertise and notions of competency which then guide re￾quests for assistance with difficult problems. Escalation is the mechanism that fixes breakdowns in iden￾tification and selection. Participants had difficulty with identification and selection because it requires fine judge￾ments often based on incomplete and imperfect informa￾tion. Escalation is an iteration of identification and selec￾tion given additional knowledge and information gained from prior iterations. During escalation the individual who initially engaged the problem maintains problem owner￾ship. Escalation results in a wider, spreading activation through the organization that involves more people in the eventual solution to a difficult problem. The architecture that we describe later in this paper pro￾vides specific support for this model of expertise location. An implementation of this architecture, however, requires specifics of how people perform these behaviors. The next section describes two specific identification heuristics that were used by participants at MSC. Expertise Locating Heuristics The MSC field study provides a detailed understanding of the following two identification heuristics. We found this understanding essential to building a nuanced system. These heuristics, Change History and Tech Support, are behaviors and rules about interpreting implied meaning using specific historical artifacts at MSC when identifying potential expertise at MSC. These historical artifacts have many, potentially crucial, features, but the participants at￾tend to only a few through specific rules of thumb. The details are quite important to the behavior of the partici￾pants and the eventual behavior of a system. Change History Heuristic Change History is a single heuristic designed to augment an expertise locating behavior common to MSC’s developers. Developers called this behavior the “Line 10 Rule.” The “Line 10 Rule” is a function of the organization that is MSC and the work practices that are enforced there. The developers follow a common work practice. For each soft￾ware change, they check the appropriate module out of the version control system, make the changes, test the changes, and then check the changes back into the version control system. Each change is annotated with the module name, the module version, the developer responsible for the change, the check-in date, and a short text description of the change. Since many changes are the result of a contrac￾tual obligation, an administrative assistant cross-validates the changes in the version control system, the work-orders that divide up multiple changes, the time spent on changes and the total time and accounts for the contract. Developers at MSC do not specialize in any part of the sys￾tem. In practice, however, there is some attempt to assign a developer work in an area of the code where she has 233

worked in the past. Yet, the expanse of the system, the Detailed heuristics and social interaction guide an ex number of developers, and developer turnover will result in pertise seeker to identify candidates who are likely to a developer being assigned work in portions of the code have the required expertise where she has not worked before. Developers use the"Line 10 Rule"in an attempt to overcome the problems that arise Social and organizational norms and factors guide an when working with unfamiliar code expertise seeker to pick a small number of candidates to pursue for help On the surface the"Line 10 Rule"is a simple heuristic Given a problem with a module a developer looks into the The details matter. The heuristics described here are version control system to see who last modified the code extremely bound to the organizational environment. nd then approaches that person for help. Developers say Systems that augment expertise locating must be capa that this rule works because the person who last made a ble of handling a large number of details that vary hange has the code"freshest "in mind The version control based on the specific context and problem. system is not specifically designed to support this exact The focus of this paper now shifts from the details of the use. As a result, the rule is not strictly followed, but most requirements(as exemplified by MSC) to the technical developers state that the heuristic is"good enough"to often design of the expertise recommendation system. THE EXPERTISE RECOMMENDER (ER) Tech Support Heuristic The Expertise Recommender(Er) is firmly grounded in The tech support identification heuristic is not known by a the findings from the MSC study. Er is really two things at specific name. The heuristic augments a behavior of tech- once, at two different levels of abstraction. At the most nical support representatives when faced with an unfamiliar abstract, ER is an architecture designed to handle a range of difficult support problem. recommendation problems. However, recommendation The heuristic is applied to the support database. The sup- architectures cannot solve specific recommendation prob- port database mediates all work performed by technical lems at specific organizations; only specific implemer support representatives. New problems ("calls")can be tions of an architecture can do so. Therefore, we also show entered by a support rep or by customers via email how our work addresses this issue by implementing a spe- When faced with a difficult problem, a representative will cific instantiation suitable for the expertise locating beha perform multiple, separate queries over the support data ior found in the field study of MSC base using the symptoms, customer, or program module To avoid confusion, for the remainder of this paper the ar- volved. The support rep then scans the records sequen- chitecture will be referred to as ER-Arch and the imple tially looking for similarities between the current problem mentation will be known simply as er and any past problems as returned by the different queries This section first presents a simple usage scenario that pro- In scanning records a support rep looks to identify people vides an overview of a client and how the user interacts who have previously solved similar problems with the system. We follow the scenario with a description The support database is not designed to enable this type of of ER-Arch, and conclude with details of the Msc imple activity. Each query(symptom, customer or program)must mentation of ER. be completed separately, so finding similarities among the A Brief Usage Scenario three primary characteristics is mostly done in the support This scenario provides a picture of the user interaction with representative's head. At MSC support representatives are the system and several key features of an ER client. The assigned to specific customers. This benefits the customer scenario describes a problem that MSC support representa- because a single support representative can remember more tives might encounter in the course of their day-to-day details about the context of a customers installation. A side work with msc customers effect is that, while the database does not facilitate queries e re Madhu is a junior technical support representative. Support tween an MSC customer and the support rep who worl reps like Madhu are assigned to handle all incoming call with them is relatively simple from a specific set of MSC customers. However, today The tech support identification heuristic consists of attach- tion. Madhu receives a call from a customer. PCL. who g the symptoms, customers, and program modules solved problems to the person who solved them. As a sim states that the system is reporting a file error in patient ple rule of thumb this scanning behavior works fairly well demographics. Even as a relatively new support 1 is fairly knowledgeable about patient demographics. How- profitable when applied to more difficult support problernly ever. Madhu is not familiar with the specific customer, PCL, However, the process can be time consuming and is and why that customers system might be reporting an er In summary, prior work and our field work demonstrates ror. tant requirements of expertise locating: Madhu recognizes that he will need help resolving this problem. He launches the ER client, logs into the system and gets the main window(figure 1). The main window

worked in the past. Yet, the expanse of the system, the number of developers, and developer turnover will result in a developer being assigned work in portions of the code where she has not worked before. Developers use the “Line 10 Rule” in an attempt to overcome the problems that arise when working with unfamiliar code. On the surface the “Line 10 Rule” is a simple heuristic. Given a problem with a module a developer looks into the version control system to see who last modified the code and then approaches that person for help. Developers say that this rule works because the person who last made a change has the code “freshest” in mind. The version control system is not specifically designed to support this exact use. As a result, the rule is not strictly followed, but most developers state that the heuristic is “good enough” to often get the help they need. Tech Support Heuristic The tech support identification heuristic is not known by a specific name. The heuristic augments a behavior of tech￾nical support representatives when faced with an unfamiliar or difficult support problem. The heuristic is applied to the support database. The sup￾port database mediates all work performed by technical support representatives. New problems (“calls”) can be entered by a support rep or by customers via email. When faced with a difficult problem, a representative will perform multiple, separate queries over the support data￾base using the symptoms, customer, or program module involved. The support rep then scans the records sequen￾tially looking for similarities between the current problem and any past problems as returned by the different queries. In scanning records a support rep looks to identify people who have previously solved similar problems. The support database is not designed to enable this type of activity. Each query (symptom, customer or program) must be completed separately, so finding similarities among the three primary characteristics is mostly done in the support representative’s head. At MSC support representatives are assigned to specific customers. This benefits the customer because a single support representative can remember more details about the context of a customer’s installation. A side effect is that, while the database does not facilitate queries across the three indices, establishing the relationship be￾tween an MSC customer and the support rep who works with them is relatively simple. The tech support identification heuristic consists of attach￾ing the symptoms, customers, and program modules of solved problems to the person who solved them. As a sim￾ple rule of thumb this scanning behavior works fairly well. However, the process can be time consuming and is only profitable when applied to more difficult support problems. In summary, prior work and our field work demonstrates several important requirements of expertise locating: • Detailed heuristics and social interaction guide an ex￾pertise seeker to identify candidates who are likely to have the required expertise. • Social and organizational norms and factors guide an expertise seeker to pick a small number of candidates to pursue for help. • The details matter. The heuristics described here are extremely bound to the organizational environment. Systems that augment expertise locating must be capa￾ble of handling a large number of details that vary based on the specific context and problem. The focus of this paper now shifts from the details of the requirements (as exemplified by MSC) to the technical design of the expertise recommendation system. THE EXPERTISE RECOMMENDER (ER) The Expertise Recommender (ER) is firmly grounded in the findings from the MSC study. ER is really two things at once, at two different levels of abstraction. At the most abstract, ER is an architecture designed to handle a range of recommendation problems. However, recommendation architectures cannot solve specific recommendation prob￾lems at specific organizations; only specific implementa￾tions of an architecture can do so. Therefore, we also show how our work addresses this issue by implementing a spe￾cific instantiation suitable for the expertise locating behav￾ior found in the field study of MSC. To avoid confusion, for the remainder of this paper the ar￾chitecture will be referred to as ER-Arch and the imple￾mentation will be known simply as ER. This section first presents a simple usage scenario that pro￾vides an overview of a client and how the user interacts with the system. We follow the scenario with a description of ER-Arch, and conclude with details of the MSC imple￾mentation of ER. A Brief Usage Scenario This scenario provides a picture of the user interaction with the system and several key features of an ER client. The scenario describes a problem that MSC support representa￾tives might encounter in the course of their day-to-day work with MSC customers. Madhu is a junior technical support representative. Support reps like Madhu are assigned to handle all incoming calls from a specific set of MSC customers. However, today Madhu is also covering for a support rep who is on vaca￾tion. Madhu receives a call from a customer, PCI, who states that the system is reporting a file error in patient demographics. Even as a relatively new support rep, Madhu is fairly knowledgeable about patient demographics. How￾ever, Madhu is not familiar with the specific customer, PCI, and why that customer’s system might be reporting an er￾ror. Madhu recognizes that he will need help resolving this problem. He launches the ER client, logs into the system and gets the main window (figure 1). The main window 234

contains a menu bar and a list of recent prior requests that dress. Additionally, each pane contains a""Contact "button Madhu made. Prior requests are live elements(save sets) that is active only when the recommended person is logged that can be revisited. When revisiting a prior req quest, the into the er server. The contact button, when active estab- user can review the people who were recommended or es- lishes a synchronous chat with the recommended person. calate the request to gain additional recommendations File Edit Special Help Topic Area: Tech support Request: Io Error 16 in pregram M013 customer PCI social Networ rk Reporting Labels MOS5 M054 Departme (949)82 change History Social Network insurance file maintenance m086 m08 81363 suite 101 x1255 klashneresupportmsc.com escalate Close Figure 1. Expertise Recommender Main Window Figure 3. Recommendation Response(Escalated Madhu has not made any prior requests for this problem, so Madhu s escalated request returns two additional people he will have to make a new request. He chooses"New Re- one from development and one from training. Madhu rec quest” from the“File” menu and the client displays the expertise request dialog(figure 2). Through conversation ognizes that Er has recommended a friend in development with the person reporting the error, Madhu is able to get the as someone who is likely to have expertise. Madhu walks specific error code and determines that the error is gener- over to the development suite to look for his friend ated in a module of the patient demographics system called This brief overview of a user's interaction with ER demon- M013 strates several key aspects of Er: EDts吧R就 The user can pick the heuristic and the associated data Expertise Request: that will Tech support Filter: social Network rror 16 in program M 013 cus The user can choose the mechanism that tailors the Cancel Recommend Recommendations remain live so that they can be Figure 2. Expertise Request Dialog tickly revisited and escalated should the initial Madhu decides that the problem is most closely associated ommendations prove unfruitful. with the"Tech Support"topic area and picks this topic in Moving on from the scenario, we next turn to a description the new request dialog. Madhu chooses the"Social Net- of the ER-Arch and the implementation details of an ER ork filter because he would like the results filtered based system for MSC on the people whom he knows best. The dialog lists the The architecture- ER-Arch available identification heuristics and selection techniques Like many recommendation systems, ER-Arch is server in the "Topic Area"and"Filter"pop-up menus respec- based, and a client is necessary to access the functionality tively. In the"Request"text field, Madhu enters the error ER-Arch can support simple clients, like a Web based in (0 Error 16), the program module (program M013)and terface, or clients tailored to support specific features of the the customer(customer PCD). (These names would be natu- ER server. Regardless of the client, the important architec- ral to a support rep at MSC )Madhu clicks the"Recom- tural details are in mend"button, sending the request to the server cated, the components and interconnections should be as After a short wait, the server responds with a list of re sumed to exist in the ommendations. Madhu quickly recognizes that the first few At a high level, ER-Arch is a pipe and filter architecture people will not work. The first recommendations include 9]. This architecture is widely used in information re- the support rep whose calls Madhu is now covering, as well trieval and information filtering, and it has great utility in as a person from training who is currently off-site. So, expertise recommendation (as will be discussed below) Madhu immediately escalates the request to get additional Accordingly, ER-Arch is a collection of high-level supervi sors, easily extensible heuristic modules, and their data The response dialog displays the request followed by a list stores. The supervisors provide general services and con- of recommended people. Each person is listed in a single nections that facilitate a specific implementation. They also ane containing contact information that includes office coordinate the underlying heuristic modules to provide re number, phone number, phone extension, and email ad- quired services( such as identification)to the system. The databases provide persistent storage for profiles and various 235

contains a menu bar and a list of recent prior requests that Madhu made. Prior requests are live elements (save sets) that can be revisited. When revisiting a prior request, the user can review the people who were recommended or es￾calate the request to gain additional recommendations. Figure 1. Expertise Recommender Main Window Madhu has not made any prior requests for this problem, so he will have to make a new request. He chooses “New Re￾quest” from the “File” menu and the client displays the expertise request dialog (figure 2). Through conversation with the person reporting the error, Madhu is able to get the specific error code and determines that the error is gener￾ated in a module of the patient demographics system called M.013. Figure 2. Expertise Request Dialog Madhu decides that the problem is most closely associated with the “Tech Support” topic area and picks this topic in the new request dialog. Madhu chooses the “Social Net￾work” filter because he would like the results filtered based on the people whom he knows best. The dialog lists the available identification heuristics and selection techniques in the “Topic Area” and “Filter” pop-up menus respec￾tively. In the “Request” text field, Madhu enters the error (I/O Error 16), the program module (program M.013) and the customer (customer PCI). (These names would be natu￾ral to a support rep at MSC.) Madhu clicks the “Recom￾mend” button, sending the request to the server. After a short wait, the server responds with a list of rec￾ommendations. Madhu quickly recognizes that the first few people will not work. The first recommendations include the support rep whose calls Madhu is now covering, as well as a person from training who is currently off-site. So, Madhu immediately escalates the request to get additional recommendations (figure 3). The response dialog displays the request followed by a list of recommended people. Each person is listed in a single pane containing contact information that includes office number, phone number, phone extension, and email ad￾dress. Additionally, each pane contains a “Contact” button that is active only when the recommended person is logged into the ER server. The contact button, when active, estab￾lishes a synchronous chat with the recommended person. Figure 3. Recommendation Response (Escalated) Madhu’s escalated request returns two additional people, one from development and one from training. Madhu rec￾ognizes that ER has recommended a friend in development as someone who is likely to have expertise. Madhu walks over to the development suite to look for his friend. This brief overview of a user’s interaction with ER demon￾strates several key aspects of ER: • The user can pick the heuristic and the associated data that will be used to generate a recommendation. • The user can choose the mechanism that tailors the recommendations to the user. • Recommendations remain live so that they can be quickly revisited and escalated should the initial rec￾ommendations prove unfruitful. Moving on from the scenario, we next turn to a description of the ER-Arch and the implementation details of an ER system for MSC. The Architecture — ER-Arch Like many recommendation systems, ER-Arch is server based, and a client is necessary to access the functionality. ER-Arch can support simple clients, like a Web based in￾terface, or clients tailored to support specific features of the ER server. Regardless of the client, the important architec￾tural details are in the ER server. Unless otherwise indi￾cated, the components and interconnections should be as￾sumed to exist in the server. At a high level, ER-Arch is a pipe and filter architecture [19]. This architecture is widely used in information re￾trieval and information filtering, and it has great utility in expertise recommendation (as will be discussed below). Accordingly, ER-Arch is a collection of high-level supervi￾sors, easily extensible heuristic modules, and their data stores. The supervisors provide general services and con￾nections that facilitate a specific implementation. They also coordinate the underlying heuristic modules to provide re￾quired services (such as identification) to the system. The databases provide persistent storage for profiles and various 235