《电子商务 E-business》阅读文献：User Modeling Adaptation Personalization.pdf_P26-P30

Can Concept-Based User Modeling Improve Adaptive Visualization? r keywords and y NEs. Therefore, k5n5 means 5 keywords and 5 NEs, while k10n0 means 10 keywords only. We chose these combinations considering the optimal number of user model POls displayed in the visualization. Because we didn't want to place too many user model POIs on the screen and make users frustrated, we configured x +y(total number of user model POIs displayed at the same time on the screen) to be no more than 20. Using these combinations of keyword/NE mixtures, we could test various conditions such as equivalent importance(e.g. k5n5), keyword only(e.g. k10n0), and NE only(e. g. kOn10) The NEs were extracted from user feedback information(notes saved by users in our prototype system) just like the case of keywords [1,3. Among the can- didates, NEs with higher TF-IDF value es were selected for constructing the ne based user models. When calculating the TF-IDF values. the ne normalization process introduced in the previous section was utilized. That is, "ski lover"and "who"were recognized as the same terms and counted as TF=2 4 Study Design 4.1 Hypotheses and Measures We defined two hypotheses in this study in order to test the validity of the NE-based adaptive visualization H1) The proposed NE-based adaptive visualization will better separate relevant and non-relevant documents in the visualization H2)In the NE-based adaptive visualization, the relevant documents will be more attracted by the user models fes they were defined considering the nature of an ideal information access sys- n. An ideal information access system has to have the ability to sort out valuable information from noise and to provide such information to users ef- ficiently. The hypotheses exactly reflect those characteristics. Adaptive VIBe aims to distinguish relevant documents and then locate them spatially close to the user models. In order to measure the separation of relevant documents from non-relevant ones, we adopted the Davies-Bouldin Validity Index(DB-index. It determines the quality of clustering by measuring the compactness and separa- tion of the clusters of those two types of documents [6. It is a ratio of the spread of elements in clusters and the distances between those clusters(Equation 1) Therefore, it produces smaller scores as the clusters become compact and as the clusters are far from each other which means better clusterings DB、1 Sn(Qi)+Sn(Q) nai= s(Qi, Q,) S(Q)=average distance within a cluster Q S(Q1, Q2)= distance between two cluster centroids

Can Concept-Based User Modeling Improve Adaptive Visualization? 9 x keywords and y NEs. Therefore, k5n5 means 5 keywords and 5 NEs, while k10n0 means 10 keywords only. We chose these combinations considering the optimal number of user model POIs displayed in the visualization. Because we didn’t want to place too many user model POIs on the screen and make users frustrated, we configured x + y (total number of user model POIs displayed at the same time on the screen) to be no more than 20. Using these combinations of keyword/NE mixtures, we could test various conditions such as equivalent importance (e.g. k5n5), keyword only (e.g. k10n0), and NE only (e.g. k0n10). The NEs were extracted from user feedback information (notes saved by users in our prototype system) just like the case of keywords [1, 3]. Among the candidates, NEs with higher TF-IDF values were selected for constructing the NEbased user models. When calculating the TF-IDF values, the NE normalization process introduced in the previous section was utilized. That is, “ski lover” and “who” were recognized as the same terms and counted as TF=2. 4 Study Design 4.1 Hypotheses and Measures We defined two hypotheses in this study in order to test the validity of the NE-based adaptive visualization. H1) The proposed NE-based adaptive visualization will better separate relevant and non-relevant documents in the visualization. H2) In the NE-based adaptive visualization, the relevant documents will be more attracted by the user models. They were defined considering the nature of an ideal information access system. An ideal information access system has to have the ability to sort out valuable information from noise and to provide such information to users ef- ficiently. The hypotheses exactly reflect those characteristics. Adaptive VIBE aims to distinguish relevant documents and then locate them spatially close to the user models. In order to measure the separation of relevant documents from non-relevant ones, we adopted the Davies-Bouldin Validity Index (DB-index). It determines the quality of clustering by measuring the compactness and separation of the clusters of those two types of documents [6]. It is a ratio of the spread of elements in clusters and the distances between those clusters (Equation 1). Therefore, it produces smaller scores as the clusters become compact and as the clusters are far from each other, which means better clusterings. DB = 1 n n i=1 maxi=j Sn(Qi) + Sn(Qj ) S(Qi, Qj ) (1) S(Q) = average distance within a cluster Q S(Q1, Q2) = distance between two cluster centroids

J -w. Ahn and P. Brusilovsky 4.2 Dataset As mentioned briefly earlier, we constructed a dataset from the log data of our text-based personalized information retrieval study 3. It aimed to help users to search the TDT4 news corpus for information by mediating the user query and the user model with a text-based user interface. The TDT4 corpus was built for constructing a news understanding systems and is comprised of 96, 260 news articles. We chose tDT4 because NEs could represent important concepts appearing in news texts. From the log file of the study, we could extract the 1. userid and query 2. retrieved documents and the relevance of each document 3. user notes - explicit user feedbacks from which user model ke NES would be extracted That is, we had stored a snapshot of every users' search activity, the output from the system, and the user model constructed by the system(or the source of user model). Using this data, we were able to rebuild the user models using keywords and NEs(as shown in the previous section), and then re-situate the Adaptive Vibe visualizations according to each user model. Moreover, we had the relevance information of each document and could observe how they were represented in the visualizations. This relevance information was not available to the users during the user study, but we could take advantage of its availability to evaluate the quality of the adaptive visualizations(as in Fig. 2). The next section shows the analysis result of those adaptive visualizations and discusses their properties. 5 Experimental Results 5.1 Separation of Relevant and Non-relevant Documents by Concept-Based Adaptive Visualization In our previous study, we found that the adaptive visualization was able to produce clusters of relevant and non-relevant documents and that the relevant document cluster was more attracted to the user model side [1. However, the user model of the study made use of keywords only and the power of the user model in the visualization was assumed to be limited. Therefore, we prepared various combinations of user model elements(keywords plus NEs as introduced in Sect. 3)and tested them with our adaptive visualization system. The first step of the analysis was to examine how well the relevant and non-relevant doc ument clusters were formed Using the DB-index, we could calculate the quality of the clusterings. Table 1 and Fig. 3 show the DB-indices of three different POI layouts of Adaptive VIBE using eight different mixtures of keywords and NEs. From this data, it can be easily seen that using only keywords TopicDetectionandtrAckingProject,<http://www.ldc.upennedu/tdt>

10 J.-w. Ahn and P. Brusilovsky 4.2 Dataset As mentioned briefly earlier, we constructed a dataset from the log data of our text-based personalized information retrieval study [3]. It aimed to help users to search the TDT41 news corpus for information by mediating the user query and the user model with a text-based user interface. The TDT4 corpus was built for constructing a news understanding systems and is comprised of 96,260 news articles. We chose TDT4 because NEs could represent important concepts appearing in news texts. From the log file of the study, we could extract the information as below. 1. userid and query 2. retrieved documents and the relevance of each document 3. user notes – explicit user feedbacks from which user model keywords and NEs would be extracted That is, we had stored a snapshot of every users’ search activity, the output from the system, and the user model constructed by the system (or the source of user model). Using this data, we were able to rebuild the user models using keywords and NEs (as shown in the previous section), and then re-situate the Adaptive VIBE visualizations according to each user model. Moreover, we had the relevance information of each document and could observe how they were represented in the visualizations. This relevance information was not available to the users during the user study, but we could take advantage of its availability to evaluate the quality of the adaptive visualizations (as in Fig. 2). The next section shows the analysis result of those adaptive visualizations and discusses their properties. 5 Experimental Results 5.1 Separation of Relevant and Non-relevant Documents by Concept-Based Adaptive Visualization In our previous study, we found that the adaptive visualization was able to produce clusters of relevant and non-relevant documents and that the relevant document cluster was more attracted to the user model side [1]. However, the user model of the study made use of keywords only and the power of the user model in the visualization was assumed to be limited. Therefore, we prepared various combinations of user model elements (keywords plus NEs as introduced in Sect. 3) and tested them with our adaptive visualization system. The first step of the analysis was to examine how well the relevant and non-relevant document clusters were formed. Using the DB-index, we could calculate the quality of the clusterings. Table 1 and Fig. 3 show the DB-indices of three different POI layouts of Adaptive VIBE using eight different mixtures of keywords and NEs. From this data, it can be easily seen that using only keywords or NEs 1 Topic Detection and Tracking Project, <http://www.ldc.upenn.edu/TDT>

J -w. Ahn and P. Brusilovsky Table 2. Comparison of mean DB-index among three mixtures(k10n0, k5n5, kOn10) Kruskal-Wallis x 0.0115 0.1842 Table 3. Pairwise Wilcox signed rank tests by keyword/NE mixture Layout=Parallel klOng p=0.002 kOn10 p<0.001 =0.007 Table 4. Comparing within-cluster spread and between-cluster distance Layout Within-cluster spread kOno k5n5 10n0 kOn10 49.94 Parellel 4.47 152.13 161.44 151.63 138.05 125.31 ere110.54 109.50110.54 74.78 models could produce good results but we needed deeper analysis By separating the nominator and denominator of the DB-index equation, we could compare the within-cluster spreads and between-cluster distances in terms of two other vari- ables: keyword/NE mixture and Adaptive VIBE POI layout(Table 4). It shows that the differences among the keyword /NE mixtures were not evident when w observed the cluster spreads, but that there were bigger differences in terms of the between-cluster distances across the three mixtures. In all cases. the k5n5 mixture showed the largest distance and the differences between other mixtures were always statistically significant(Wilcox signed rank test, p 0.01). Thi result suggests that the significant differences of overall DB-indices among the mixtures(where k5n5 was the best)observed in the previous section were caused by the cluster distance, rather than the different inner-compactness of clusters 5.2 User Model Effects on Adaptive Visualization So far, we have seen that the adaptive visualization could separate the relevant and non-relevant document clusters. It could also work more effectively when the user models were constructed using the mixture of keywords and NEs. However this just tells us that there were separations and cannot let us know what they really looked like. Therefore, the following analysis focused on the distribution of relevant and non-relevant document clusters in the visualization table 5 pares the horizontal positions of the cluster centroids in various conditions Th relevant document clusters were always located closer to the user models (larger in their horizontal positions). Particularly, the distances between the cluster cen- troids were largest when the Parallel layout (which separates the user model and

12 J.-w. Ahn and P. Brusilovsky Table 2. Comparison of mean DB-index among three mixtures (k10n0, k5n5, k0n10) Layout Radial Parallel Hemisphere Kruskal-Wallis χ2 0.0462 8.93 3.3834 p 0.9772 0.0115 0.1842 Table 3. Pairwise Wilcox signed rank tests by keyword/NE mixture Layout=Parallel k5n5 k10n0 k5n5 - p=0.002 k0n10 p < 0.001 p=0.007 Table 4. Comparing within-cluster spread and between-cluster distance Layout Within-cluster spread Between-cluster distance k5n5 k10n0 k0n10 k5n5 k10n0 k0n10 Radial 87.67 81.25 87.67 55.23 49.94 47.68 Parellel 154.47 152.13 161.44 151.63 138.05 125.31 Hemisphere 110.54 109.50 110.54 82.97 70.76 74.78 models could produce good results but we needed deeper analysis. By separating the nominator and denominator of the DB-index equation, we could compare the within-cluster spreads and between-cluster distances in terms of two other variables: keyword/NE mixture and Adaptive VIBE POI layout (Table 4). It shows that the differences among the keyword/NE mixtures were not evident when we observed the cluster spreads, but that there were bigger differences in terms of the between-cluster distances across the three mixtures. In all cases, the k5n5 mixture showed the largest distance and the differences between other mixtures were always statistically significant (Wilcox signed rank test, p < 0.01). This result suggests that the significant differences of overall DB-indices among the mixtures (where k5n5 was the best) observed in the previous section were caused by the cluster distance, rather than the different inner-compactness of clusters. 5.2 User Model Effects on Adaptive Visualization So far, we have seen that the adaptive visualization could separate the relevant and non-relevant document clusters. It could also work more effectively when the user models were constructed using the mixture of keywords and NEs. However, this just tells us that there were separations and cannot let us know what they really looked like. Therefore, the following analysis focused on the distribution of relevant and non-relevant document clusters in the visualization. Table 5 compares the horizontal positions of the cluster centroids in various conditions. The relevant document clusters were always located closer to the user models (larger in their horizontal positions). Particularly, the distances between the cluster centroids were largest when the Parallel layout (which separates the user model and

Can Concept-Based User Modeling Improve Adaptive Visualization? 13 Table 5. Comparing horizontal positions of cluster centroids(in pixels) Keyword/NE Mixture Clusters Radial Parallel 313.58 318.56 Non-relevant 188.19 304.74 klOng 302.99 315.73 332.02 3617 k5n5 301.46 192.71 308.44 14.27 139.31 kOn10 Non-relevant 294.71 161 291.77 Distance 107.80 (p<0.01,“p=0.038) query space the most was used whereas the Radial layout(non-personalized duced very small between-cluster distances. The differences between relevant and non-relevant clusters'horizontal positions(or distances)were all statistically significant(Wilcox signed rank test). This result confirms our second hypoth- esis that the user model attracts more the relevant documents than the query side. We should note that the mixture of five keywords and five NEs shows the biggest distance in the Parallel layout(139.31)and thus supports the validity of concept-based user modeling for adaptive visualization. The mean differences of cluster distances across three mixtures were all statistically significant(Wilcox signed rank test, P < 0.001) 6 Conclusions In this paper, we introduced our innovative approach for adaptive visualization and concept-based user modeling. Adaptive visualization is a promising per- sonalized information access method that can efficiently guide users to relevant information Concept-based user modeling is an alternative to old keyword-based approaches, which can enrich user models by adding more semantics. We inte- grated named-entities into user models and examined the quality of the adaptive visualization method equipped with the concept-based user models. An experiment was conducted using the proposed approach and the result showed that the mixture of key words and NEs provided the best results in terms of separating relevant documents from non-relevant ones. We also discovered that the cluster separation was due more to the between-cluster distances rather than cluster compactness. Moreover, the effect that user models could attract relevant documents around them was seen, which supports the utility of our idea that adaptive visualization can help users to access relevant information more easily. In our future work, we plan to conduct a large-scale user study and exam- ine user behaviors about our adaptive visualization and the concept-based user model. We are going to determine if the systems will work as expected and will observe under what situations users can benefit from the potential of the stems. We are also planning to migrate the adaptive visualization concept into

Can Concept-Based User Modeling Improve Adaptive Visualization? 13 Table 5. Comparing horizontal positions of cluster centroids (in pixels) Keyword/NE Mixture Clusters Radial Parallel Hemisphere k10n0 Relevant 313.58 318.56 350.35 Non-relevant 302.99 188.19 304.74 Distance 10.59∗ 130.37∗ 45.61∗ k5n5 Relevant 315.73 332.02 361.77 Non-relevant 301.46 192.71 308.44 Distance 14.27∗ 139.31∗ 53.33∗ k0n10 Relevant 300.68 269.44 328.31 Non-relevant 294.71 161.63 291.77 Distance 5.96∗∗ 107.80∗ 36.54∗ ( ∗p < 0.01, ∗∗p = 0.038) the query space the most) was used whereas the Radial layout (non-personalized) produced very small between-cluster distances. The differences between relevant and non-relevant clusters’ horizontal positions (or distances) were all statistically significant (Wilcox signed rank test). This result confirms our second hypothesis that the user model attracts more the relevant documents than the query side. We should note that the mixture of five keywords and five NEs shows the biggest distance in the Parallel layout (139.31) and thus supports the validity of concept-based user modeling for adaptive visualization. The mean differences of cluster distances across three mixtures were all statistically significant (Wilcox signed rank test, p < 0.001). 6 Conclusions In this paper, we introduced our innovative approach for adaptive visualization and concept-based user modeling. Adaptive visualization is a promising personalized information access method that can efficiently guide users to relevant information. Concept-based user modeling is an alternative to old keyword-based approaches, which can enrich user models by adding more semantics. We integrated named-entities into user models and examined the quality of the adaptive visualization method equipped with the concept-based user models. An experiment was conducted using the proposed approach and the result showed that the mixture of keywords and NEs provided the best results in terms of separating relevant documents from non-relevant ones. We also discovered that the cluster separation was due more to the between-cluster distances rather than cluster compactness. Moreover, the effect that user models could attract relevant documents around them was seen, which supports the utility of our idea that adaptive visualization can help users to access relevant information more easily. In our future work, we plan to conduct a large-scale user study and examine user behaviors about our adaptive visualization and the concept-based user model. We are going to determine if the systems will work as expected and will observe under what situations users can benefit from the potential of the systems. We are also planning to migrate the adaptive visualization concept into