Recommending related Articles in Wikipedia via a Topic-Based Model Wongkot Sriurai, Phayung Meesad, Choochart Haruechaiyasak Department of Information Technology Faculty of Information Technology King Mongkut's University of Technology North Bangkok(KMUTNB) 1518 Pibulsongkarm Rd, Bangsue, Bangkok 10800 Department of Teacher Training in Electrical En Faculty of Technical Education King Mongkut's University of Technology North Bangkok(KMUTNB) 1518 Pibulsongkarm Rd, Bangsue, Bangkok 10800 Human Language Technology Laboratory(HLT) National Electronics and Computer Technology Center (NECTEC) Thailand Science Park. Pathumthani 12120. Thailand $4970290021@ kmutnb ac th pym@kmutnb ac th choochart. haruechaiyasak( @nectec or th Abstract: Wikipedia is currently the largest encyclopedia publicly available on the b. In addition to keyword search and subject browsing, users may quickly access articles by following hyperlinks embedded within each article. The main drawback of this method is that some links to related articles could be missing from the current article. Also, a related article could not be inserted as a hyperlink if there is no term describing it within the current article. In this paper, we propose an approach for recommending related articles based on the Latent Dirichlet Allocation(LDA)algorithm. By applying the lda on the anchor texts from each article, a set of diverse topics could be generated. An article can be represented as a probability distribution over this topic set. Two articles with similar topi stributions are considered conceptually related. We performed an experiment on the wikipedia Selection for Schools which is a collection of 4, 625 selected articles from the wikipedia. Based on some initial evaluation, our proposed method could generate a set of recommended articles which are more relevant than the linked rticles given on the test articles. 194
Recommending Related Articles in Wikipedia via a Topic-Based Model Wongkot Sriurai, Phayung Meesad, Choochart Haruechaiyasak Department of Information Technology Faculty of Information Technology King Mongkut's University of Technology North Bangkok (KMUTNB) 1518 Pibulsongkarm Rd., Bangsue, Bangkok 10800 Department of Teacher Training in Electrical Engineering Faculty of Technical Education King Mongkut's University of Technology North Bangkok (KMUTNB) 1518 Pibulsongkarm Rd., Bangsue, Bangkok 10800 Human Language Technology Laboratory (HLT) National Electronics and Computer Technology Center (NECTEC) Thailand Science Park, Pathumthani 12120, Thailand s4970290021@ kmutnb.ac.th pym@kmutnb.ac.th choochart.haruechaiyasak@nectec.or.th Abstract: Wikipedia is currently the largest encyclopedia publicly available on the Web. In addition to keyword search and subject browsing, users may quickly access articles by following hyperlinks embedded within each article. The main drawback of this method is that some links to related articles could be missing from the current article. Also, a related article could not be inserted as a hyperlink if there is no term describing it within the current article. In this paper, we propose an approach for recommending related articles based on the Latent Dirichlet Allocation (LDA) algorithm. By applying the LDA on the anchor texts from each article, a set of diverse topics could be generated. An article can be represented as a probability distribution over this topic set. Two articles with similar topic distributions are considered conceptually related. We performed an experiment on the Wikipedia Selection for Schools which is a collection of 4,625 selected articles from the Wikipedia. Based on some initial evaluation, our proposed method could generate a set of recommended articles which are more relevant than the linked articles given on the test articles. 194
1 Introduction L, Ikipedia is a well-known free-content encyclopedia written collaboratively by lunteers and sponsored by the non-profit wikipedia Foundation.The aim of the project is to develop a free encyclopedia for many different languages. At present, there are over 2, 400,000 articles available in English and many in other languages. The full volume of wikipedia contents, however, contains some articles which are unsuitable for children. In May 2007, the SOs Childrens Villages, the world,'s largest orphan charity, launched the Wikipedia Selection for Schools. The collection contains 4, 625 selected articles based on the uk national curriculum and similar curricula elsewhere in the world. All articles in the collection have been cleaned up and checked for suitability for children The content of wikipedia for Schools can be navigated by browsing on a pictorial subject index or a title word index of all topics. Table 1 lists the first-level subject ategories available from the collection. Organizing articles into the subject category set provides users a convenient way to access the articles on the same subject. Each article contains many hypertext links to other articles which are related to the current article. However, the links which were assigned by the authors of the article cannot fully cover all related articles. One of the reasons is due to the fact that there is no term describing related articles within the current article Table 1: The subject categories under the Wikipedia Selection for Schools. Articles Business studies Citizenship Countries Design and Technology 250 Everyday life tory Language and literature Mathematics People Religion Science Wikipediahttp://en.wikipedia.org/wiki/wikipedia wikipediaSelectionforSchoolshttp://schools-wikipedia.org
1 Introduction Wikipedia is a well-known free-content encyclopedia written collaboratively by volunteers and sponsored by the non-profit Wikipedia Foundation1 .The aim of the project is to develop a free encyclopedia for many different languages. At present, there are over 2,400,000 articles available in English and many in other languages. The full volume of Wikipedia contents, however, contains some articles which are unsuitable for children. In May 2007, the SOS Children's Villages, the world's largest orphan charity, launched the Wikipedia Selection for Schools2 . The collection contains 4,625 selected articles based on the UK National Curriculum and similar curricula elsewhere in the world. All articles in the collection have been cleaned up and checked for suitability for children. The content of Wikipedia for Schools can be navigated by browsing on a pictorial subject index or a title word index of all topics. Table 1 lists the first-level subject categories available from the collection. Organizing articles into the subject category set provides users a convenient way to access the articles on the same subject. Each article contains many hypertext links to other articles which are related to the current article. However, the links which were assigned by the authors of the article cannot fully cover all related articles. One of the reasons is due to the fact that there is no term describing related articles within the current article. Table 1: The subject categories under the Wikipedia Selection for Schools. Category Articles Category Articles Art 74 Business Studies 88 Citizenship 224 Countries 220 Design and Technology 250 Everyday life 380 Geography 650 History 400 IT 64 Language and literature 196 Mathematics 45 Music 140 People 680 Religion 146 Science 1068 1Wikipedia. http://en.wikipedia.org/wiki/WikiPedia 2Wikipedia Selection for Schools. http://schools-wikipedia.org 195
Some previous works have identified this problem as the missing link problem and also proposed some methods for automatically generating links to related articles. J. Voss [Vo05] presented an analysis of wikipedia snapshot on March 2005. The study showed that Wikipedia links form a scale-free network and the distribution of in-degree and out degree of wikipedia pages follows a power law. S. Fissaha Adafre and M. de rijke [FROS] presented an automated approach in finding related pages by exploring potential links in a wiki page. They proposed a method of discovering missing links in Wikipedia pages via a clustering approach. The clustering process is performed by grouping topically related pages using LTRank and then performing identification of link candidates by matching the anchor texts. Cosley et al. [Co07] presented Suggest Bot, software that performs intelligent task routing (matching people with tasks)in Wikipedia. Suggest Bot uses broadly applicable strategies of text analysis, collaborative filtering, and hyperlink following to recommend tasks In this paper, we propose a method for recommending related articles in wikipedia based on the Latent Dirichlet Allocation(LDA)algorithm. We adopt the dot product omputation for calculating the similarity between two topic distributions which represent two different articles. Using the proposed approach, we can find the relation between two articles and use this relation to recommend links for each article. The rest of paper is organized as follows. In Section 2, we describe the topic-based mode for article recommendation. Section 3 presents experiments and discussion. Finally, we conclude our work and put forward the directions of our future work in Section 4 2 The Topic-Based Model for Article Recommendation There have been many studies on discovering latent topics from text collections [SG06 Latent Semantic Analysis(LSA)uses singular value decomposition(SVD) to map high- dimensional term-by-document matrix to a lower dimensional representation called latent semantic space ] However, SVD is actually designed for normally distributed data. Such a distribution is inappropriate for count data which is what a term- by-document matrix consists of. LSA has been applied to a wide variety of learning tasks, such as search and retrieval [De90] and classification [Bi08]. Although LSa have achieved important success but Lsa have some drawbacks such as overfitting and inappropriate generative semantics [BNJ031
Some previous works have identified this problem as the missing link problem and also proposed some methods for automatically generating links to related articles. J. Voss [Vo05] presented an analysis of Wikipedia snapshot on March 2005. The study showed that Wikipedia links form a scale-free network and the distribution of in-degree and outdegree of Wikipedia pages follows a power law. S. Fissaha Adafre and M. de Rijke [FR05] presented an automated approach in finding related pages by exploring potential links in a wiki page. They proposed a method of discovering missing links in Wikipedia pages via a clustering approach.The clustering process is performed by grouping topically related pages using LTRank and then performing identification of link candidates by matching the anchor texts. Cosley et al. [Co07] presented SuggestBot, software that performs intelligent task routing (matching people with tasks) in Wikipedia. SuggestBot uses broadly applicable strategies of text analysis, collaborative filtering, and hyperlink following to recommend tasks. In this paper, we propose a method for recommending related articles in Wikipedia based on the Latent Dirichlet Allocation (LDA) algorithm. We adopt the dot product computation for calculating the similarity between two topic distributions which represent two different articles. Using the proposed approach, we can find the relation between two articles and use this relation to recommend links for each article. The rest of paper is organized as follows. In Section 2, we describe the topic-based mode for article recommendation. Section 3 presents experiments and discussion. Finally, we conclude our work and put forward the directions of our future work in Section 4. 2 The Topic-Based Model for Article Recommendation There have been many studies on discovering latent topics from text collections [SG06]. Latent Semantic Analysis (LSA) uses singular value decomposition (SVD) to map highdimensional term-by-document matrix to a lower dimensional representation called latent semantic space [De90]. However, SVD is actually designed for normallydistributed data. Such a distribution is inappropriate for count data which is what a termby-document matrix consists of. LSA has been applied to a wide variety of learning tasks, such as search and retrieval [De90] and classification [Bi08]. Although LSA have achieved important success but LSA have some drawbacks such as overfitting and inappropriate generative semantics [BNJ03]. 196
Due to the drawbacks of the LSA, the Latent Dirichlet Allocation (LDA)has been introduced as a generative probabilistic model for a set of documents [BNJ03]. The basic idea behind this approach is that documents are represented as random mixtures over latent topics. Each topic is represented by a probability distribution over the terms. Each article is represented by a probability distribution over the topics. LDA has also been applied for identification of topics in a number of different areas. For example, LDa has been used to find scientific topics from abstracts of papers published in the proceedings of the national academy of sciences [GS04]. McCallum et al. [MCO5] proposed an LDA- based approach to extract topics from social networks and applied it to a collection of 250,000 Enron emails. Newman et al.(2006)applied LDa to derive 400 topics such as Basketball, Harry Potter and Holidays from a corpus of 330,000 New York Times news articles and represent each news article as a mixture of these topics [Ne06] Haruechaiyasak and Damrongrat [HDo8] applied the LDa algorithm for recommending elated articles in Wikipedia Selection for Schools, however, without providing any comparative evaluation 6 Figure 1: The Latent Dirichlet Allocation(LDA)model Generally, an LDA model can be represented as a probabilistic graphical model as shown in Figure 2 [BNJ03]. There are three levels to the LDA representation. The variables a and P are the corpus-level parameters, which are assumed to be sampled during the process of generating a corpus. u is the parameter of the uniform Dirichlet prior on the per-document topic distributions. b is the parameter of the uniform Dirichlet prior on the per-topic word distribution. 8 is a document-level variable, sampled once per document. Finally, the variables z and w are word-level variables and are sampled once for each word in each document. The variable N is the number of word tokens in a document and variable m is the number of documents The LDA model [BNJ03] introduces a set of K latent variables, called topics. Each word in the document is assumed to be generated by one of the topics. The generative process for each document w can be described as follows
Due to the drawbacks of the LSA, the Latent Dirichlet Allocation (LDA) has been introduced as a generative probabilistic model for a set of documents [BNJ03]. The basic idea behind this approach is that documents are represented as random mixtures over latent topics. Each topic is represented by a probability distribution over the terms. Each article is represented by a probability distribution over the topics. LDA has also been applied for identification of topics in a number of different areas. For example, LDA has been used to find scientific topics from abstracts of papers published in the proceedings of the national academy of sciences [GS04]. McCallum et al. [MC05] proposed an LDAbased approach to extract topics from social networks and applied it to a collection of 250,000 Enron emails. Newman et al. (2006) applied LDA to derive 400 topics such as Basketball, Harry Potter and Holidays from a corpus of 330,000 New York Times news articles and represent each news article as a mixture of these topics [Ne06]. Haruechaiyasak and Damrongrat [HD08] applied the LDA algorithm for recommending related articles in Wikipedia Selection for Schools, however, without providing any comparative evaluation. Figure 1: The Latent Dirichlet Allocation (LDA) model Generally, an LDA model can be represented as a probabilistic graphical model as shown in Figure 2 [BNJ03]. There are three levels to the LDA representation. The variables α and β are the corpus-level parameters, which are assumed to be sampled during the process of generating a corpus. α is the parameter of the uniform Dirichlet prior on the per-document topic distributions. β is the parameter of the uniform Dirichlet prior on the per-topic word distribution. θ is a document-level variable, sampled once per document. Finally, the variables z and w are word-level variables and are sampled once for each word in each document. The variable N is the number of word tokens in a document and variable M is the number of documents. The LDA model [BNJ03] introduces a set of K latent variables, called topics. Each word in the document is assumed to be generated by one of the topics. The generative process for each document w can be described as follows: 197
1. Choose 0-Dir(a) Choose a latent topics mixture vector 0 from the Dirichlet distribution 2. For each word w∈w (a) Choose a topic En -Multinomial(0): Choose a latent topic E,from the multinomial distribution (b) Choose a word W, from p(wnl-n B) a multinomial probability In this paper, we focus on the wikipedia Selection for schools for evaluating proposed recommendation algorithm. Our proposed approach based on the topic me for recommending related articles and discovering missing links consists of three processes as shown in Figure 2. Extracto LOA Dot Product Sorting Anchor Texts Article Topics Figure 2: The proposed topic-based model via LDa algorithm for article recommendation 1. Extract anchor-text links from all 4, 625 Wikipedia Selection for School articles and store anchor texts in the database 2. Prepare article titles and anchor texts from previous process as the input to generate the topic mode based on the Lda algorithm. The output from this ep is the topic probability for eac 3. The article similarity is computed by using the dot product between two topic probability vectors. The scores from the dot-product calculation are used to rank the top-10 articles that are related to the current article
1. Choose θ ~ Dir ( ) α : Choose a latent topics mixture vector θ from the Dirichlet distribution. 2. For each word n ∈ ww (a) Choose a topic n z ~ Multinomial( ) θ : Choosealatent topic n z from the multinomial distribution. (b) Choose a word wn from ( β), n, |z wp n a multinomial probability conditioned on the topic n z . In this paper, we focus on the Wikipedia Selection for schools for evaluating our proposed recommendation algorithm. Our proposed approach based on the topic model for recommending related articles and discovering missing links consists of three main processes as shown in Figure 2. . Figure 2: The proposed topic-based model via LDA algorithm for article recommendation. 1. Extract anchor-text links from all 4,625 Wikipedia Selection for School articles and store anchor texts in the database. 2. Prepare article titles and anchor texts from previous process as the input to generate the topic mode based on the LDA algorithm. The output from this step is the topic probability for each article. 3. The article similarity is computed by using the dot product between two topic probability vectors. The scores from the dot-product calculation are used to rank the top-10 articles that are related to the current article. 198