Relevance Feedback and Query Expansion Web Search and Mining Lecture 10: Query expansion
Relevance Feedback and Query Expansion Lecture 10: Query expansion Web Search and Mining 1
Relevance Feedback and Query Expansion Recap of the last lecture Evaluating a search engine Benchmarks Precision and recall Results summaries
Relevance Feedback and Query Expansion Recap of the last lecture ▪ Evaluating a search engine ▪ Benchmarks ▪ Precision and recall ▪ Results summaries 2
Relevance Feedback and Query Expansion Recap: Unranked retrieval evaluation Precision and recall Precision fraction of retrieved docs that are relevant P(relevant retrieved Recall fraction of relevant docs that are retrieved P(retrieved relevant Relevant Nonrelevant Retrieved Not Retrieved fn Precision P= tp/tp fp) Recall r=tp/tp+ fn)
Relevance Feedback and Query Expansion 3 Recap: Unranked retrieval evaluation: Precision and Recall ▪ Precision: fraction of retrieved docs that are relevant = P(relevant|retrieved) ▪ Recall: fraction of relevant docs that are retrieved = P(retrieved|relevant) ▪ Precision P = tp/(tp + fp) ▪ Recall R = tp/(tp + fn) Relevant Nonrelevant Retrieved tp fp Not Retrieved fn tn
Relevance Feedback and Query Expansion Recap: A combined measure: F Combined measure that assesses precision/recall tradeoff is F measure weighted harmonic mean F (B2+1)PR a+(1-a) BP+R R People usually use balanced F, measure e,withβ=1orω= Harmonic mean is a conservative average
Relevance Feedback and Query Expansion 4 Recap: A combined measure: F ▪ Combined measure that assesses precision/recall tradeoff is F measure (weighted harmonic mean): ▪ People usually use balanced F1 measure ▪ i.e., with = 1 or = ½ ▪ Harmonic mean is a conservative average P R PR P R F + + = + − = 2 2 ( 1) 1 (1 ) 1 1
Relevance Feedback and Query Expansion This lecture Improving results For high recall E.g. searching for aircraft doesn't match with plane; nor thermodynamic with heat Options for improving results Local methods Relevance feedback Pseudo relevance feedback Global methods Query expansion thesaurus Automatic thesaurus generation
Relevance Feedback and Query Expansion This lecture ▪ Improving results ▪ For high recall. ▪ E.g., searching for aircraft doesn’t match with plane; nor thermodynamic with heat ▪ Options for improving results… ▪ Local methods ▪ Relevance feedback ▪ Pseudo relevance feedback ▪ Global methods ▪ Query expansion ▪ Thesaurus ▪ Automatic thesaurus generation 5