Evaluation Web Search and Mining Lecture 9: Evaluation
Evaluation Lecture 9: Evaluation Web Search and Mining
Evaluation This lecture How do we know if our results are good? Evaluating a search engine Benchmarks Precision and recall Results summaries. Making our good results usable to a user
Evaluation 2 This lecture ▪ How do we know if our results are good? ▪ Evaluating a search engine ▪ Benchmarks ▪ Precision and recall ▪ Results summaries: ▪ Making our good results usable to a user
Evaluation EVALUATING SEARCH ENGINES
Evaluation EVALUATING SEARCH ENGINES
Evaluation Measures easures for a search engine How fast does it index Number of documents/hour (Average document size) How fast does it search Latency as a function of index size Expressiveness of query language ability to express complex information needs Speed on complex queries Uncluttered U Is it free?
Evaluation 4 Measures for a search engine ▪ How fast does it index ▪ Number of documents/hour ▪ (Average document size) ▪ How fast does it search ▪ Latency as a function of index size ▪ Expressiveness of query language ▪ Ability to express complex information needs ▪ Speed on complex queries ▪ Uncluttered UI ▪ Is it free? Measures
Evaluation Measures easures for a search engine All of the preceding criteria are measurable: we can quantify speed /size we can make expressiveness precise The key measure: user happiness What is this? Speed of response size of index are factors But blindingly fast, useless answers wont make a user happy Need a way of quantifying user happiness
Evaluation 5 Measures for a search engine ▪ All of the preceding criteria are measurable: we can quantify speed/size ▪ we can make expressiveness precise ▪ The key measure: user happiness ▪ What is this? ▪ Speed of response/size of index are factors ▪ But blindingly fast, useless answers won’t make a user happy ▪ Need a way of quantifying user happiness Measures