Poster Title:

Automatic Search Engine Performance Evaluation with Click-through Data Analysis

Authors:

• Yiqun Liu (Tsinghua University)
• Yupeng Fu (Tsinghua University)
• Min Zhang (Tsinghua University)
• Shaoping Ma (Tsinghua University)
• Liyun Ru (Sohu Incorporation)

Abstract:

Performance evaluation is an important issue in Web search engine researches. Traditional evaluation methods rely on lots of human efforts and are therefore quite time-consuming. With click through data analysis, we proposed an automatic search engine performance evaluation method. This method is a Cranfield-like one and it generates navigational type query topics and answers automatically based on a search user's query and click behavior. Experimental results based on a commercial Chinese search engine's user logs show that the automatically method gets a similar evaluation result with traditional assessor-based ones.