Evaluation of Score Standardization Methods for Web Search in Support of Results Diversification

  • Zhongmin Zhang
  • Chunlin Xu
  • Shengli WuEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11242)


Score standardization is a necessary step for many different types of Web search tasks in which results from multiple components need to be combined or re-ranked. Some recent studies suggest that score standardization may have impact on the performance of some typical explicit search result diversification methods such as XQuAD. In this paper, we evaluate the performance of six score standardization methods. Experiments with TREC data are carried out with two typical explicit result diversification methods XQuAD and PM2. We find that the reciprocal standardization method performs better than other score standardization methods in all the cases. Furthermore, we improve the reciprocal standardization method by scaling those scores up so as to better satisfy the requirement of probability scores and obtain better results with XQuAD. We confirm that score standardization has significant impact on the performance of explicit search result diversification methods and such a fact can be used to obtain more profitable score standardization methods and result diversification methods.


Score standardization Explicit search result diversification Web search Information retrieval 


  1. 1.
    Carbonell, J., Goldstein, J.: The use of MMR, diversity-based reranking for reordering documents and producing summaries. In: SIGIR, pp. 335–336 (1998)Google Scholar
  2. 2.
    Cormack, G.V., Clarke, C.L.A., Büttcher, S.: Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In: SIGIR, pp. 758–759 (2009)Google Scholar
  3. 3.
    Dang, V., Croft, W.B.: Diversity by proportionality: an election-based approach to search result diversification. In: SIGIR, pp. 65–74 (2012)Google Scholar
  4. 4.
    Lee, J.H.: Analyses of multiple evidence combination. In: SIGIR, pp. 267–276 (1997)CrossRefGoogle Scholar
  5. 5.
    Montague, M., Aslam, J.A.: Relevance score standardization for meta-search. In: CIKM, pp. 427–433 (2001)Google Scholar
  6. 6.
    Ozdemiray, A.M., Altingovde, I.S.: Explicit search result diversification using score and rank aggregation methods. JASIST 66(6), 1212–1228 (2015)Google Scholar
  7. 7.
    Renda, M.E., Straccia, U.: Web meta-search: rank vs. score based rank aggregation methods. In: SAC, pp. 841–846 (2003)Google Scholar
  8. 8.
    Santos, R.L.T., Macdonald, C., Ounis, C.I.: Exploiting query reformulations for web search result diversification. In: WWW, pp. 881–890 (2010)Google Scholar
  9. 9.
    Wu, S.: Data Fusion in Information Retrieval. Springer, Berlin (2012). Scholar
  10. 10.
    Zhai, C.X., Cohen, W.W., Lafferty, J.: Beyond independent relevance: methods and evaluation metrics for subtopic retrieval. In: SIGIR, pp. 10–17 (2003)Google Scholar
  11. 11.
    Shokouhi, M., Si, L.: Federated search. Found. Trends Inf. Retr. 5(1), 1–102 (2011)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.School of Computer ScienceJiangsu UniversityZhenjiangChina
  2. 2.School of ComputingUlster UniversityNewtownabbeyUK

Personalised recommendations