Advertisement

Predicting IR personalization performance using pre-retrieval query predictors

  • Eduardo Vicente-López
  • Luis M. de Campos
  • Juan M. Fernández-Luna
  • Juan F. Huete
Article
  • 43 Downloads

Abstract

Although personalization generally improves query performance, it may also occasionally harm how queries perform. If we are able to predict and therefore disable personalization for such situations, overall performance will be higher and users will be more satisfied with personalized systems. We use various state-of-the-art, pre-retrieval query performance predictors and propose several others including user profile information for this purpose. We study the correlations between these predictors and the difference between personalized and original queries. We also use classification and regression techniques to improve the results and finally achieve slightly more than one third of maximum ideal performance. We consider this to be a good starting point within this research line, which will undoubtedly result in further work and improvements.

Keywords

Personalization Information retrieval Query difficulty Performance prediction 

Notes

Acknowledgements

This work has been supported by the Spanish Andalusian “Consejería de Innovación, Ciencia y Empresa” postdoctoral phase of project P09-TIC-4526, the Spanish “Ministerio de Economía y Competitividad” projects TIN2013-42741-P and TIN2016-77902-C3-2-P, and the European Regional Development Fund (ERDF-FEDER).

References

  1. Amati, G., Carpineto, C., Romano, G. (2004). Query difficulty, robustness, and selective application of query expansion. In European conference on information retrieval (pp. 127–137): Springer.Google Scholar
  2. Aslam, J.A., & Pavlu, V. (2007). Query hardness estimation using jensen-shannon divergence among multiple scoring functions. In European conference on information retrieval (pp. 198–209): Springer.Google Scholar
  3. Carmel, D., & Yom-Tov, E. (2010). Estimating the query difficulty for information retrieval. Synthesis Lectures on Information Concepts, Retrieval, and Services, 2(1), 1–89.CrossRefMATHGoogle Scholar
  4. Carmel, D., Yom-Tov, E., Darlow, A., Pelleg, D. (2006). What makes a query difficult? In Proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval (pp. 390–397): ACM.Google Scholar
  5. Chen, C., Yang, M., Li, S., Zhao, T., Qi, H. (2010). Predicting query potential for personalization, classification or regression? In Proceedings of the 33rd international ACM SIGIR conference on research and development in information retrieval (pp. 725–726): ACM.Google Scholar
  6. Cronen-Townsend, S., Zhou, Y., Croft, W.B. (2002). Predicting query performance. In Proceedings of the 25th annual international ACM SIGIR conference on research and development in information retrieval (pp. 299–306): ACM.Google Scholar
  7. Dan, O., & Davison, B.D. (2016). Measuring and predicting search engine users’ satisfaction. ACM Computing Surveys, 49(1), 18.CrossRefGoogle Scholar
  8. de Campos, L.M., Fernández-Luna, J.M., Huete, J.F., Romero, A.E. (2006). Garnata: an information retrieval system for structured documents based on probabilistic graphical models. In Proceedings of the eleventh international conference of information processing and management of uncertainty in knowledge-based systems (pp. 1024–1031).Google Scholar
  9. de Campos, L.M., Fernández-Luna, J. M., Huete, J.F., Martín-dancausa, C., Romero, A.E. (2009). New utility models for the garnata information retrieval system at inex’08. In Advances in focused retrieval (pp. 39–45): Springer.Google Scholar
  10. de Campos, L.M., Fernández-Luna, J.M., Huete, J.F., Vicente-López, E. (2014). Using personalization to improve xml retrieval. IEEE Transactions on Knowledge and Data Engineering, 26(5), 1280–1292.CrossRefGoogle Scholar
  11. Dou, Z., Song, R., Wen, J.R. (2007). A large-scale evaluation and analysis of personalized search strategies. In Proceedings of the 16th international conference on world wide web (pp. 581–590): ACM.Google Scholar
  12. Ghorab, M.R., Zhou, D., O’Connor, A., Wade, V. (2013). Personalised information retrieval: survey and classification. User Modeling and User-Adapted Interaction, 23(4), 381–443.CrossRefGoogle Scholar
  13. Guo, Q., White, R.W., Dumais, S.T., Wang, J., Anderson, B. (2010). Predicting query performance using query, result, and user interaction features. In Adaptivity, personalization and fusion of heterogeneous information (pp. 198–201): Le Centre de hautes Etudes Internationales d’Informatique Documentaire.Google Scholar
  14. Hauff, C. (2010). Predicting the effectiveness of queries and retrieval systems. Thesis, Centre for Telematics and Information Technology University of Twente.Google Scholar
  15. Hauff, C., Hiemstra, D., de Jong, F. (2008a). A survey of pre-retrieval query performance predictors. In Proceedings of the 17th ACM conference on information and knowledge management (pp. 1419–1420): ACM.Google Scholar
  16. Hauff, C., Murdock, V., Baeza-Yates, R. (2008b). Improved query difficulty prediction for the web. In Proceedings of the 17th ACM conference on information and knowledge management (pp. 439–448): ACM.Google Scholar
  17. Hauff, C., Kelly, D., Azzopardi, L. (2010). A comparison of user and system query performance predictions. In Proceedings of the 19th ACM international conference on information and knowledge management (pp. 979–988): ACM.Google Scholar
  18. He, B., & Ounis, I. (2004). Inferring query performance using pre-retrieval predictors. In International symposium on string processing and information retrieval (pp. 43–54): Springer.Google Scholar
  19. He, J., Larson, M., De Rijke, M. (2008). Using coherence-based measures to predict query difficulty. In European conference on information retrieval (pp. 689–694): Springer.Google Scholar
  20. Järvelin, K., & Kekäläinen, J. (2002). Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems, 20(4), 422–446.CrossRefGoogle Scholar
  21. Liu, F., Yu, C., Meng, W. (2004). Personalized web search for improving retrieval effectiveness. IEEE transactions on Knowledge and Data Engineering, 16(1), 28–40.CrossRefGoogle Scholar
  22. Micarelli, A., Gasparetti, F., Sciarrone, F., Gauch, S. (2007). Personalized search on the world wide web. In The adaptive web (pp. 195–230): Springer.Google Scholar
  23. Mothe, J., & Tanguy, L. (2005). Linguistic features to predict query difficulty. In ACM conference on research and development in information retrieval, SIGIR, predicting query difficulty-methods and applications workshop (pp. 7–10).Google Scholar
  24. Shtok, A., Kurland, O., Carmel, D., Raiber, F., Markovits, G. (2012). Predicting query performance by query-drift estimation. ACM Transactions on Information Systems, 30(2), 11.CrossRefGoogle Scholar
  25. Steichen, B., Ashman, H., Wade, V. (2012). A comparative survey of personalised information retrieval and adaptive hypermedia techniques. Information Processing & Management, 48(4), 698–724.CrossRefGoogle Scholar
  26. Teevan, J., Dumais, S.T., Liebling, D.J. (2008). To personalize or not to personalize: modeling queries with variation in user intent. In Proceedings of the 31st annual international ACM SIGIR conference on research and development in information retrieval (pp. 163–170): ACM.Google Scholar
  27. Teevan, J., Dumais, S.T., Horvitz, E. (2010). Potential for personalization. ACM Transactions on Computer-Human Interaction, 17(1), 4.CrossRefGoogle Scholar
  28. Vicente-López, E., de Campos, L.M., Fernández-Luna, J.M., Huete, J.F., Tagua-Jiménez, A., Tur-Vigil, C. (2015). An automatic methodology to evaluate personalized information retrieval systems. User Modeling and User-Adapted Interaction, 25(1), 1–37.CrossRefGoogle Scholar
  29. Vicente-López, E., de Campos, L.M., Fernández-Luna, J.M., Huete, J.F. (2016). Use of textual and conceptual profiles for personalized retrieval of political documents. Knowledge-Based Systems, 112, 127–141.CrossRefGoogle Scholar
  30. Younus, A., Qureshi, M.A., O’Riordan, C., Pasi, G. (2013). Personalization for difficult queries. SIGIR Workshop on Modeling User Behavior for Information Retrieval Evaluation, 15–16.Google Scholar
  31. Zhang, W., Wang, J., Chen, B., Zhao, X. (2013). To personalize or not: a risk management perspective. In Proceedings of the 7th ACM conference on recommender systems (pp. 229–236): ACM.Google Scholar
  32. Zhao, Y., Scholer, F., Tsegay, Y. (2008). Effective pre-retrieval query performance prediction using similarity and variability evidence. In European conference on information retrieval (pp. 52–64): Springer,.Google Scholar
  33. Zhou, Y., & Croft, W.B. (2006). Ranking robustness: a novel framework to predict query performance. In Proceedings of the 15th ACM international conference on information and knowledge management (pp. 567–574): ACM.Google Scholar
  34. Zhou, Y., & Croft, W.B. (2007). Query performance prediction in web search environments. In Proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval (pp. 543–550): ACM.Google Scholar
  35. Zhou, B., & Yao, Y. (2010). Evaluating information retrieval system performance based on user preference. Journal of Intelligent Information Systems, 34(3), 227–248.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Departamento de Ciencias de la Computación e Inteligencia Artificial, E.T.S.I.I.T., CITIC-UGRUniversidad de GranadaGranadaSpain

Personalised recommendations