In this chapter we present the most usual experimental protocols and metrics employed for offline evaluation of tag recommender systems. By offline we mean that the algorithms are evaluated on a snapshot of some real-world STS dataset, which, in turn, is typically split into training and test datasets. This corresponds to the most typical evaluation scenario found in the literature since researchers do not need to have a STS up and running for assessing the performance of his/her algorithms. We also summarize the main tag recommendation algorithms presented in this book, pointing out pros and cons in terms of the metrics and protocols introduced in this chapter.
Unable to display preview. Download preview PDF.
- 1.Folke Eisterlehner, Andreas Hotho, and Robert Jäschke, editors. ECML PKDD Discovery Challenge 2009 (DC09), volume 497 of CEUR-WS.org, September 2009.Google Scholar
- 2.Jonathan L. Herlocker, Joseph A. Konstan, Loren G. Terveen, and John T. Riedl. Evaluating collaborative filtering recommender systems. ACM Trans. Inf. Syst., 22(1):5–53, 2004.Google Scholar
- 3.Robert Jäschke, Leandro Marinho, Andreas Hotho, Lars Schmidt- Thieme, and Gerd Stumme. Tag recommendations in social bookmarking systems. AI Communications, 21(4):231–247, 2008.Google Scholar
- 4.Robert Jäschke, Leandro Balby Marinho, Andreas Hotho, Lars Schmidt- Thieme, and Gerd Stumme. Tag recommendations in folksonomies. In Joost N. Kok, Jacek Koronacki, Ramon López de Mántaras, Stan Matwin, Dunja Mladenic, and Andrzej Skowron, editors, Knowledge Discovery in Databases: PKDD 2007, 11th European Conference on Principles and Practice of Knowledge Discovery in Databases, volume 4702 of Lecture Notes in Computer Science, pages 506–514, Berlin, Heidelberg, 2007. Springer.Google Scholar
- 5.Marek Lipczak, Yeming Hu, Yael Kollet, and Evangelos Milios. Tag sources for recommendation in collaborative tagging systems. In Folke Eisterlehner, Andreas Hotho, and Robert Jäschke, editors, ECML PKDD Discovery Challenge 2009 (DC09), volume 497 of CEUR-WS.org, pages 157–172, 2009.Google Scholar
- 6.Leandro Balby Marinho, Christine Preisach, and Lars Schmidt-Thieme. Relational classification for personalized tag recommendation. In Folke Eisterlehner, Andreas Hotho, and Robert Jäschke, editors, ECML PKDD Discovery Challenge 2009 (DC09), volume 497 of CEUR-WS.org, pages 7–16, 2009.Google Scholar
- 7.Leandro Balby Marinho and Lars Schmidt-Thieme. Collaborative tag recommendations. In GFKL ’07: Proceedings of the 31st Annual Conference of the Gesellschaft für Klassifikation (GfKl), Freiburg, pages 533–540. Springer, 2007.Google Scholar
- 8.Christine Preisach, Leandro Balby Marinho, and Lars Schmidt-Thieme. Semi-supervised tag recommendation - using untagged resources to mitigate cold-start problems. In PAKDD ’10: Proceedings of the 14th Pacific- Asia Conference on Advances in Knowledge Discovery and Data Mining, 2010. to appear.Google Scholar
- 9.Steffen Rendle, Leandro B. Marinho, Alexandros Nanopoulos, and Lars Schimdt-Thieme. Learning optimal ranking with tensor factorization for tag recommendation. In KDD ’09: Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 727–736. ACM, 2009.Google Scholar
- 10.Steffen Rendle and Lars Schmidt-Thieme. Factor models for tag recommendation in BibSonomy. volume 497 of CEUR-WS.org, pages 235–242, 2009.Google Scholar
- 11.Steffen Rendle and Lars Schmidt-Thieme. Pairwise interaction tensor factorization for personalized tag recommendation. In WSDM ’10: Proceedings of the Third ACM International Conference on Web Search and Data Mining. ACM, 2010.Google Scholar
- 12.Guy Shani and Asela Gunawardana. Evaluating recommendation systems. In Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor, editors, Recommender Systems Handbook, pages 257–297. Springer US, 2011.Google Scholar