Abstract
This chapter presents the different evaluation methods for a recommender system. We introduce the existing metrics, as well as the pros and cons of each method. This chapter is the background for the following Chaps. 6 and 7, where the proposed metrics are used in real, large size, recommendation datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
J. S. Breese, D. Heckerman, and C. Kadie, “Empirical analysis of predictive algorithms for collaborative filtering,” tech. rep., Microsoft Research, 1998.
J. L. Herlocker, J. A. Konstan, L. G. Terveen, and J. T. Riedl, “Evaluating collaborative filtering recommender systems,” ACM Transaction on Information Systems, vol. 22, no. 1, pp. 5–53, 2004.
G. Shani and A. Gunawardana, “Evaluating recommender systems,” tech. rep., Microsoft Research, MSR-TR-2009–159, November 2009.
M. Balabanovic and Y. Shoham, “Fab: Content-based, collaborative recommendation,” Communications of the ACM, vol. 40, pp. 66–72, 1997.
Y. Y. Yao, “Measuring retrieval effectiveness based on user preference of documents,” Journal of the American Society for Information Science, vol. 46, no. 2, pp. 133–145, 1995.
G. Adomavicius and A. Tuzhilin, “Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions,” IEEE Transactions on Knowledge and Data Engineering, vol. 17, no. 6, pp. 734–749, 2005.
S. M. McNee, J. Riedl, and J. A. Konstan, “Being accurate is not enough: How accuracy metrics have hurt recommender systems,” in Computer Human Interaction. Human factors in computing systems, (New York, NY), pp. 1097–1101, ACM, 2006.
P. Erdös and A. Réyi, “On random graphs,” Science, vol. 6, no. 290, pp. 290–298, 1959.
A. L. Barabási and R. Albert, “Emergence of scaling in random networks,” Science, vol. 286, pp. 509–512, October 1999.
M. E. J. Newman, “Assortative mixing in networks,” Physical Review Letters, vol. 89, no. 20, 2002.
M. E. J. Newman, “Mixing patterns in networks,” Physical Review E, vol. 67, 2003.
D. J. Watts and S. H. Strogatz, “Collective dynamics of ‘small-world’ networks,” Nature, vol. 393, pp. 440–442, June 1998.
E. Ravasz and A. L. Barabási, “Hierarchical organization in complex networks,” Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics, vol. 67, February 2003.
E. Ravasz, A. L. Somera, D. A. Mongru, Z. N. Oltvai, and A. L. Barabási, “Hierarchical organization of modularity in metabolic networks,” Science, vol. 297, no. 5586, pp. 1551–1555, 2002.
M. Newman, “A measure of betweenness centrality based on random walks,” Social Networks, vol. 27, pp. 39–54, January 2005.
G. Sabidussi, “The centrality index of a graph,” Psychometrika, vol. 31, pp. 581–603, December 1966.
L. C. Freeman, “Centrality in social networks: Conceptual clarification,” Social Networks, vol. 1, no. 3, pp. 215–239, 1979.
P. Cano, O. Celma, M. Koppenberger, and J. Martin-Buldú, “Topology of music recommendation networks,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 16, no. 013107, 2006.
J. Martin-Buldú, P. Cano, M. Koppenberger, J. Almendral, and S. Boccaletti, “The complex network of musical tastes,” New Journal of Physics, vol. 9, no. 172, 2007.
J. Park, O. Celma, M. Koppenberger, P. Cano, and J. Martin-Buldú, “The social network of contemporary popular musicians,” International Journal of Bifurcation and Chaos, vol. 17, no. 7, pp. 2281–2288, 2007.
A. Anglade, M. Tiemann, and F. Vignoli, “Complex-network theoretic clustering for identifying groups of similar listeners in p2p systems,” in Proceedings of the ACM conference on Recommender systems, (Minneapolis, MN), pp. 41–48, ACM, 2007.
J.-J. Aucouturier and F. Pachet, “A scale-free distribution of false positives for a large class of audio similarity measures,” Pattern Recognition, vol. 41, no. 1, pp. 272–284, 2008.
K. Jacobson and M. Sandler, “Musically meaningful or just noise? an analysis of on-line artist networks,” in Proceedings of the 6th International Symposium on Computer Music Modeling and Retrieval, (Copenhagen, Denmark), 2008.
R. Lambiotte and M. Ausloos, “Uncovering collective listening habits and music genres in bipartite networks,” Physical Review E, vol. 72, 2005.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2010 Springer Berlin Heidelberg
About this chapter
Cite this chapter
Celma, Ò. (2010). Evaluation Metrics. In: Music Recommendation and Discovery. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13287-2_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-13287-2_5
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13286-5
Online ISBN: 978-3-642-13287-2
eBook Packages: Computer ScienceComputer Science (R0)