Abstract
In this paper we report experiments that we conducted using an implementation of a recommender system called “Knowledge Pump" (KP) developed at Xerox. We repeat well-known methods such as the Pearson method, but also address common problems of recommender systems, in particular the sparsity problem. The sparsity problem is the problem of having too few ratings and hence too few correlations between users. We address this problem in two different manners. First, we introduce “transitive correlations", a mechanism to increase the number of correlations between existing users. Second, we add “agents", artificial users that rate in accordance with some predefined preferences. We show that both ideas pay off, albeit in different ways: Transitive correlations provide a small help for virtually no price, whereas rating agents improve the coverage of the system significantly but also have a negative impact on the system performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Each Movie collaborative filtering data set, http://research.compaq.com/SRC/eachmovie/
Aggarwal, C.C., Wolf, J.L., Wu, K.L., Yu, P.S.: Horting hatches an egg: A new graph-theoretic approach to collaborative filtering. In: Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA, August 1999, pp. 201–212 (1999)
Balabanovic, M., Shoham, Y.: Fab: Content-based, collaborative recommendation. Communications of the ACM 40(3), 66–72 (1997)
Claypool, M., Brown, D., Le, P., Waseda, M.: Inferring user interest. IEEE Internet Computing 5(6), 32–39 (2001)
Glance, N., Arregui, D., Dardenne, M.: Knowledge Pump: Supporting the flow and use of knowledge. In: Borghoff, U., Pareschi, R. (eds.) Information Technology for Knowledge Management, pp. 35–51. Springer, Heidelberg (1998)
Good, N., Schafer, B., Konstan, J.A., Borchers, A., Sarwar, B., Herlocker, J., Riedl, J.: Combining collaborative filtering with personal agents for better recommendations. In: Proceedings of the National Conference on Artificial Intelligence (AAAI), Orlando, FL, USA, July 1999, pp. 439–446 (1999)
Herlocker, J., Konstan, J.A., Borchers, A., Riedl, J.: An algorithmic framework for performing collaborative filtering. In: Proceedings of the ACM SIGIR International Conference on Research and Development in Information Retrieval, Berkeley, CA, USA, August 1999, pp. 230–237 (1999)
Hill, W., Stead, L., Rosenstein, M., Furnas, G.: Recommending and evaluating choices in a virtual community of use. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI), Denver, CO, USA, May 1995, pp. 194–201 (1995)
Kautz, H., Selman, B., Shah, M.: Referral Web: Combining social networks and collaborative filtering. Communications of the ACM 40(3), 63–65 (1997)
Malone, T.W., Grant, K.R., Turbak, F.A., Brobst, S.A., Cohen, M.D.: Intelligent information sharing systems. Communications of the ACM 30(5), 390–402 (1987)
Maneeroj, S., Kanai, H., Hakozaki, K.: Combining dynamic agents and collaborative filtering without sparsity rating problem for better recommendation quality. In: Proceedings of the DELOS Network of Excellence Workshop on Personalisation and Recommender Systems in Digital Libraries, Dublin, Ireland (June 2001)
Melville, P., Mooney, R.J., Nagarajan, R.: Content-boosted collaborative filtering. In: Proceedings of the ACM SIGIR Workshop on Recommender Systems, New Orleans, LA, USA (September 2001)
Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., Riedl, J.: GroupLens: An open architecture for collaborative filtering of netnews. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW), Chapel Hill, NC, USA, October 1994, pp. 175–186 (1994)
Sarwar, B.M., Konstan, J.A., Borchers, A., Herlocker, J., Miller, B., Riedl, J.: Using filtering agents to improve prediction quality in the GroupLens research collaborative filtering system. In: Proceedings of the ACM Conference on Computer Supported Cooperative Work (CSCW), Seattle, WA, USA, November 1998, pp. 345–354 (1998)
Shardanand, U., Maes, P.: Social information filtering: Algorithms for automating “word of mouth”. In: Proceedings of the Conference on Human Factors in Computing Systems (CHI), Denver, CO, USA, May 1995, pp. 210–217 (1995)
Terveen, L., Hill, W., Amento, B., McDonald, D., Creter, J.: PHOAKS: A system for sharing recommendations. Communications of the ACM 40(3), 59–62 (1997)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bergholz, A. (2003). Coping with Sparsity in a Recommender System. In: Zaïane, O.R., Srivastava, J., Spiliopoulou, M., Masand, B. (eds) WEBKDD 2002 - Mining Web Data for Discovering Usage Patterns and Profiles. WebKDD 2002. Lecture Notes in Computer Science(), vol 2703. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-39663-5_6
Download citation
DOI: https://doi.org/10.1007/978-3-540-39663-5_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20304-9
Online ISBN: 978-3-540-39663-5
eBook Packages: Springer Book Archive