Abstract
Collaborative Filtering (CF) techniques are important in the e-business era as vital components of many recommender systems, for they facilitate the generation of high-quality recommendations by leveraging the similar preferences of community users. However, there is still a major problem preventing CF algorithms from achieving better effectiveness, the sparsity of training data. Lots of ratings in the training matrix are not collected. Few current CF methods try to do data smoothing before predicting the ratings of an active user. In this work, we have validated the effectiveness of data smoothing for memory-based and hybrid collaborative filtering algorithms. Our experiments show that all these algorithms achieve a higher accuracy after proper smoothing. The average mean absolute error improvements of the three CF algorithms, Item Based, k Nearest Neighbor and Personality Diagnosis, are 6.32%, 8.85% and 38.0% respectively. Moreover, we have compared different smoothing methods to show which works best for each of the algorithms.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Breese, J.S., Heckerman, D., Kadie, C.: Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of the 14th Conference on Uncertainty in Artifical Intelligence, pp. 43–52 (1998)
Kohrs, A., Merialdo, B.: Clustering for clooaborative filtering applications. IOS Press, Amsterdam (1999)
Ungar, L.H., Foster, D.P.: Clustering methods for collaborative filtering. In: Proceedings of the Workshop on Recommendation Systems, AAAI Press, Menlo Park (1998)
Hofmann, T.: Latent semantic models for collaborative filtering. ACM Transactions on Information System 22(1), 89–115 (2004)
Konstan, J.A., Miller, B.N., Maltz, D., Herlocker, J.L., Gordon, L.R., Riedl, J.: Grouplens: applying collaborative filtering to usenet news. Communications of the ACM 40(3), 77–87 (1997)
Soboroff, I., Nicholas, C.: Collaborative filtering and the generalized vector space model (poster session). In: Proceedings of the 23rd annual international conference on Research and development in information retrieval, pp. 351–353 (2000)
Penmnock, D.M., Horvitz, E., Lawrence, S., Giles, C.L.: Collaborative filtering by personality diagnosis: A hybrid memory-and-model-based approach. In: Proc. of the 16th Conference on Uncertainty in Artifical Intelligence, pp. 473–480 (2000)
Goldberg, K.Y., Roeder, T., Gupta, D., Perkins, C.: Eigentaste: A constant time collaborative filtering algorithm. Information Retrieval 4, 133–151 (2001)
Fisher, D., Hildrum, K., Hong, J., Newman, M., Thomas, M., Vuduc, R.: Swami: a framework for collaborative filtering algorithm development and evaluation. In: Proceedings of the 23rd annual international conference on Research and development in information retrieval, pp. 366–368 (2000)
Sarwar, B.M., Karypis, G., Konstan, J.A., Riedl, J.T.: Application of dimensionality reduction in recommender system – a case study. In: ACM WebKDD 2000 Web Mining for E-Commerce Workshop (2000)
Zeng, C., Xing, C.-X., Zhou, L.-Z.: Similarity measure and instance selection for collaborative filtering. In: Proceedings of the 12th international conference on World Wide Web, pp. 652–658 (2003)
Balabanovic, M., Shoham, Y.: Fab: content-based, collaborative recommendation. Communication of the ACM 40, 66–72 (1997)
Claypool, M., Gokhale, A., Mirands, T., Murnikov, P., Netes, D., Sartin, M.: Combining content-based and collaborative filters in an online newspaper. In: ACM SIGIR Workshop on Recommender Systems - Implementation and Evaluation (1999)
Popescul, A., Ungar, L.H., Pennock, D.M., Lawrence, S.: Probabilistic models for unified collaborative and content-based recommendation in sparse-data environments. In: Proceedings of the 17th Conference on Uncertainty in Artifical Intelligence, pp. 437–444 (2001)
Xue, G.R., Lin, C., Yang, Q., Xi, W., Zeng, H.J., Yu, Y., Chen, Z.: Scalable collaborative filtering using cluster-based smoothing. In: Proceedings of the 28th annual international conference on Research and development in information retrieval, pp. 114–121 (2005)
Sarwar, B., Karypis, G., Konstan, J., Reidl, J.: Item-based collaborative filtering recommendation algorithms. In: Proceedings of the 10th international conference on World Wide Web, pp. 285–295 (2001)
Herlocker, J.L., Konstan, J.A., Terveen, L.G., Riedl, J.T.: Evaluating collaborative filtering recommender systems. Trans. on Information System 22, 5–53 (2004)
Zhao, Y., Karypis, G., Fayyad, U.: Hierarchical clustering algorithms for document datasets. Data Mining and Knowledge Discovery 10, 141–168 (2005)
Zhao, Y., Karypis, G.: Soft clustering criterion functions for partitional document clustering: a summary of results. In: Proceedings of the thirteenth ACM conference on Information and knowledge management, pp. 246–247 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Han, D., Xue, GR., Yu, Y. (2006). An Empirical Study of Data Smoothing Methods for Memory-Based and Hybrid Collaborative Filtering. In: Yang, Q., Webb, G. (eds) PRICAI 2006: Trends in Artificial Intelligence. PRICAI 2006. Lecture Notes in Computer Science(), vol 4099. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-36668-3_11
Download citation
DOI: https://doi.org/10.1007/978-3-540-36668-3_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-36667-6
Online ISBN: 978-3-540-36668-3
eBook Packages: Computer ScienceComputer Science (R0)