A Cluster-Based Incremental Recommendation Algorithm on Stream Processing Architecture

  • Yuqi Wang
  • Yin Zhang
  • Yanfei Yin
  • Deng Yi
  • Baogang Wei
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8279)


By helping users discover books they may be interested in, recommender systems fully exploit the resources of digital libraries and better facilitate users’ reading demands. Traditional memory-based collaborative filtering (CF) methods are effective and easy to interpret. However, when datasets become larger, the traditional way turns to be infeasible in both time and space. In order to address this challenge, we propose an incremental, cluster-based algorithm on Stream Processing Architecture, which is scalable and suitable to real-time environment. Our experimental results on MovieLens datasets and CADAL user-chapter logs show our algorithm is efficient, while still maintains comparable accuracy and interpretability.


Incremental Recommendations Cluster-based Stream Processing Architecture 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Breese, J.S., Heckerman, D., Kadie, C.: Empirical analysis of predictive algorithms for collaborative filtering. In: UAI 1998, pp. 43–52 (1998)Google Scholar
  2. 2.
    Cao, H., Jiang, D., Pei, J., He, Q., Liao, Z., Chen, E., Li, H.: Context-aware query suggestion by mining click-through and session data. In: KDD 2008, pp. 875–883 (2008)Google Scholar
  3. 3.
    Cremonesi, P., Koren, Y., Turrin, R.: Performance of recommender algorithms on top-n recommendation tasks. In: RecSys 2010, pp. 39–46 (2010)Google Scholar
  4. 4.
    Deshpande, M., Karypis, G.: Item-based top-n recommendation algorithms. ACM Trans. Inf. Syst. 22(1), 143–177 (2004)CrossRefGoogle Scholar
  5. 5.
    Giraud-Carrier, C.G.: A note on the utility of incremental learning. AI Commun. 13(4), 215–224 (2000)zbMATHGoogle Scholar
  6. 6.
    Jamali, M., Ester, M.: Using a trust network to improve top-n recommendation. In: RecSys 2009, pp. 181–188 (2009)Google Scholar
  7. 7.
    Papagelis, M., Rousidis, I., Plexousakis, D., Theoharopoulos, E.: Incremental collaborative filtering for highly-scalable recommendation algorithms. In: Hacid, M.-S., Murray, N.V., Raś, Z.W., Tsumoto, S. (eds.) ISMIS 2005. LNCS (LNAI), vol. 3488, pp. 553–561. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  8. 8.
    Resnick, P., Iacovou, N., Suchak, M., Bergstrom, P., Riedl, J.: Grouplens: An open architecture for collaborative filtering of netnews. In: CSCW 1994, pp. 175–186 (1994)Google Scholar
  9. 9.
    Salakhutdinov, R., Mnih, A.: Probabilistic matrix factorization. In: NIPS 2007 (2007)Google Scholar
  10. 10.
    Yang, X., Zhang, Z., Wang, K.: Scalable collaborative filtering using incremental update and local link prediction. In: CIKM 2012, pp. 2371–2374 (2012)Google Scholar
  11. 11.
    Zhang, S., Wang, W., Ford, J., Makedon, F.: Learning from incomplete ratings using non-negative matrix factorization. In: SDM 2006 (2006)Google Scholar
  12. 12.
    Zhang, T., Ramakrishnan, R., Livny, M.: Birch: an efficient data clustering method for very large databases. In: SIGMOD 1996, pp. 103–114 (1996)Google Scholar
  13. 13.
    Zhang, Y., Wang, X., Yu, H., Li, R., Wei, B., Pan, J.: When personalization meets socialization: an icadal approach. In: JCDL 2011, pp. 459–460 (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2013

Authors and Affiliations

  • Yuqi Wang
    • 1
  • Yin Zhang
    • 1
  • Yanfei Yin
    • 1
  • Deng Yi
    • 1
  • Baogang Wei
    • 1
  1. 1.College of Computer ScienceZhejiang UniversityHangzhouChina

Personalised recommendations