Data Mining and Knowledge Discovery

, Volume 32, Issue 3, pp 651–674 | Cite as

An optimization model for collaborative recommendation using a covariance-based regularizer



This paper suggests a convex regularized optimization model to produce recommendations, which is adaptable, fast, and scalable—while remaining very competitive to state-of-the-art methods in terms of accuracy. We introduce a regularizer based on the covariance matrix such that the model minimizes two measures ensuring that the recommendations provided to a user are guided by both the preferences of the other users in the system and the known preferences of the user being processed. It is adaptable since (1) it can be viewed from both user and item perspectives (allowing to choose, depending on the task, the formulation with fewer decision variables) and (2) multiple constraints depending on the context (and not only based on the accuracy, but also on the utility of personalized recommendations) can easily be added, as shown in this paper through two examples. Since our regularizer is based on the covariance matrix, this paper also describes how to improve computational and space complexities by using matrix factorization techniques in the optimization model, leading to a fast and scalable model. To illustrate all these concepts, experiments were conducted on four real datasets of different sizes (i.e., FilmTrust, Ciao, MovieLens, and Netflix) and comparisons with state-of-the-art methods are provided, showing that our context-sensitive approach is very competitive in terms of accuracy.


Recommender system Collaborative filtering Regularizer Convex optimization Matrix factorization 



The authors would like to thank Marco Saerens, Nicolas Gillis, and Arnaud Vandaele for insightful comments on this work.


  1. Adomavicius G, Tuzhilin A (2005) Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans Knowl Data Eng 17(6):734–749CrossRefGoogle Scholar
  2. Agarwal D, Chen BC, Elango P, Wang X (2011) Click shaping to optimize multiple objectives. In: Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining (KDD’11), pp 132–140Google Scholar
  3. Armstrong J (2001) Principles of forecasting, a handbook for researchers and practitioners. Kluwer Academic, NorwellCrossRefGoogle Scholar
  4. Bennett J, Lanning S (2007) The Netflix prize. In: KDD Cup and Workshop in conjunction with KDDGoogle Scholar
  5. Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge University Press, CambridgeCrossRefMATHGoogle Scholar
  6. Cremonesi P, Koren Y, Turrin R (2010) Performance of recommender algorithms on top-n recommendation tasks. In: Proceedings of the fourth ACM conference on recommender systems (RecSys ’10), pp 39–46Google Scholar
  7. Devooght R, Kourtellis N, Mantrach A (2015) Dynamic matrix factorization with priors on unknown values. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp 189–198Google Scholar
  8. Deza M, Deza E (2014) Encyclopedia of distances, 3rd edn. Springer, HeidelbergMATHGoogle Scholar
  9. Feldman S (2012) Multi-task averaging: theory and practice. PhD thesis, University of WashingtonGoogle Scholar
  10. Fouss F, Francoisse K, Yen L, Pirotte A, Saerens M (2012) An experimental investigation of kernels on graphs for collaborative recommendation and semi-supervised classification. Neural Netw 31:53–72CrossRefMATHGoogle Scholar
  11. Fouss F, Saerens M, Shimbo M (2016) Algorithms and models for network data and link analysis. Cambridge University Press, CambridgeCrossRefGoogle Scholar
  12. Gantner Z, Rendle S, Freudenthaler C, Schmidt-Thieme L (2011) MyMediaLite: a free recommender system library. In: Proceedings of the 5th ACM conference on recommender systems (RecSys 2011)Google Scholar
  13. Gill PE, Murray W, Wright MH (1981) Practical optimization. Academic Press, LondonMATHGoogle Scholar
  14. Gillis N (2011) Nonnegative matrix factorization: complexity, algorithms and applications. PhD thesis, Université catholique de LouvainGoogle Scholar
  15. Gillis N (2014) The why and how of nonnegative matrix factorization. In: Suykens JAK, Signoretto M, Argyriou A (eds) Regularization, optimization, kernels, and support vector machines. Chapman & Hall/CRC, Machine Learning and Pattern Recognition Series, pp 257–291Google Scholar
  16. Gillis N, Glineur F (2012) Accelerated multiplicative updates and hierarchical Als algorithms for nonnegative matrix factorization. Neural Comput 24(4):1085–1105MathSciNetCrossRefGoogle Scholar
  17. Gu Q, Zhou J, Ding C (2010) Collaborative filtering: weighted nonnegative matrix factorization incorporating user and item graphs. In: Proceedings of the 2010 SIAM international conference on data mining, pp 199–210Google Scholar
  18. Guo G, Zhang J, Yorke-Smith N (2013) A novel bayesian similarity measure for recommender systems. In: Proceedings of the 23rd international joint conference on artificial intelligence (IJCAI), pp 2619–2625Google Scholar
  19. Herlocker J, Konstan J, Riedl J (2002) An empirical analysis of design choices in neighborhood-based collaborative filtering algorithms. Inf Retr 5:287–310CrossRefGoogle Scholar
  20. Hill W, Stead L, Rosenstein M, Furnas G (1995) Recommending and evaluating choices in a virtual community of use. In: Proceedings of ACM CHI’95 conference on human factors in computing systems, pp 194–201Google Scholar
  21. Hsieh CJ, Olsen P (2014) Nuclear norm minimization via active subspace selection. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 575–583Google Scholar
  22. Hu Y, Koren Y, Volinsky C (2008) Collaborative filtering for implicit feedback datasets. In: Proceedings of the Eighth IEEE international conference on data mining (ICDM), pp 263–272Google Scholar
  23. Jambor T, Wang J (2010) Optimizing multiple objectives in collaborative filtering. In: Proceedings of the fourth ACM conference on recommender systems (RecSys ’10), pp 55–62Google Scholar
  24. Järvelin K, Kekäläinen J (2002) Cumulated gain-based evaluation of ir techniques. ACM Trans Inf Syst 20(4):422–446CrossRefGoogle Scholar
  25. Johnson R, Wichern D (2002) Applied multivariate statistical analysis, 5th edn. Prentice Hall, Englewood CliffsMATHGoogle Scholar
  26. Kannan R, Ishteva M, Park H (2012) Bounded matrix low rank approximation. In: Proceedings of the 12th IEEE international conference on data mining (ICDM), pp 319–328Google Scholar
  27. Kim D, Yum BJ (2005) Collaborative filtering based on iterative principal component analysis. Expert Syst Appl 28(4):823–830CrossRefGoogle Scholar
  28. Koren Y (2008) Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining (KDD’08), pp 426–434Google Scholar
  29. Koren Y (2009) Collaborative filtering with temporal dynamics. In: Proceedings of the 15th ACM SIGKDD international conference on knowledge discovery and data mining (KDD ’09), pp 447–456Google Scholar
  30. Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37CrossRefGoogle Scholar
  31. Lilien G, Smith B, Moorthy K (1992) Marketing models. Prentice Hall, Englewood CliffsGoogle Scholar
  32. Lü L, Medo M, Yeung CH, Zhang YC, Zhang ZK, Zhou T (2012) Recommender systems. Phys Rep 519:1–49CrossRefGoogle Scholar
  33. Luo X, Zhou M, Xia Y, Zhu Q (2014) An efficient non-negative matrix-factorization-based approach to collaborative filtering for recommender systems. IEEE Trans Ind Inf 10(2):1273–1284CrossRefGoogle Scholar
  34. Murthi B, Sarkar S (2003) The role of the management sciences in research on personalization. Manag Sci 49(10):1344–1362CrossRefGoogle Scholar
  35. Ning X, Karypis G (2011) SLIM: sparse linear methods for top-n recommender systems. In: Proceedings of the IEEE international conference on data mining (ICDM), pp 497–506Google Scholar
  36. Ning X, Desrosiers C, Karypis G (2015) A comprehensive survey of neighborhood-based recommendation methods. Springer, Heidelberg, pp 37–76Google Scholar
  37. Paterek A (2007) Improving regularized singular value decomposition for collaborative filtering. In: Proceedings of KDD cup and workshop, pp 39–42Google Scholar
  38. Rao N, Yu H-F, Ravikumar P, Dhillon IS (2015) Collaborative filtering with graph information: consistency and scalable methods. In: Cortes C, Lee DD, Sugiyama M, Garnett R (eds) Proceedings of the 28th international conference on neural information processing systems, vol 2, MIT Press, Cambridge, MA, pp 2107–2115Google Scholar
  39. Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L (2009) BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence (AUAI), pp 452–461Google Scholar
  40. Resnick P, Neophytos I, Mitesh S, Bergstrom P, Riedl J (1994) GroupLens: an open architecture for collaborative filtering of netnews. In: Proceedings of the conference on computer supported cooperative work, pp 175–186Google Scholar
  41. Rich E (1979) User modeling via stereotypes. Cogn Sci 3(4):329–354CrossRefGoogle Scholar
  42. Rodriguez M, Posse C, Zhang E (2012) Multiple objective optimization in recommender systems. In: Proceedings of the Sixth ACM conference on recommender systems (RecSys ’12), pp 11–18Google Scholar
  43. Salakhutdinov R, Mnih A (2008) Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In: Proceedings of the 25th international conference on machine learning (ICML ’08), pp 880–887Google Scholar
  44. Salton G (1989) Automatic text processing. Addison-Wesley, ReadingGoogle Scholar
  45. Shan H, Banerjee A (2010) Generalized probabilistic matrix factorizations for collaborative filtering. In: Proceedings of the 10th IEEE international conference on data mining (ICDM), pp 1025–1030Google Scholar
  46. Shani G, Gunawardana A (2011) Evaluating recommendation systems. In: Ricci F, Rokach L, Shapira B, Kantor P (eds) Recommender systems handbook. Springer, Boston, MA, pp 257–297CrossRefGoogle Scholar
  47. Shardanand U, Maes P (1995) Social information filtering: algorithms for automating ’word of mouth’. In: Proceedings of the conference on human factors in computing systems, pp 210–217Google Scholar
  48. Smola AJ, Kondor R (2003) Kernels and regularization on graphs. Learning theory and kernel machines, vol 2777. Lecture notes in computer science. Springer, Berlin, pp 144–158MATHGoogle Scholar
  49. Tang J, Gao H, Liu H (2012) mTrust: discerning multi-faceted trust in a connected world. In: Proceedings of the fifth ACM international conference on web search and data mining, pp 93–102Google Scholar
  50. Yu K, Zhu S, Lafferty J, Gong Y (2009) Fast nonparametric matrix factorization for large-scale collaborative filtering. In: Proceedings of the 32nd international ACM SIGIR conference on research and development in information retrieval (SIGIR ’09), pp 211–218Google Scholar
  51. Zhang S, Wang W, Ford J, Makedon F (2006) Learning from incomplete ratings using non-negative matrix factorization. In: Proceedings of the 2006 SIAM international conference on data mining, pp 549–553Google Scholar
  52. Zhang Y, Chen W, Yin Z (2013) Collaborative filtering with social regularization for TV program recommendation. Knowl Based Syst 54:310–317CrossRefGoogle Scholar

Copyright information

© The Author(s) 2018

Authors and Affiliations

  1. 1.Department of Engineering Innovation Management, Faculty of EngineeringUniversity of MonsMonsBelgium
  2. 2.Louvain School of Management (LSM), Louvain Research Institute in Management and Organizations (LouRIM)Université catholique de Louvain (UCL)MonsBelgium

Personalised recommendations