Embedding metadata using deep collaborative filtering to address the cold start problem for the rating prediction task

Multimedia Tools and Applications (2021)Cite this article

Abstract

In recent years, deep learning has yielded success in many research fields including machine translation, natural language processing, computer vision, and social network filtering. The area of deep learning in the recommender system is flourishing. Previous research has relied on incorporating metadata information in various application domains using deep learning techniques to achieve better recommendation accuracy. The use of metadata is desirable to address the cold start problem and better learning the user-item interaction, which is not captured by the user-item rating matrix. Existing methods rely on fixed user-item latent representation and ignore the metadata information. It restricts the model performance to correctly identify actual latent vectors, which results in high rating prediction error. To tackle these problems, we propose a generalized recommendation model named Meta Embedding Deep Collaborative Filtering (MEDCF), which inputs user demographics and item genre as metadata features together with the rating matrix. The proposed framework primarily comprises of Generalized Matrix Factorization (GMF), Multilayer Perceptron (MLP), and Neural Matrix Factorization (NeuMF) methods. GMF is applied to the rating matrix, whereas MLP is applied to metadata. Using NeuMF, the outputs for GMF and MLP are then concatenated and input to a neural network for rating prediction. To prove the effectiveness of proposed model, two metrics are used, Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). The MEDCF model is experimented on MovieLens and Amazon Movies datasets showing a significant improvement over the baseline methods.

This is a preview of subscription content, access via your institution.

We’re sorry, something doesn't seem to be working properly.

Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Notes

  1. 1.

    https://grouplens.org/datasets/movielens/100K/.

  2. 2.

    https://grouplens.org/datasets/movielens/1M/.

  3. 3.

    http://jmcauley.ucsd.edu/data/amazon/.

  4. 4.

    https://keras.io/.

References

  1. 1.

    Al-Shamri MYH (2016) User profiling approaches for demographic recommender systems. Knowl-Based Syst 100:175–187

    Article  Google Scholar 

  2. 2.

    Bai T, Wen JR, Zhang J, Zhao WX (2017) A neural collaborative filtering model with interaction-based neighborhood. In: Proceedings of the 2017 ACM on conference on information and knowledge management. ACM, pp 1979–1982

  3. 3.

    Bennett J, Lanning S, et al. (2007) The netflix prize. In: Proceedings of KDD cup and workshop. Citeseer, vol 2007, pp 35

  4. 4.

    Callvik J, Liu A (2017) Using demographic information to reduce the new user problem in recommender systems. KTH, School of Computer Science and Communication (CSC)

  5. 5.

    Cheng Z, Chang X, Zhu L, Kanjirathinkal RC, Kankanhalli M (2019) Mmalfm: Explainable recommendation by leveraging reviews and images. ACM Trans Inf Syst (TOIS) 37(2):1–28

    Article  Google Scholar 

  6. 6.

    Cheng Z, Ding Y, He X, Zhu L, Song X, Kankanhalli MS (2018) A3̂ncf: An adaptive aspect attention model for rating prediction. In: IJCAI, pp 3748–3754

  7. 7.

    D’Addio RM, Marinho RS, Manzato MG (2019) Combining different metadata views for better recommendation accuracy. Inf Syst 83:1–12

    Article  Google Scholar 

  8. 8.

    Do TDT, Cao L (2018) Coupled poisson factorization integrated with user/item metadata for modeling popular and sparse ratings in scalable recommendation. In: Thirty-second AAAI conference on artificial intelligence

  9. 9.

    Dong X, Yu L, Wu Z, Sun Y, Yuan L, Zhang F (2017) A hybrid collaborative filtering model with deep structure for recommender systems. In: Proceedings of the thirty-first AAAI conference on artificial intelligence, pp 1309–1315

  10. 10.

    Dureddy HV, Kaden Z (2018) Handling cold-start collaborative filtering with reinforcement learning. arXiv:180606192

  11. 11.

    Ekstrand MD, Tian M, Azpiazu IM, Ekstrand JD, Anuyah O, McNeill D, Pera MS (2018) All the cool kids, how do they fit in?: Popularity and demographic biases in recommender evaluation and effectiveness. In: Conference on fairness, accountability and transparency, pp 172–186

  12. 12.

    Elkahky AM, Song Y, He X (2015) A multi-view deep learning approach for cross domain user modeling in recommendation systems. In: Proceedings of the 24th international conference on World Wide Web, international World Wide Web conferences steering committee, pp 278–288

  13. 13.

    Fernández-Tobías I, Cantador I, Tomeo P, Anelli VW, Di Noia T (2019) Addressing the user cold start with cross-domain collaborative filtering: exploiting item metadata in matrix factorization. User Model User-Adap Inter 29 (2):443–486

    Article  Google Scholar 

  14. 14.

    Gleichman S, Eldar YC (2011) Blind compressed sensing. IEEE Trans Inf Theory 57(10):6958–6975

    MathSciNet  Article  Google Scholar 

  15. 15.

    Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp 315–323

  16. 16.

    Gogna A, Majumdar A (2015) Blind compressive sensing framework for collaborative filtering. arXiv:150501621

  17. 17.

    Guan X, Cheng Z, He X, Zhang Y, Zhu Z, Peng Q, Chua TS (2019) Attentive aspect modeling for review-aware recommendation. ACM Trans Inf Syst (TOIS) 37(3):1–27

    Article  Google Scholar 

  18. 18.

    Gunawardana A, Shani G (2009) A survey of accuracy evaluation metrics of recommendation tasks. J Mach Learn Res 10:2935–2962

    MathSciNet  MATH  Google Scholar 

  19. 19.

    Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems, pp 1024–1034

  20. 20.

    Hastie T, Mazumder R, Lee JD, Zadeh R (2015) Matrix completion and low-rank svd via fast alternating least squares. J Mach Learn Res 16 (1):3367–3402

    MathSciNet  MATH  Google Scholar 

  21. 21.

    He X, Deng K, Wang X, Li Y, Zhang Y, Wang M (2020) Lightgcn: Simplifying and powering graph convolution network for recommendation. arXiv:200202126

  22. 22.

    He X, Liao L, Zhang H, Nie L, Hu X, Chua TS (2017) Neural collaborative filtering. In: Proceedings of the 26th international conference on world wide web, international world wide web conferences steering committee, pp 173–182

  23. 23.

    He X, Zhang H, Kan MY, Chua TS (2016) Fast matrix factorization for online recommendation with implicit feedback. In: Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval. ACM, pp 549–558

  24. 24.

    He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  25. 25.

    Henk V, Vahdati S, Nayyeri M, Ali M, Yazdi HS, Lehmann J (2019) Metaresearch recommendations using knowledge graph embeddings. In: RecNLP workshop of AAAI conference

  26. 26.

    Hornik K, Stinchcombe M, White H (1989) Multilayer feedforward networks are universal approximators. Neural Netw 2(5):359–366

    Article  Google Scholar 

  27. 27.

    Hsieh CK, Yang L, Cui Y, Lin TY, Belongie S, Estrin D (2017) Collaborative metric learning. In: Proceedings of the 26th international conference on world wide web, pp 193–201

  28. 28.

    Hu Y, Koren Y, Volinsky C (2008) Collaborative filtering for implicit feedback datasets. In: 2008 Eighth IEEE international conference on data mining. IEEE, pp 263–272

  29. 29.

    Kim KS, Chang DS, Choi YS (2019) Boosting memory-based collaborative filtering using content-metadata. Symmetry 11(4):561

    Article  Google Scholar 

  30. 30.

    Kim D, Park C, Oh J, Lee S, Yu H (2016) Convolutional matrix factorization for document context-aware recommendation. In: Proceedings of the 10th ACM conference on recommender systems. ACM, pp 233–240

  31. 31.

    Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv:14126980

  32. 32.

    Koren Y (2008) Factorization meets the neighborhood: a multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 426–434

  33. 33.

    Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37

    Article  Google Scholar 

  34. 34.

    Kula M (2015) Metadata embeddings for user and item cold-start recommendations. arXiv:150708439

  35. 35.

    Li D, Chen C, Liu W, Lu T, Gu N, Chu S (2017) Mixture-rank matrix approximation for collaborative filtering. In: Guyon I, Luxburg U V, Bengio S, Wallach H, Fergus R, Vishwanathan S, Garnett R (eds) Advances in neural information processing systems. Curran Associates Inc., vol 30, pp 477–485

  36. 36.

    Liu F, Cheng Z, Sun C, Wang Y, Nie L, Kankanhalli M (2019b) User diverse preference modeling by multimodal attentive metric learning. In: Proceedings of the 27th ACM international conference on multimedia, pp 1526–1534

  37. 37.

    Liu D, Li J, Du B, Chang J, Gao R (2019a) Daml: Dual attention mutual learning between ratings and reviews for item recommendation. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 344–352

  38. 38.

    Liu T, Wang Z, Tang J, Yang S, Huang GY, Liu Z (2019) Recommender systems with heterogeneous side information. In: The World Wide Web conference, pp 3027–3033

  39. 39.

    Luo X, Yuan Y, Zhou M, Liu Z, Shang M (2019) Non-negative latent factor model based on β-divergence for recommender systems. IEEE Trans Syst Man Cybern Syst, 1–12. https://doi.org/10.1109/TSMC.2019.2931468

  40. 40.

    Luo X, Zhou M, Li S, Hu L, Shang M (2019) Non-negativity constrained missing data estimation for high-dimensional and sparse matrices from industrial applications. IEEE Trans Cyber 50(5):1844–1855

    Article  Google Scholar 

  41. 41.

    Ma H, Yang H, Lyu MR, King I (2008) Sorec: social recommendation using probabilistic matrix factorization. In: Proceedings of the 17th ACM conference on information and knowledge management. ACM, pp 931–940

  42. 42.

    McAuley J, Leskovec J (2013) Hidden factors and hidden topics: understanding rating dimensions with review text. In: Proceedings of the 7th ACM conference on recommender systems, pp 165–172

  43. 43.

    Mittal P, Jain A, Majumdar A (2014) Metadata based recommender systems. In: 2014 International conference on advances in computing, communications and informatics (ICACCI). IEEE, pp 2659–2664

  44. 44.

    Mnih A, Salakhutdinov RR (2008) Probabilistic matrix factorization. In: Advances in neural information processing systems, pp 1257–1264

  45. 45.

    Rendle S (2010) Factorization machines. In: IEEE International conference on data mining. IEEE, pp 995–1000

  46. 46.

    Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L (2009) Bpr: Bayesian personalized ranking from implicit feedback. In: Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence. AUAI Press, pp 452–461

  47. 47.

    Sarwar B, Karypis G, Konstan J, Riedl J (2002) Incremental singular value decomposition algorithms for highly scalable recommender systems. In: Fifth international conference on computer and information science. Citeseer, vol 27, pp 28

  48. 48.

    Sedhain S, Menon AK, Sanner S, Xie L (2015) Autorec: Autoencoders meet collaborative filtering. In: Proceedings of the 24th international conference on World Wide Web. ACM, pp 111–112

  49. 49.

    Shang M, Luo X, Liu Z, Chen J, Yuan Y, Zhou M (2018b) Randomized latent factor model for high-dimensional and sparse matrices from industrial applications. IEEE/CAA J Autom Sin 6(1):131–141

    MathSciNet  Article  Google Scholar 

  50. 50.

    Shang J, Sun M, Collins-Thompson K (2018a) Demographic inference via knowledge transfer in cross-domain recommender systems. In: 2018 IEEE international conference on data mining (ICDM). IEEE, 1218–1223

  51. 51.

    Singh AP, Gordon GJ (2008) Relational learning via collective matrix factorization. In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 650–658

  52. 52.

    Soares M, Viana P (2015) Tuning metadata for better movie content-based recommendation systems. Multimed Tools Appl 74(17):7015–7036

    Article  Google Scholar 

  53. 53.

    Socher R, Chen D, Manning CD, Ng A (2013) Reasoning with neural tensor networks for knowledge base completion. In: Advances in neural information processing systems, pp 926–934

  54. 54.

    Srebro N, Rennie J, Jaakkola TS (2005) Maximum-margin matrix factorization. In: Advances in neural information processing systems, pp 1329–1336

  55. 55.

    Sridevi M, Rao RR (2017) Decors: A simple and efficient demographic collaborative recommender system for movie recommendation. Adv Comput Sci Technol 10(7):1969–1979

    Google Scholar 

  56. 56.

    Srivastava N, Salakhutdinov RR (2012) Multimodal learning with deep boltzmann machines. In: Advances in neural information processing systems, pp 2222–2230

  57. 57.

    Strub F, Gaudel R, Mary J (2016) Hybrid recommender system based on autoencoders. In: Proceedings of the 1st workshop on deep learning for recommender systems. https://doi.org/10.1145/2988450.2988456. ACM, New York, NY, USA, DLRS 2016, pp 11–16

  58. 58.

    Tang D, Qin B, Liu T, Yang Y (2015) User modeling with neural network for review rating prediction. In: Twenty-fourth international joint conference on artificial intelligence

  59. 59.

    Vasile F, Smirnova E, Conneau A (2016) Meta-prod2vec: Product embeddings using side-information for recommendation. In: Proceedings of the 10th ACM conference on recommender systems. ACM, pp 225–232

  60. 60.

    Vozalis M, Margaritis KG (2004) Collaborative filtering enhanced by demographic correlation. In: AIAI symposium on professional practice in AI, of the 18th world computer congress

  61. 61.

    Wang C, Blei DM (2011) Collaborative topic modeling for recommending scientific articles. In: Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, 448–456

  62. 62.

    Wang M, Fu W, Hao S, Tao D, Wu X (2016) Scalable semi-supervised learning by efficient anchor graph regularization. IEEE Trans Knowl Data Eng 28(7):1864–1877

    Article  Google Scholar 

  63. 63.

    Wang X, He X, Wang M, Feng F, Chua TS (2019) Neural graph collaborative filtering. In: Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pp 165–174

  64. 64.

    Wang S, Tang J, Wang Y, Liu H (2018) Exploring hierarchical structures for recommender systems. IEEE Trans Knowl Data Eng 30(6):1022–1035

    Article  Google Scholar 

  65. 65.

    Wang H, Wang N, Yeung DY (2015) Collaborative deep learning for recommender systems. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 1235–1244

  66. 66.

    Wu Y, DuBois C, Zheng AX, Ester M (2016) Collaborative denoising auto-encoders for top-n recommender systems. In: Proceedings of the ninth ACM international conference on Web search and data mining. ACM, pp 153–162

  67. 67.

    Wu D, Luo X, Shang M, He Y, Wang G, Zhou M (2019) A deep latent factor model for high-dimensional and sparse matrices in recommender systems. IEEE Trans Syst Man Cybernet Syst, 1–12. https://doi.org/10.1109/TSMC.2019.2931393

  68. 68.

    Xiao T, Liang S, Shen W, Meng Z (2019) Bayesian deep collaborative matrix factorization. In: Proceedings of the thirty-third AAAI conference on artificial intelligence (AAAI 2019). AAAI

  69. 69.

    Yoon YC, Lee JW (2018) Movie recommendation using metadata based word2vec algorithm. In: 2018 international conference on platform technology and service (PlatCon). IEEE pp 1–6

  70. 70.

    Zhang H, Shen F, Liu W, He X, Luan H, Chua TS (2016) Discrete collaborative filtering. In: Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval. ACM, pp 325–334

  71. 71.

    Zhang H, Yang Y, Luan H, Yang S, Chua TS (2014) Start from scratch: Towards automatically identifying, modeling, and naming visual attributes. In: Proceedings of the 22nd ACM international conference on multimedia. ACM, pp 187–196

Download references

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Affiliations

  1. Department of Computer Science and Engineering, Malaviya National Institute of Technology, Jaipur, Rajasthan, India

    Ravi Nahta, Yogesh Kumar Meena, Dinesh Gopalani & Ganpat Singh Chauhan

Authors
  1. Ravi Nahta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yogesh Kumar Meena
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dinesh Gopalani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ganpat Singh Chauhan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ravi Nahta.

Ethics declarations

Conflict of Interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Human and Animal Rights Statements

All authors declared that there is no research involving human and/or animal.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Nahta, R., Meena, Y.K., Gopalani, D. et al. Embedding metadata using deep collaborative filtering to address the cold start problem for the rating prediction task. Multimed Tools Appl (2021). https://doi.org/10.1007/s11042-021-10529-4

Download citation

Keywords

  • Recommender systems
  • Collaborative filtering
  • Cold start problem
  • Neural networks
  • Matrix factorization
  • Metadata