AspeRa: Aspect-Based Rating Prediction Model

  • Sergey I. Nikolenko
  • Elena Tutubalina
  • Valentin Malykh
  • Ilya Shenbin
  • Anton AlekseevEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11438)


We propose a novel end-to-end Aspect-based Rating Prediction model (AspeRa) that estimates user rating based on review texts for the items and at the same time discovers coherent aspects of reviews that can be used to explain predictions or profile users. The AspeRa model uses max-margin losses for joint item and user embedding learning and a dual-headed architecture; it significantly outperforms recently proposed state-of-the-art models such as DeepCoNN, HFT, NARRE, and TransRev on two real world data sets of user reviews. With qualitative examination of the aspects and quantitative evaluation of rating prediction models based on these aspects, we show how aspect embeddings can be used in a recommender system.


Aspect-based sentiment analysis Recommender systems Aspect-based recommendation Explainable recommendation User reviews Neural network Deep learning 



This research was done at the Samsung-PDMI Joint AI Center at PDMI RAS and was supported by Samsung Research.


  1. 1.
    Alekseev, A., Nikolenko, S.: Word embeddings for user profiling in online social networks. Computación y Sistemas 21(2) (2017)Google Scholar
  2. 2.
    Bouma, G.: Normalized (pointwise) mutual information in collocation extraction. In: Proceedings of GSCL, pp. 31–40 (2009)Google Scholar
  3. 3.
    Catherine, R., Cohen, W.: Transnets: learning to transform for recommendation. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, pp. 288–296. ACM (2017)Google Scholar
  4. 4.
    Chen, C., Zhang, M., Liu, Y., Ma, S.: Neural attentional rating regression with review-level explanations. In: Proceedings of the 2018 World Wide Web Conference, WWW 2018, pp. 1583–1592. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland (2018).
  5. 5.
    García-Durán, A., Gonzalez, R., Oñoro-Rubio, D., Niepert, M., Li, H.: Transrev: modeling reviews as translations from users to items. CoRR abs/1801.10095 (2018).
  6. 6.
    Goldberg, Y.: A primer on neural network models for natural language processing. CoRR abs/1510.00726 (2015).
  7. 7.
    He, R., Lee, W.S., Ng, H.T., Dahlmeier, D.: An unsupervised neural attention model for aspect extraction. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), vol. 1, pp. 388–397 (2017)Google Scholar
  8. 8.
    He, R., McAuley, J.: Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. In: Proceedings of the 25th International Conference on World Wide Web, pp. 507–517. International World Wide Web Conferences Steering Committee (2016)Google Scholar
  9. 9.
    Hoffman, M., Bach, F.R., Blei, D.M.: Online learning for latent dirichlet allocation. In: Advances in Neural Information Processing Systems, pp. 856–864 (2010)Google Scholar
  10. 10.
    Hsieh, C.K., Yang, L., Cui, Y., Lin, T.Y., Belongie, S., Estrin, D.: Collaborative metric learning. In: Proceedings of the 26th International Conference on World Wide Web, pp. 193–201. International World Wide Web Conferences Steering Committee (2017)Google Scholar
  11. 11.
    Kádár, A., Chrupała, G., Alishahi, A.: Representation of linguistic form and function in recurrent neural networks. Comput. Linguist. 43(4), 761–780 (2017)MathSciNetCrossRefGoogle Scholar
  12. 12.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2014).
  13. 13.
    Koren, Y., Bell, R.M., Volinsky, C.: Matrix factorization techniques for recommender systems. IEEE Comput. 42(8), 30–37 (2009)CrossRefGoogle Scholar
  14. 14.
    Lau, J.H., Newman, D., Baldwin, T.: Machine reading tea leaves: automatically evaluating topic coherence and topic model quality. In: Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pp. 530–539 (2014)Google Scholar
  15. 15.
    Liu, B.: Sentiment Analysis and Opinion Mining, Synthesis Lectures on Human Language Technologies, vol. 5. Morgan & Claypool Publishers, San Rafael (2012)Google Scholar
  16. 16.
    Lloyd, S.: Least squares quantization in PCM. IEEE Trans. Inf. Theory 28(2), 129–137 (1982)MathSciNetCrossRefGoogle Scholar
  17. 17.
    MacQueen, J., et al.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Oakland, CA, USA, vol. 1, pp. 281–297 (1967)Google Scholar
  18. 18.
    McAuley, J., Leskovec, J.: Hidden factors and hidden topics: understanding rating dimensions with review text. In: Proceedings of the 7th ACM Conference on Recommender Systems, pp. 165–172 (2013)Google Scholar
  19. 19.
    McAuley, J., Targett, C., Shi, Q., Van Den Hengel, A.: Image-based recommendations on styles and substitutes. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 43–52. ACM (2015)Google Scholar
  20. 20.
    Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. CoRR abs/1301.3781 (2013).
  21. 21.
    Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: Burges, C.J.C., Bottou, L., Welling, M., Ghahramani, Z., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 26, pp. 3111–3119. Curran Associates, Inc. (2013).
  22. 22.
    Mitcheltree, C., Wharton, V., Saluja, A.: Using aspect extraction approaches to generate review summaries and user profiles. arXiv preprint arXiv:1804.08666 (2018)
  23. 23.
    Newman, D., Karimi, S., Cavedon, L.: External evaluation of topic models. In: Australasian Document Computing Symposium, 2009. Citeseer (2009)Google Scholar
  24. 24.
    Newman, D., Lau, J.H., Grieser, K., Baldwin, T.: Automatic evaluation of topic coherence. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, HLT 2010, pp. 100–108. Association for Computational Linguistics, Stroudsburg, PA, USA (2010).
  25. 25.
    Alekseev, A., Nikolenko, S.: User profiling in text-based recommender systems based on distributed word representations. In: Ignatov, D.I., et al. (eds.) AIST 2016. CCIS, vol. 661, pp. 196–207. Springer, Cham (2017). Scholar
  26. 26.
    Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends Inf. Retrieval 2(1–2), 1–135 (2008)CrossRefGoogle Scholar
  27. 27.
    Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543. Association for Computational Linguistics, Doha, Qatar (2014).
  28. 28.
    Radford, A., Jozefowicz, R., Sutskever, I.: Learning to generate reviews and discovering sentiment. arXiv preprint arXiv:1704.01444 (2017)
  29. 29.
    Rehurek, R., Sojka, P.: Software framework for topic modelling with large corpora. In: Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pp. 45–50. ELRA, Valletta, Malta, May 2010.
  30. 30.
    Seo, S., Huang, J., Yang, H., Liu, Y.: Interpretable convolutional neural networks with dual local and global attention for review rating prediction. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, pp. 297–305. ACM (2017)Google Scholar
  31. 31.
    Seo, S., Huang, J., Yang, H., Liu, Y.: Representation learning of users and items for review rating prediction using attention-based convolutional neural network. In: 3rd International Workshop on Machine Learning Methods for Recommender Systems (MLRec) (SDM 2017) (2017)Google Scholar
  32. 32.
    Solovyev, V., Ivanov, V.: Dictionary-based problem phrase extraction from user reviews. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds.) TSD 2014. LNCS (LNAI), vol. 8655, pp. 225–232. Springer, Cham (2014). Scholar
  33. 33.
    Srivastava, A., Sutton, C.: Autoencoding variational inference for topic models. arXiv preprint arXiv:1703.01488 (2017)
  34. 34.
    Steinhaus, H.: Sur la division des corp materiels en parties. Bull. Acad. Polon. Sci 1(804), 801 (1956)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Tutubalina, E., Nikolenko, S.: Inferring sentiment-based priors in topic models. In: Lagunas, O.P., Alcántara, O.H., Figueroa, G.A. (eds.) MICAI 2015. LNCS (LNAI), vol. 9414, pp. 92–104. Springer, Cham (2015). Scholar
  36. 36.
    Tutubalina, E., Nikolenko, S.: Constructing aspect-based sentiment lexicons with topic modeling. In: Ignatov, D.I., et al. (eds.) AIST 2016. CCIS, vol. 661, pp. 208–220. Springer, Cham (2017). Scholar
  37. 37.
    Wang, C., Blei, D.M.: Collaborative topic modeling for recommending scientific articles. In: Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 448–456. ACM (2011)Google Scholar
  38. 38.
    Wang, H., Wang, N., Yeung, D.Y.: Collaborative deep learning for recommender systems. In: Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1235–1244. ACM (2015)Google Scholar
  39. 39.
    Weston, J., Bengio, S., Usunier, N.: WSABIE: scaling up to large vocabulary image annotation. IJCAI 11, 2764–2770 (2011)Google Scholar
  40. 40.
    Zhang, S., Wang, W., Ford, J., Makedon, F.: Learning from incomplete ratings using non-negative matrix factorization. In: Proceedings of the 2006 SIAM International Conference on Data Mining, pp. 549–553. SIAM (2006)Google Scholar
  41. 41.
    Zheng, L., Noroozi, V., Yu, P.S.: Joint deep modeling of users and items using reviews for recommendation. In: Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, pp. 425–434. ACM (2017)Google Scholar
  42. 42.
    Zhu, X., Blei, D., Lafferty, J.: TagLDA: bringing document structure knowledge into topic models. Tech. report, UWisc Technical Report TR-1533 (2006).

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Sergey I. Nikolenko
    • 1
    • 4
  • Elena Tutubalina
    • 1
    • 2
    • 4
  • Valentin Malykh
    • 1
    • 3
  • Ilya Shenbin
    • 1
  • Anton Alekseev
    • 1
    Email author
  1. 1.Samsung-PDMI Joint AI CenterSteklov Mathematical Institute at St. PetersburgSaint PetersburgRussia
  2. 2.Chemoinformatics and Molecular Modeling LaboratoryKazan Federal UniversityKazanRussia
  3. 3.Neural Systems and Deep Learning LaboratoryMoscow Institute of Physics and TechnologyDolgoprudnyRussia
  4. 4.Neuromation OUTallinnEstonia

Personalised recommendations