Advertisement

Dynamic attention-integrated neural network for session-based news recommendation

  • Lemei ZhangEmail author
  • Peng Liu
  • Jon Atle Gulla
Article
  • 111 Downloads
Part of the following topical collections:
  1. Special Issue of the ACML 2018 Journal Track

Abstract

Online news recommendation aims to continuously select a pool of candidate articles that meet the temporal dynamics of user preferences. Most of the existing methods assume that all user-item interaction history are equally importance for recommendation, which is not alway applied in real-word scenario since the user-item interactions are sometime full of stochasticity and contingency. In addition, previous work on session-based algorithms only considers user sequence behaviors within current session without incorporating users’ historical interests or pointing out users’ main purposes within such session. In this paper, we propose a novel neural network framework, dynamic attention-integrated neural network, to tackle the problems. Specifically, we propose a dynamic neural network to model users’ dynamic interests over time in a unified framework for personalized news recommendations. News article semantic embedding, user interests modelling, session-based public behavior mining and an attention scheme that used to learn the attention score of user and item interaction within sessions are four key factors for online sequences mining and recommendation strategy. Experimental results on three real-world datasets show significant improvements over several baselines and state-of-the-art methods on session-based neural networks.

Keywords

Personalized news recommendation Recurrent neural networks User interest modelling Attention model 

Notes

Acknowledgements

This work is partially funded by the Research Council of Norway (No. 245469).

References

  1. Adomavicius, G., & Tuzhilin, A. (2005). Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE Transactions on Knowledge & Data Engineering, 17(6), 734–749.CrossRefGoogle Scholar
  2. Agarwal, D., & Chen, B.-C. (2009). Regression-based latent factor models. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 19–28). ACM.Google Scholar
  3. Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural machine translation by jointly learning to align and translate. ArXiv preprint arXiv:1409.0473.
  4. Bamshad, M., Dai, H., Tao, L., & Nakagawa, M. (2002). Using sequential and non-sequential patterns in predictive web usage mining tasks. In Proceedings of the IEEE international conference on data mining (ICDM ’02). IEEE Computer Society, Washington, DC, USA (pp. 669–672).Google Scholar
  5. Baur, D., Buttgen, J., & Butz, A. (2012). Listening factors: A large-scale principal components analysis of long-term music listening histories. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1273–1276). ACM.Google Scholar
  6. Blei, D. M., & Lafferty, J. D. (2006). Dynamic topic models. In Proceedings of the 23rd international conference on machine learning (pp. 113–120). ACM.Google Scholar
  7. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022.zbMATHGoogle Scholar
  8. Chen, S., Moore, J. L., Turnbull, D., & Joachims, T. (2012). Playlist prediction via metric embedding. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining (pp. 714–722). ACM.Google Scholar
  9. Chen, X., Xu, H., Zhang, Y., Tang, J., Cao, Y., Qin, Z., & Zha, H. (2018). Sequential recommendation with user memory networks. In Proceedings of the eleventh ACM international conference on web search and data mining (pp. 108–116). ACM.Google Scholar
  10. Cho, K., Van Merriënboer, B., Bahdanau, D., & Bengio, Y. (2014). On the properties of neural machine translation: Encoder–decoder approaches. ArXiv preprint arXiv:1409.1259.
  11. Cui, P., Wang, Z., & Su, Z. (2014). What videos are similar with you? Learning a common attributed representation for video recommendation. In Proceedings of the 22nd ACM international conference on multimedia (pp. 597–606). ACM.Google Scholar
  12. Das, A. S., Mayur, D., Ashutosh, G., & Shyam, R. (2007). Google news personalization: scalable online collaborative filtering. In Proceedings of the 16th international conference on world wide web (pp. 271–280). ACM.Google Scholar
  13. Ding, Y., & Li, X. (2005). Time weight collaborative filtering. In Proceedings of the 14th ACM international conference on information and knowledge management (pp. 485–492). ACM.Google Scholar
  14. Gopalan, P. K., Charlin, L., & Blei, D. (2014). Content-based recommendations with Poisson factorization. In Advances in neural information processing systems (pp. 3176–3184).Google Scholar
  15. Graves, A., Mohamed, A., & Hinton, G. (2013). Speech recognition with deep recurrent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing (ICASSP) (pp. 6645–6649). IEEE.Google Scholar
  16. Greene, D., & Cross, J. P. (2016). Exploring the political agenda of the European parliament using a dynamic topic modelling approach. ArXiv preprint arXiv:1607.03055.
  17. Grefenstette, E., Hermann, K. M., Suleyman, M., & Blunsom, P. (2015). Learning to transduce with unbounded memory. In Advances in neural information processing systems (pp. 1828–1836).Google Scholar
  18. Hidasi, B., Karatzoglou, A., Baltrunas, L., & Tikk, D. (2015). Session-based recommendations with recurrent neural networks. ArXiv preprint arXiv:1511.06939.
  19. Hidasi, B., Quadrana, M., Karatzoglou, A., & Tikk, D. (2016). Parallel recurrent neural network architectures for feature-rich session-based recommendations. In Proceedings of the 10th ACM conference on recommender systems (pp. 241–248). ACM.Google Scholar
  20. Hofmann, T. (2004). Latent semantic models for collaborative filtering. ACM Transactions on Information Systems (TOIS), 22(1), 89–115.MathSciNetCrossRefGoogle Scholar
  21. Hong, J., & Fang, M. (2015). Sentiment analysis with deeply learned distributed representations of variable length texts. Technical report, Stanford University.Google Scholar
  22. Huang, J., Zhao, W. X., Dou, H., Wen, J.-R., & Chang, E. Y. (2018). Improving sequential recommendation with knowledge-enhanced memory networks. In The 41st international ACM SIGIR conference on research and development in information retrieval (pp. 505–514). ACM.Google Scholar
  23. Jing, Z., Liu, B., Tang, J., Chen, T., & Li, J. (2013). Social influence locality for modelling retweeting behaviors. IJCAI, 13, 2761–2767.Google Scholar
  24. Jurafsky, D., & Martin, J. H. (2014). Speech and language processing (Vol. 3). London: Pearson.Google Scholar
  25. Kim, Y. (2014). Convolutional neural networks for sentence classification. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) (pp. 1746–1751).Google Scholar
  26. Koren, Y., Robert, B., & Chris, V. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8), 30–37.CrossRefGoogle Scholar
  27. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097–1105).Google Scholar
  28. Kumar, A., Irsoy, O., Ondruska, P., Iyyer, M., Bradbury, J., Gulrajani, I., Zhong, V., Paulus, R., & Socher, R. (2016). Ask me anything: Dynamic memory networks for natural language processing. In International conference on machine learning (pp. 1378–1387).Google Scholar
  29. Lai, S., Xu, L., Liu, K., & Zhao, J. (2015). Recurrent convolutional neural networks for text classification. AAAI, 333, 2267–2273.Google Scholar
  30. LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444.CrossRefGoogle Scholar
  31. Li, L., Chu, W., Langford, J., & Schapire, R. E. (2010). A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on world wide web (pp. 661–670). ACM.Google Scholar
  32. Li, J., Ren, P., Chen, Z., Ren, Z., Lian, T., & Ma, J. (2017). Neural attentive session-based recommendation. In Proceedings of the 2017 ACM on conference on information and knowledge management (pp. 1419–1428). ACM.Google Scholar
  33. Melville, P., Mooney, R. J., & Nagarajan, R. (2002). Content-boosted collaborative filtering for improved recommendations. AAAI/IAAI, 23, 187–192.Google Scholar
  34. Pan, Y., Mei, T., Yao, T., Li, H., & Rui, Y. (2016). Jointly modelling embedding and translation to bridge video and language. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4594–4602).Google Scholar
  35. Pei, W., Yang, J., Sun, Z., Zhang, J., Bozzon, A., & Tax, D. M. (2017) Interacting attention-gated recurrent networks for recommendation. In Proceedings of the 2017 ACM on conference on information and knowledge management (pp. 1459–1468). ACM.Google Scholar
  36. Powers, D. M. W. (2011). Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation. Journal of Machine Learning Technologies, 2(1), 37–63.MathSciNetGoogle Scholar
  37. Quadrana, M., Karatzoglou, A., Hidasi, B., & Cremonesi, P. (2017). Personalizing session-based recommendations with hierarchical recurrent neural networks. ArXiv preprint arXiv:1706.04148.
  38. Rao, J., Jia, A., Feng, Y., & Zhao, D. (2013). Personalized news recommendation using ontologies harvested from the web. In International conference on web-age information management (pp. 781–787). Berlin: Springer.Google Scholar
  39. Rendle, S., Freudenthaler, C., Gantner, Z., & Schmidt-Thieme, L. (2009). BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the twenty-fifth conference on uncertainty in artificial intelligence (pp. 452–461). AUAI Press.Google Scholar
  40. Salakhutdinov, R., Mnih, A., & Hinton, G. (2007). Restricted Boltzmann machines for collaborative filtering. In Proceedings of the 24th international conference on machine learning (pp. 791–798). ACM.Google Scholar
  41. Sarwar, B., Karypis, G., Konstan, J., & Riedl J. (2001). Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on world wide web (pp. 285–295). ACM.Google Scholar
  42. Schafer, J. B., Konstan, J., & Riedl, J. (1999). Recommender systems in e-commerce. In Proceedings of the 1st ACM conference on electronic commerce (pp. 158–166). ACM.Google Scholar
  43. Schedl, M. (2016). The LFM-1b dataset for music retrieval and recommendation. In Proceedings of the ACM international conference on multimedia retrieval (ICMR 2016), New York, USA.Google Scholar
  44. Shani, G., Heckerman, D., & Brafman, R. I. (2005). An MDP-based recommender system. Journal of Machine Learning Research, 6, 1265–1295.MathSciNetzbMATHGoogle Scholar
  45. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013). Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing (pp. 1631–1642).Google Scholar
  46. Song, Y., Elkahky, A. M., & He, X. (2016). Multi-rate deep learning for temporal recommendation. In Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval (pp. 909–912). ACM.Google Scholar
  47. Sukhbaatar, S., Weston, J., & Fergus, R. (2015). End-to-end memory networks. In Advances in neural information processing systems (pp. 2440–2448).Google Scholar
  48. Su, X., & Khoshgoftaar, T. M. (2009). A survey of collaborative filtering techniques. Advances in Artificial Intelligence, 2009, 2–5.CrossRefGoogle Scholar
  49. Tai, K. S., Socher, R., & Manning, C. D. (2015). Improved semantic representations from tree-structured long short-term memory networks. ArXiv preprint arXiv:1503.00075.
  50. Tan, Y. K., Xu, X., & Liu, Y. (2016). Improved recurrent neural networks for session-based recommendations. In Proceedings of the 1st workshop on deep learning for recommender systems (pp. 17–22). ACM.Google Scholar
  51. Van den Oord, A., Dieleman, S., & Schrauwen, B. (2013). Deep content-based music recommendation. In Advances in neural information processing systems (pp. 2643–2651).Google Scholar
  52. Vinyals, O., Kaiser, A., Koo, T., Petrov, S., Sutskever, I., & Hinton, G. (2015). Grammar as a foreign language. In Advances in neural information processing systems (pp. 2773–2781).Google Scholar
  53. Voorhees, E. M. (1999). The TREC-8 question answering track report. In TREC (Vol. 99, pp. 77–82).Google Scholar
  54. Wang, C., & Blei, D. M. (2011). Collaborative topic modelling for recommending scientific articles. In Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 448–456). ACM.Google Scholar
  55. Wang, H., Wang, N., & Yeung, D.-Y. (2015). Collaborative deep learning for recommender systems. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining (pp. 1235–1244). ACM.Google Scholar
  56. Wang, H., Zhang, F., Xie, X., & Guo, M. (2018). DKN: Deep knowledge-aware network for news recommendation. ArXiv preprint arXiv:1801.08284.
  57. Weimer, M., Karatzoglou, A., Le, Q. V., & Smola, A. J. (2008). Cofi rank-maximum margin matrix factorization for collaborative ranking. In Advances in neural information processing systems (pp. 1593–1600).Google Scholar
  58. Wu, C., & Yan, M. (2017). Session-aware information embedding for E-commerce product recommendation. In Proceedings of the 2017 ACM on conference on information and knowledge management (pp. 2379–2382). ACM.Google Scholar
  59. Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhudinov, R., Zemel, R., & Bengio, Y. (2015). Show, attend and tell: Neural image caption generation with visual attention. In International conference on machine learning (pp. 2048–2057).Google Scholar
  60. Yang, L., Ai, Q., Guo, J., & Croft, W. B. (2016). aNMM: Ranking short answer texts with attention-based neural matching model. In Proceedings of the 25th ACM international on conference on information and knowledge management (pp. 287–296). ACM.Google Scholar
  61. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., & Hovy, E. (2016). Hierarchical attention networks for document classification. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: Human language technologies (pp. 1480–1489).Google Scholar
  62. Yap, G.-E., Li, X.-L., & Philip, S. Y. (2012). Effective next-items recommendation via personalized sequential pattern mining. In International conference on database systems for advanced applications (pp. 48–64). Berlin: Springer.Google Scholar
  63. Yu, F., Liu, Q., Wu, S., Wang, L., & Tan, T. (2016). A dynamic recurrent model for next basket recommendation. In Proceedings of the 39th international ACM SIGIR conference on research and development in information retrieval (pp. 729–732). ACM.Google Scholar
  64. Zhang, Q., Gong, Y., Wu, J., Huang, H., & Huang, X. (2016). Retweet prediction with attention-based deep neural network. In Proceedings of the 25th ACM international on conference on information and knowledge management (pp. 75–84). ACM.Google Scholar
  65. Zhang, X., Zhao, J., & LeCun, Y. (2015). Character-level convolutional networks for text classification. In Advances in neural information processing systems (pp. 649–657).Google Scholar
  66. Zheleva, E., Guiver, J., Mendes Rodrigues, E., & Milić-Frayling, N. (2010). Statistical models of music-listening sessions in social media. In Proceedings of the 19th international conference on world wide web (pp. 1019–1028). ACM.Google Scholar
  67. Zimdars, A., Chickeringm, D. M., & Meek, C. (2001). Using temporal data for making recommendations. In Proceedings of the seventeenth conference on uncertainty in artificial intelligence (pp. 580–588). Burlington: Morgan Kaufmann Publishers Inc.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  1. 1.Department of Computer ScienceNorwegian University of Science and TechnologyTrondheimNorway

Personalised recommendations