Sequential Recommendation Based on Long-Term and Short-Term User Behavior with Self-attention

  • Xing Wei
  • Xianglin Zuo
  • Bo YangEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11775)


Product recommenders based on users’ interests are becoming increasingly essential in e-commerce. With the continuous development of the recommendation system, the available information is further enriched. In the case, user’s click or purchase behavior could be a visual representation of his or her interest. Due to the rapid update of products, users’ interests are not static, but change over time. In order to cope with the users’ interest changes, we propose a desirable work on the basis of representative recommendation algorithm. The sequence of user interaction behavior is thoroughly utilized, and the items that users interact at different times have different significance for the reflect of users’ interests. By considering the user’s sequential behaviors, this paper focuses on the recent ones to obtains the real interest of user. In this process, user behavior is divided into long-term and short-term, modeled by LSTM and Attention-based model respectively for user’s next click recommendation. We refer this model as LANCR and analyze the model in experiment. The experiment demonstrates that the proposed model has superior improvement compared with standard approaches. We deploy our model on two real datasets to verify the superior performance made in predicting user preferences.


Recommender system Long short-term memory Sequential recommendation 



This work was supported in part by National Natural Science Foundation of China under grants 61876069 and 61572226, and Jilin Province Key Scientific and Technological Research and Development project under grants 20180201067GX and 20180201044GX.


  1. 1.
    Liu, D.R., Lai, C.H., Lee, W.J.: A hybrid of sequential rules and collaborative filtering for product recommendation. Inf. Sci. 179(20), 3505–3519 (2009)CrossRefGoogle Scholar
  2. 2.
    Rendle, S., Freudenthaler, C., Schmidt-Thieme, L.: Factorizing personalized markov chains for next-basket recommendation. In: Proceedings of the 19th International Conference on World Wide Web, pp. 811–820. ACM (2010)Google Scholar
  3. 3.
    Tang, J., Wang, K.: Personalized top-n sequential recommendation via convolutional sequence embedding. In: Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, pp. 565–573. ACM (2018)Google Scholar
  4. 4.
    Zhou, C., et al.: ATRank: an attention-based user behavior modeling framework for recommendation. In: Proceeding of the 32nd AAAI Conference on Artificial Intelligence (2018)Google Scholar
  5. 5.
    Breese, J.S., Heckerman, D., Kadie, C.: Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of the 14th Conference on Uncertainty in Artificial Intelligence, pp. 43–52 (1998)Google Scholar
  6. 6.
    Ni, Y., et al.: Perceive your users in depth: learning universal user representations from multiple e-commerce tasks. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 596–605. ACM (2018)Google Scholar
  7. 7.
    Villatel, K., Smirnova, E., Mary, J., et al.: Recurrent neural networks for long and short-term sequential recommendation. arXiv preprint arXiv:1807.09142 (2018)
  8. 8.
    Wu, S., Tang, Y., Zhu, Y., Wang, L., Xie, X., Tan, T.: Session-based recommendation with graph neural networks. arXiv preprint arXiv:1811.00855 (2018)
  9. 9.
    Zhang, S., Tay, Y., Yao, L., Sun, A.: Next item recommendation with self-attention. arXiv preprint arXiv:1808.06414 (2018)
  10. 10.
    Bai, T., Du, P., Zhao, W.X., Wen, J.R., Nie, J.Y.: A long-short demands-aware model for next-item recommendation. arXiv preprint arXiv:1903.00066 (2019)
  11. 11.
    Li, Z., Zhao, H., Liu, Q., Huang, Z., Mei, T., Chen, E.: Learning from history and present: next-item recommendation via discriminatively exploiting user behaviors. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1734–1743. ACM (2018)Google Scholar
  12. 12.
    Chung, J., Ahn, S., Bengio, Y.: Hierarchical multiscale recurrent neural networks. arXiv preprint arXiv:1609.01704 (2016)
  13. 13.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRefGoogle Scholar
  14. 14.
    Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)Google Scholar
  15. 15.
    Wang, S., Zhang, J., Zong, C.: Learning sentence representation with guidance of human attention. arXiv preprint arXiv:1609.09189 (2016)
  16. 16.
    Yang, Z., He, X., Gao, J., Deng, L., Smola, A.: Stacked attention networks for image question answering. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 21–29 (2016)Google Scholar
  17. 17.
    Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
  18. 18.
    Rendle, S.: Factorization machines. In: 2010 IEEE International Conference on Data Mining, pp. 995–1000. IEEE (2010)Google Scholar
  19. 19.
    Rendle, S., Freudenthaler, C., Gantner, Z., Schmidt-Thieme, L.: BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence, pp. 452–461. AUAI Press (2009)Google Scholar
  20. 20.
    Gilks, W.R., Richardson, S., Spiegelhalter, D.: Markov Chain Monte Carlo in Practice. Chapman and Hall/CRC, Boca Raton (1995)CrossRefGoogle Scholar
  21. 21.
    Tan, Y.K., Xu, X., Liu, Y.: Improved recurrent neural networks for session-based recommendations. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, pp. 17–22. ACM (2016)Google Scholar
  22. 22.
    Hidasi, B., Karatzoglou, A.: Recurrent neural networks with top-k gains for session-based recommendations. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp. 843–852. ACM (2018)Google Scholar
  23. 23.
    Pu, C., Wu, Z., Chen, H., Xu, K., Cao, J.: A sequential recommendation for mobile apps: what will user click next app?. In: 2018 IEEE International Conference on Web Services (ICWS), pp. 243–248. IEEE (2018)Google Scholar
  24. 24.
    Donkers, T., Loepp, B., Ziegler, J.: Sequential user-based recurrent neural network recommendations. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, pp. 152–160. ACM (2017)Google Scholar
  25. 25.
    Kang, W.C., McAuley, J.: Self-attentive sequential recommendation. In: 2018 IEEE International Conference on Data Mining (ICDM), pp. 197–206. IEEE (2018)Google Scholar
  26. 26.
    Wang, S., Hu, L., Cao, L., Huang, X., Lian, D., Liu, W.: Attention-based transactional context embedding for next-item recommendation. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.College of Computer Science and TechnologyJilin UniversityChangchunChina
  2. 2.Key Laboratory of Symbolic Computation and Knowledge Engineering Attached to the Ministry of EducationJilin UniversityChangchunChina

Personalised recommendations