Sentiment-Aware Multi-modal Recommendation on Tourist Attractions

  • Junyi Wang
  • Bing-Kun BaoEmail author
  • Changsheng Xu
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11295)


For tourist attraction recommendation, there are three essential aspects to be considered: tourist preferences, attraction themes, and sentiments on themes of attraction. By utilizing vast multi-modal media available on Internet, this paper is aiming to develop an efficient solution of tourist attraction recommendation covering all these three aspects. To achieve this goal, we propose a probabilistic generative model called Sentiment-aware Multi-modal Topic Model (SMTM), whose advantages are four folds: (1) we separate tourists and attractions into two domains for better recovering tourist topics and attraction themes; (2) we investigate tourists sentiments on topics to retain the preference ones; (3) the recommended attraction is guaranteed with positive sentiment on the related attraction themes; (4) the multi-modal data are utilized to enhance the recommendation accuracy. Qualitative and quantitative evaluation results have validated the effectiveness of our method.


Tourism recommendation Multi-modal computing Topic model Sentiment analysis 



This work is supported by the National Key Research & Development Plan of China (No. 2017YFB1002800), by the National Natural Science Foundation of China under Grant 61872424, 61572503, 61720106006, 61432019, and by NUPTSF (No. NY218001), also supported by the Key Research Program of Frontier Sciences, CAS, Grant NO. QYZDJ-SSW-JSC039, and the K.C.Wong Education Foundation.


  1. 1.
    Adomavicius, G., et al.: Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions. IEEE Trans. Knowl. Data Eng. 17(6), 734–749 (2005)CrossRefGoogle Scholar
  2. 2.
    Blei, D., Carin, L., Dunson, D.: Probabilistic topic models. IEEE Signal Process. Mag. 27(6), 55–65 (2010)Google Scholar
  3. 3.
    Blei, D.M., Jordan, M.I.: Modeling annotated data. In: Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 127–134 (2013)Google Scholar
  4. 4.
    Huang, C., Wang, Q., Yang, D., et al.: Topic mining of tourist attractions based on a seasonal context aware LDA model. Intell. Data Anal. 22(2), 383–405 (2018)CrossRefGoogle Scholar
  5. 5.
    Bao, B.K., Xu, C., Min, W., Hossain, M.S.: Cross-platform emerging topic detection and elaboration from multimedia streams. TOMCCAP 11(4), 54 (2015)CrossRefGoogle Scholar
  6. 6.
    Bao, B.-K., Liu, G., Changsheng, X., Yan, S.: Inductive robust principal component analysis. IEEE Trans. Image Process. 21(8), 3794–3800 (2012)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Bao, B.-K., Zhu, G., Shen, J., Yan, S.: Robust image analysis with sparse representation on quantized visual features. IEEE Trans. Image Process. 22(3), 860–871 (2013)MathSciNetCrossRefGoogle Scholar
  8. 8.
    Borras, J., Moreno, A., Valls, A.: Intelligent tourism recommender systems: a survey. Expert Syst. Appl. 41(16), 7370–7389 (2014)CrossRefGoogle Scholar
  9. 9.
    Leal, F., González–Vélez, H., Malheiro, B., Burguillo, J.C.: Semantic profiling and destination recommendation based on crowd-sourced tourist reviews. Distributed Computing and Artificial Intelligence, 14th International Conference. AISC, vol. 620, pp. 140–147. Springer, Cham (2018). Scholar
  10. 10.
    Yang, D., Zhang, D., Yu, Z., et al.: A sentiment-enhanced personalized location recommendation system. In: ACM Conference on Hypertext and Social Media, pp. 119-128. ACM (2013)Google Scholar
  11. 11.
    Shen, J., Deng, C., Gao, X.: Attraction recommendation: towards personalized tourism via collective intelligence. Neurocomputing 173, 789–798 (2016)CrossRefGoogle Scholar
  12. 12.
    Kurashima, T., Iwata, T., Irie, G., Fujimura, K.: Travel route recommendation using geotags in photo sharing sites. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management, Toronto, Canada, pp. 579–588. ACM, October 2010Google Scholar
  13. 13.
    Wu, Y., Ester, M.: FLAME: a probabilistic model combining aspect based opinion mining and collaborative filtering. In: Eighth ACM International Conference on Web Search and Data Mining, pp. 199–208. ACM (2015)Google Scholar
  14. 14.
    Arbelaitz, O., Gurrutxaga, I., Lojo, A., Muguerza, J., Perez, J.M., Perona, I.: Web usage and content mining to extract knowledge for modelling the users of the Bidasoa Turismo website and to adapt it. Expert Syst. Appl. 40(18), 7478–7491 (2013)CrossRefGoogle Scholar
  15. 15.
    Hao, Q., et al.: Equip tourists with knowledge mined from travelogues. In: Proceedings of the 19th International Conference on World Wide Web, pp. 401–410. ACM (2010)Google Scholar
  16. 16.
    Jiang, K., Wang, P., Yu, N.: ContextRank: personalized tourism recommendation by exploiting context information of geotagged web photos. In: 2011 Sixth International Conference on Image and Graphics, Hefei, Anhui, China, pp. 931–937. IEEE, August 2011 (2011)Google Scholar
  17. 17.
    Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J Mach. Learn. Res. 3, 993–1022 (2003)zbMATHGoogle Scholar
  18. 18.
    Mei, Q., Ling, X., Wondra, M., et al.: Topic sentiment mixture: modeling facets and opinions in weblogs. In: Proceedings of the 16th International Conference on World Wide Web, pp. 171–180 (2007)Google Scholar
  19. 19.
    Fang, Q., Xu, C., Sang, J., et al.: Word-of-mouth understanding: entity-centric multimodal aspect-opinion mining in social media. IEEE Trans. Multimedia 17(12), 2281–2296 (2015)CrossRefGoogle Scholar
  20. 20.
    Xiong, H., Xiong, H., Xiong, H., et al.: A location-sentiment-aware recommender system for both home-town and out-of-town users, pp. 1135–1143 (2017)Google Scholar
  21. 21.
    Titov, I., McDonald, R.: A joint model of text and aspect ratings for sentiment summarization. In: ACL-08: HLT, pp. 308–316. Association for Computational Linguistics (2008)Google Scholar
  22. 22.
    Olszewski, D.: Fraud detection in telecommunications using Kullback-Leibler divergence and latent Dirichlet allocation. In: Dobnikar, A., Lotrič, U., Šter, B. (eds.) ICANNGA 2011. LNCS, vol. 6594, pp. 71–80. Springer, Heidelberg (2011). Scholar
  23. 23.
    Fang, Y., Si, L., Somasundaram, N. Yu, Z.: Mining contrastive opinions on political texts using cross-perspective topic model. In: Proceedings of the fifth ACM international conference on Web search and data mining, pp. 63–72. ACM (2012)Google Scholar
  24. 24.
    Lin, C., He, Y., Everson, R., et al.: Weakly supervised joint sentiment-topic detection from text. IEEE T. Knowl. Data En. 24(6), 1134–1145 (2012)CrossRefGoogle Scholar
  25. 25.
    Qian, S., Zhang, T., Xu, C., et al.: Multi-modal event topic model for social event analysis. IEEE Trans. Multimedia 18(2), 233–246 (2016)CrossRefGoogle Scholar
  26. 26.
    Huang, F., Zhang, S., Zhang, J., et al.: Multimodal learning for topic sentiment analysis in microblogging. Neurocomputing 253(C), 144–153 (2017)CrossRefGoogle Scholar
  27. 27.
    Alam, M.H., Ryu, W.J., Lee, S.K.: Joint multi-grain topic sentiment: modeling semantic aspects for online reviews. Inf. Sci. 339, 206–223 (2016)CrossRefGoogle Scholar
  28. 28.
    Min, W., Bao, B.K., Mei, S., et al.: You are what you eat: exploring rich recipe information for cross-region food analysis. IEEE Trans. Multimed. 1 (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Hefei University of TechnologyHefeiChina
  2. 2.College of Telecommunications and Information EngineeringNanjing University of Posts and TelecommunicationsNanjingChina
  3. 3.National Lab of Pattern RecognitionInstitute of Automation, CASBeijingChina
  4. 4.University of Chinese Academy of SciencesBeijingChina

Personalised recommendations