ICSH 2018: LSTM based Sentiment Analysis for Patient Experience Narratives in E-survey Tools

  • Chenxi Xia
  • Dong Zhao
  • Jing Wang
  • Jing Liu
  • Jingdong MaEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10983)


Background: Analysis of patient experience narratives is helpful to improve care service and to promote patient satisfaction. As more and more E-survey tools went on line, a huge amount of patient comments has become a challenge to analysts. Sentiment analysis is a fully explored machine learning method to classify texts according their sentimental orientation. However, it is seldom applied to the analysis of patient comments, especially in China. Objectives: This paper aims to test the performance of the classical sentiment analysis methods and find an applicable solution of Chinese patient experience narratives analysis. Data: 20,000 patient experience narratives are collected from two hospital’s E-survey tools, a mobile patient follow-up system and a WeChat App of patient comments. Methods: Five machine learning methods, Support Vector Machine (SVM), Random Forests (RF), Gradient Boost Decision Tree (GBDT), XGBoost and Long Short-Term Memory (LSTM), are used to explore the sentiment analysis performance of Chinese patient comments. χ2 statistics is used for feature selection. And Skip-gram model is used for word-embedding. Results: The experiment results showed that LSTM achieved much better performance than SVM did. The F-Measure values of LSTM for positive category and negative category are both 98.87, which is better than other traditional machine learning methods. Conclusion: The result of this paper suggests that LSTM based sentiment analysis is a practical method to exploit the ever-increasing patient experience narratives.


Patient experience Sentiment analysis Free text Electronic survey 


  1. 1.
    Al-Abri, R., Al-Balushi, A.: Patient satisfaction survey as a tool towards quality improvement. Oman Med. J. 29, 3 (2014)CrossRefGoogle Scholar
  2. 2.
    Zgierska, A., Rabago, D., Miller, M.M.: Impact of patient satisfaction ratings on physicians and clinical care. Patient Prefer. Adherence 8, 437 (2014)CrossRefGoogle Scholar
  3. 3.
    Boos, J., et al.: Electronic kiosks for patient satisfaction survey in radiology. Am. J. Roentgenol. 208, 577–584 (2017)CrossRefGoogle Scholar
  4. 4.
    McPeake, J., Bateson, M., O’Neill, A.: Electronic surveys: how to maximise success. Nurse Res. (2014+) 21, 24 (2014)CrossRefGoogle Scholar
  5. 5.
    Repplinger, M.D., et al.: The impact of an emergency department front-end redesign on patient-reported satisfaction survey results. West. J. Emerg. Med. 18, 1068 (2017)CrossRefGoogle Scholar
  6. 6.
    Gillespie, A., Reader, T.W.: The healthcare complaints analysis tool: development and reliability testing of a method for service monitoring and organisational learning. BMJ Qual. Saf. 25, 937–946 (2016)CrossRefGoogle Scholar
  7. 7.
    Reader, T.W., Gillespie, A., Roberts, J.: Patient complaints in healthcare systems: a systematic review and coding taxonomy. BMJ Qual. Saf. 23, 678–689 (2014)CrossRefGoogle Scholar
  8. 8.
    Hamouda, A.E.-D.A., El-taher, F.E.-Z.: Sentiment analyzer for arabic comments system. Int. J. Adv. Comput. Sci. Appl. 4, 99–103 (2013)Google Scholar
  9. 9.
    Shamshurin, I.: Data representation in machine learning-based sentiment analysis of customer reviews. In: Kuznetsov, S.O., Mandal, D.P., Kundu, M.K., Pal, S.K. (eds.) PReMI 2011. LNCS, vol. 6744, pp. 254–260. Springer, Heidelberg (2011). Scholar
  10. 10.
    Zeb, S., Qamar, U., Hussain, F.: Sentiment analysis on user reviews through lexicon and rule-based approach. In: Morishima, A., et al. (eds.) APWeb 2016. LNCS, vol. 9865, pp. 55–63. Springer, Cham (2016). Scholar
  11. 11.
    Alemi, F., Torii, M., Clementz, L., Aron, D.C.: Feasibility of real-time satisfaction surveys through automated analysis of patients’ unstructured comments and sentiments. Qual. Manag. Healthc. 21, 9–19 (2012)CrossRefGoogle Scholar
  12. 12.
    Greaves, F., Ramirez-Cano, D., Millett, C., Darzi, A., Donaldson, L.: Use of sentiment analysis for capturing patient experience from free-text comments posted online. J. Med. Internet Res. 15, e239 (2013)CrossRefGoogle Scholar
  13. 13.
    Hopper, A.M., Uriyo, M.: Using sentiment analysis to review patient satisfaction data located on the internet. J. Health Organ. Manag. 29, 221–233 (2015)CrossRefGoogle Scholar
  14. 14.
    Elmessiry, A., Cooper, W.O., Catron, T.F., Karrass, J., Zhang, Z., Singh, M.P.: Triaging patient complaints: monte carlo cross-validation of six machine learning classifiers. JMIR Med. Inform. 5, e19 (2017)CrossRefGoogle Scholar
  15. 15.
    Tang, D., Qin, B., Liu, T.: Deep learning for sentiment analysis: successful approaches and future challenges. Wiley Interdiscip. Rev.: Data Min. Knowl. Discov. 5, 292–303 (2015)Google Scholar
  16. 16.
    Gräbner, D., Zanker, M., Fliedl, G., Fuchs, M.: Classification of customer reviews based on sentiment analysis. na (2012)CrossRefGoogle Scholar
  17. 17.
    Zhang, Z., Ye, Q., Zhang, Z., Li, Y.: Sentiment classification of Internet restaurant reviews written in Cantonese. Expert Syst. Appl. 38, 7674–7682 (2011)CrossRefGoogle Scholar
  18. 18.
    Zhang, D., Xu, H., Su, Z., Xu, Y.: Chinese comments sentiment classification based on word2vec and SVMperf. Expert Syst. Appl. 42, 1857–1863 (2015)CrossRefGoogle Scholar
  19. 19.
    Mouthami, K., Devi, K.N., Bhaskaran, V.M.: Sentiment analysis and classification based on textual reviews. In: 2013 international conference on Information communication and embedded systems (ICICES), pp. 271–276. IEEE (2013)Google Scholar
  20. 20.
    ElMessiry, A., Zhang, Z., Cooper, W.O., Catron, T.F., Karrass, J., Singh, M.P.: Leveraging sentiment analysis for classifying patient complaints. In: Proceedings of the 8th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics - ACM-BCB 2017, pp. 44–51 (2017)Google Scholar
  21. 21.
    Ayata, D., Saraçlar, M., Özgür, A.: Political opinion/sentiment prediction via long short term memory recurrent neural networks on Twitter. In: 2017 25th Signal Processing and Communications Applications Conference (SIU), pp. 1–4. IEEE (2017)Google Scholar
  22. 22.
    Romanowski, A., Skuza, M.: Towards predicting stock price moves with aid of sentiment analysis of Twitter social network data and big data processing environment. In: Pełech-Pilichowski, T., Mach-Król, M., Olszak, C.M. (eds.) Advances in Business ICT: New Ideas from Ongoing Research. SCI, vol. 658, pp. 105–123. Springer, Cham (2017). Scholar
  23. 23.
    Hu, M., Liu, B.: Mining and summarizing customer reviews. In: Proceedings of the tenth ACM SIGKDD International Conference on KNOWLEDGE DISCOVERY and Data Mining, pp. 168–177. ACM (2004)Google Scholar
  24. 24.
    Ding, X., Liu, B., Yu, P.S.: A holistic lexicon-based approach to opinion mining. In: Proceedings of the 2008 International Conference on Web Search and Data Mining, pp. 231–240. ACM (2008)Google Scholar
  25. 25.
    Stone, P.J., Dunphy, D.C., Smith, M.S.: The general inquirer: a computer approach to content analysis (1966)Google Scholar
  26. 26.
    Pennebaker, J.W., Francis, M.E., Booth, R.J.: Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Assoc. 71, 2001 (2001)Google Scholar
  27. 27.
    Baccianella, S., Esuli, A., Sebastiani, F.: Sentiwordnet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. In: LREC, pp. 2200–2204 (2010)Google Scholar
  28. 28.
    Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up?: sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 Conference on Empirical methods in Natural Language Processing-Volume 10, pp. 79–86. Association for Computational Linguistics (2002)Google Scholar
  29. 29.
    Dave, K., Lawrence, S., Pennock, D.M.: Mining the peanut gallery: opinion extraction and semantic classification of product reviews. In: Proceedings of the 12th International Conference on World Wide Web, pp. 519–528. ACM (2003)Google Scholar
  30. 30.
    Mullen, T., Collier, N.: Sentiment analysis using support vector machines with diverse information sources. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing (2004)Google Scholar
  31. 31.
    Mikolov, T.: Statistical language models based on neural networks. Presentation at Google, Mountain View, 2nd April 2012Google Scholar
  32. 32.
    Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.: A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003)zbMATHGoogle Scholar
  33. 33.
    Poria, S., Cambria, E., Gelbukh, A.: Aspect extraction for opinion mining with a deep convolutional neural network. Knowl.-Based Syst. 108, 42–49 (2016)CrossRefGoogle Scholar
  34. 34.
    Ganin, Y., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17, 1–35 (2016)MathSciNetzbMATHGoogle Scholar
  35. 35.
    Chau, N.P., Phan, V.A., Le Nguyen, M.: Deep learning and sub-tree mining for document level sentiment classification. In: 2016 Eighth International Conference on Knowledge and Systems Engineering (KSE), pp. 268–273. IEEE (2016)Google Scholar
  36. 36.
    del Arco, F.M.P., Valdivia, M.T.M., Zafra, S.M.J., González, M.D.M., Cámara, E.M.: COPOS: corpus of patient opinions in Spanish. Application of sentiment analysis techniques. Proces. Leng. Nat. 57, 83–90 (2016)Google Scholar
  37. 37.
    Zhang, H.-P., Yu, H.-K., Xiong, D.-Y., Liu, Q.: HHMM-based Chinese lexical analyzer ICTCLAS. In: Proceedings of the Second SIGHAN Workshop on Chinese Language Processing-Volume 17, pp. 184–187. Association for Computational Linguistics (2003)Google Scholar
  38. 38.
  39. 39.
    来斯惟: 基于神经网络的词和文档语义向量表示方法研究. 中国科学院大学 (2016)Google Scholar
  40. 40.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20, 273–297 (1995)zbMATHGoogle Scholar
  41. 41.
    Liaw, A., Wiener, M.: Classification and regression by randomForest. R News 2, 18–22 (2002)Google Scholar
  42. 42.
    Friedman, J.H.: Greedy function approximation: a gradient boosting machine. Ann. Stat. 29, 1189–1232 (2001)MathSciNetCrossRefGoogle Scholar
  43. 43.
    Chen, T., Guestrin, C.: Xgboost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785–794. ACM (2016)Google Scholar
  44. 44.
    Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9, 1735–1780 (1997)CrossRefGoogle Scholar
  45. 45.
    Galavotti, L., Sebastiani, F., Simi, M.: Experiments on the use of feature selection and negative evidence in automated text categorization. In: Borbinha, J., Baker, T. (eds.) ECDL 2000. LNCS, vol. 1923, pp. 59–68. Springer, Heidelberg (2000). Scholar
  46. 46.
    Caropreso, M.F., Matwin, S., Sebastiani, F.: A learner-independent evaluation of the usefulness of statistical phrases for automated text categorization. In: Text databases and Document Management: Theory and practice, vol. 5478, pp. 78–102 (2001)Google Scholar
  47. 47.
    Sebastiani, F., Sperduti, A., Valdambrini, N.: An improved boosting algorithm and its application to text categorization. In: Proceedings of the Ninth International Conference on Information and Knowledge Management, pp. 78–85. ACM (2000)Google Scholar
  48. 48.
    Salton, G., Buckley, C.: Term-weighting approaches in automatic text retrieval. Inf. Process. Manag. 24, 513–523 (1988)CrossRefGoogle Scholar
  49. 49.
    Kinga, D., Adam, J.B.: A method for stochastic optimization. In: International Conference on Learning Representations (ICLR) (2014)Google Scholar
  50. 50.
    Li, J., Xu, H., He, X., Deng, J., Sun, X.: Tweet modeling with LSTM recurrent neural networks for hashtag recommendation. In: 2016 International Joint Conference on Neural Networks (IJCNN), pp. 1570–1577. IEEE (2016)Google Scholar
  51. 51.
    Uysal, A.K., Murphey, Y.L.: Sentiment classification: feature selection based approaches versus deep learning. In: 2017 IEEE International Conference on Computer and Information Technology (CIT), pp. 23–30. IEEE (2017)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • Chenxi Xia
    • 1
  • Dong Zhao
    • 1
  • Jing Wang
    • 1
  • Jing Liu
    • 1
  • Jingdong Ma
    • 1
    Email author
  1. 1.School of Medicine and Health Management, Tongji Medical CollegeHuazhong University of Science and TechnologyWuhanChina

Personalised recommendations