Mining patient opinion to evaluate the service quality in healthcare: a deep-learning approach

Abstract

The emergence of social media has created several opportunities for patients to evaluate the quality of healthcare services by posting online reviews. The rich text and photographic information in the online reviews results insights into how patients’ experience with the doctor and their satisfaction with healthcare service delivery. Various studies have performed patients’ opinion analysis using textual contents. This study presents a novel multimodal approach to analyze the patients’ sentiment regarding the quality of healthcare service delivery (high vs. low). In comparison with existing studies, we consider not only the unique textual contents but photographic contents as well from the Yelp.com platform, which is more challenging due to feature extraction. We evaluate the performance of the baseline, and deep learning algorithms across textual cues, visual cues, and the fusion of both text and visual cues. Experimental results indicated that the addition of novel features increases the models’ accuracy to 15.0% across text and visual cues. Furthermore, fusing text and visual contents improve the classification accuracy by at least 12.64% instead of considering the contents alone. Preliminary comparative experiments result show that the deep learning model outperformed all other algorithms. In comparison with several other state-of-the-art methods in the biomedical domain, the proposed model could significantly enhance the performance of the classifier indicating the effectiveness and suitability of the methodology.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3

References

  1. Akter S, D’Ambra J, Ray P (2013) Development and validation of an instrument to measure user perceived service quality of mHealth. Inf Manag 50:181–195. https://doi.org/10.1016/j.im.2013.03.001

    Article  Google Scholar 

  2. Alemi F, Torii M, Clementz L, Aron DC (2012) Feasibility of real-time satisfaction surveys through automated analysis of patients’ unstructured comments and sentiments. Qual Manag Healthc 21:9–19. https://doi.org/10.1097/QMH.0b013e3182417fc4

    Article  Google Scholar 

  3. Alexa T (2018) Find website traffic, statistics, and analytics. https://www.alexa.com. Accessed 3 May 2018

  4. Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5:157–166. https://doi.org/10.1109/72.279181

    Article  Google Scholar 

  5. Bridle JS (1990) Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In: Soulié FF, Hérault J (eds) Proceedings of the neurocomputing. Springer, Berlin, pp 227–236

    Google Scholar 

  6. Burkle CM, Keegan MT (2015) Popularity of internet physician rating sites and their apparent influence on patients’ choices of physicians. BMC Health Serv Res 15:416. https://doi.org/10.1186/s12913-015-1099-2

    Article  Google Scholar 

  7. Cambria E, Hussain A, Durrani T, Havasi C, Eckl C, Munro J (2010) Sentic computing for patient centered applications. In: Proceedings of IEEE 10th international conference on signal processing. IEEE, Beijing, pp 1279–1282. http://dx.doi.org/10.1109/ICOSP.2010.5657072

  8. Campos V, Jou B, Giró-i-Nieto X (2017) From pixels to sentiment: fine-tuning CNNs for visual sentiment prediction. Image Vis Comput 65:15–22. https://doi.org/10.1016/j.imavis.2017.01.011

    Article  Google Scholar 

  9. Cao X, Liu Y, Zhu Z, Hu J, Chen X (2017) Online selection of a physician by patients: empirical study from elaboration likelihood perspective. Comput Hum Behav 73:403–412. https://doi.org/10.1016/j.chb.2017.03.060

    Article  Google Scholar 

  10. CDC (2016) Deaths and mortality. https://www.cdc.gov/nchs/data/hus/2017/019.pdf. Accessed 2 May 2018

  11. Chen X, Wang Y, Liu Q (2017) Visual and textual sentiment analysis using deep fusion convolutional neural networks. In: Proceedings of the 2017 IEEE international conference on image processing (ICIP). IEEE, Beijing, pp 1557–1561. https://doi.org/10.1109/ICIP.2017.8296543

  12. Csáji BC (2001) Approximation with artificial neural networks, vol 24. Faculty of Sciences, Etvs Lornd University, Budapest, p 48

    Google Scholar 

  13. Czech D, Mishra A, Inggs M (2018) A CNN and LSTM-based approach to classifying transient radio frequency interference. Astron Comput 25:52–57. https://doi.org/10.1016/j.ascom.2018.07.002

    Article  Google Scholar 

  14. Danielson B et al (2011) Development of indicators of the quality of radiotherapy for localized prostate cancer. Radiother Oncol 99:29–36. https://doi.org/10.1016/j.radonc.2011.02.013

    Article  Google Scholar 

  15. Emmert M, Meier F (2013) An analysis of online evaluations on a physician rating website: evidence from a German public reporting instrument. J Med Internet Res 15:e157. https://doi.org/10.2196/jmir.2655

    Article  Google Scholar 

  16. Emmert M, Sander U, Pisch F (2013) Eight questions about physician-rating websites: a systematic review. J Med Internet Res 15:e24. https://doi.org/10.2196/jmir.2360

    Article  Google Scholar 

  17. Emmert M, Meszmer N, Sander U (2016) Do health care providers use online patient ratings to improve the quality of care? Results from an online-based cross-sectional study. J Med Internet Res 18:e254. https://doi.org/10.2196/jmir.5889

    Article  Google Scholar 

  18. Gao GG, McCullough JS, Agarwal R, Jha AK (2012) A changing landscape of physician quality reporting: analysis of patients’ online ratings of their physicians over a 5-year period. J Med Internet Res 14:e38. https://doi.org/10.2196/jmir.2003

    Article  Google Scholar 

  19. Garrod B (2008) Exploring place perception a photo-based analysis. Ann Tour Res 35:381–401. https://doi.org/10.1016/j.annals.2007.09.004

    Article  Google Scholar 

  20. Goodfellow I, Bengio Y, Courville A (2016) Deep learning. The MIT Press, Cambridge

    Google Scholar 

  21. Grabner-Kräuter S, Waiguny MK (2015) Insights into the impact of online physician reviews on patients’ decision making: randomized experiment. J Med Internet Res 17:e93. https://doi.org/10.2196/jmir.3991

    Article  Google Scholar 

  22. Greaves F, Pape UJ, Lee H, Smith DM, Darzi A, Majeed A, Millett C (2012) Patients’ ratings of family physician practices on the internet: usage and associations with conventional measures of quality in the English National Health Service. J Med Internet Res 14:e146. https://doi.org/10.2196/jmir.2280

    Article  Google Scholar 

  23. Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L (2013) Use of sentiment analysis for capturing patient experience from free-text comments posted online. J Med Internet Res 15:e239. https://doi.org/10.2196/jmir.2721

    Article  Google Scholar 

  24. Gu J et al (2018) Recent advances in convolutional neural networks. Pattern Recognit 77:354–377. https://doi.org/10.1016/j.patcog.2017.10.013

    Article  Google Scholar 

  25. Hao H (2015) The development of online doctor reviews in China: an analysis of the largest online doctor review website in China. J Med Internet Res 17:e134. https://doi.org/10.2196/jmir.4365

    Article  Google Scholar 

  26. Hao H, Zhang K (2016) The voice of chinese health consumers: a text mining approach to web-based physician reviews. J Med Internet Res 18:e108. https://doi.org/10.2196/jmir.4430

    Article  Google Scholar 

  27. Hao H, Zhang K, Wang W, Gao G (2017) A tale of two countries: international comparison of online doctor reviews between China and the United States. Int J Med Inform 99:37–44. https://doi.org/10.1016/j.ijmedinf.2016.12.007

    Article  Google Scholar 

  28. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, Las Vegas, pp 770–778. https://doi.org/10.1109/CVPR.2016.90

  29. Hochreiter S, Bengio Y, Frasconi P, Schmidhuber J (2001) Gradient flow in recurrent nets: the difficulty of learning long term dependencies. In: John FK, Stefan CK (eds) A field guide to dynamical recurrent networks. 1st edn. Wiley-IEEE, pp 237–243

  30. Hu Z, Zhang Z, Yang H, Chen Q, Zuo D (2017) A deep learning approach for predicting the quality of online health expert question-answering services. J Biomed Inform 71:241–253. https://doi.org/10.1016/j.jbi.2017.06.012

    Article  Google Scholar 

  31. Hu Z, Zhang Z, Yang H, Chen Q, Zhu R, Zuo D (2018) Predicting the quality of online health expert question-answering services with temporal features in a deep learning framework. Neurocomputing 275:2769–2782. https://doi.org/10.1016/j.neucom.2017.11.039

    Article  Google Scholar 

  32. James TL, Villacis Calderon ED, Cook DF (2017) Exploring patient perceptions of healthcare service quality through analysis of unstructured feedback. Expert Syst Appl 71:479–492. https://doi.org/10.1016/j.eswa.2016.11.004

    Article  Google Scholar 

  33. Jiménez-Zafra SM, Martín-Valdivia MT, Molina-González MD, Ureña-López LA (2019) How do we talk about doctors and drugs? Sentiment analysis in forums expressing opinions for medical domain. Artif Intell Med 93:50–57. https://doi.org/10.1016/j.artmed.2018.03.007

    Article  Google Scholar 

  34. Jung Y, Hur C, Jung D, Kim M (2015) Identifying key hospital service quality factors in online health communities. J Med Internet Res 17:e90. https://doi.org/10.2196/jmir.3646

    Article  Google Scholar 

  35. Kadry B, Chu LF, Kadry B, Gammas D, Macario A (2011) Analysis of 4999 online physician ratings indicates that most patients give physicians a favorable rating. J Med Internet Res 13:e95. https://doi.org/10.2196/jmir.1960

    Article  Google Scholar 

  36. Kasiri LA, Cheng KTG, Sambasivan M, Sidin SM (2017) Integration of standardization and customization: impact on service quality, customer satisfaction, and loyalty. J Retail Consum Serv 35:91–97. https://doi.org/10.1016/j.jretconser.2016.11.007

    Article  Google Scholar 

  37. Kaymak S, Helwan A, Uzun D (2017) Breast cancer image classification using artificial neural networks. In: Proceedings of the 9th international conference on theory and application of soft computing, computing with words and perception. Elsevier, Budapest, pp 126–131

  38. Khan I, Dongping H, Wahab A (2016) Does culture matter in effectiveness of social media marketing strategy? An investigation of brand fan pages. Aslib J Inf Manag 68:694–715. https://doi.org/10.1108/AJIM-03-2016-0035

    Article  Google Scholar 

  39. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint. arXiv:14126980

  40. Kuang H, Che C, Zhang Q, Wei X (2017) LSTM based classification model and its application for doctor-patient relationship evaluation. In: Proceedings of the 2017 IEEE 19th international conference on e-health networking, applications and services (Healthcom). IEEE, Dalian, pp 1–5. http://doi.org/10.1109/HealthCom.2017.8210781

  41. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436. https://doi.org/10.1038/nature14539

    Article  Google Scholar 

  42. Li Y, Cao H (2017) Prediction for tourism flow based on LSTM neural network. In: Proceedings of the 2017 international conference on identification, information and knowledge in the Internet of Things. Elsevier, pp 277–283. https://doi.org/10.1016/j.procs.2018.03.076

  43. Li W, Chen H (2014) Identifying top sellers in underground economy using deep learning-based sentiment analysis. In: Proceedings of the 2014 IEEE joint intelligence and security informatics conference, The Hague, Netherlands, pp 64–67. http://dx.doi.org/10.1109/JISIC.2014.19

  44. Li J, Liu M, Li X, Liu X, Liu J (2018) Developing embedded taxonomy and mining patients’ interests from web-based physician reviews: mixed-methods approach. J Med Internet Res 20:e254. https://doi.org/10.2196/jmir.8868

    Article  Google Scholar 

  45. Li J, Tang J, Jiang L, Yen DC, Liu X (2019) Economic success of physicians in the online consultation market: a signaling theory perspective. Int J Electron Commer 23:244–271. https://doi.org/10.1080/10864415.2018.1564552

    Article  Google Scholar 

  46. Liu Z, Yang M, Wang X, Chen Q, Tang B, Wang Z, Xu H (2017) Entity recognition from clinical texts via recurrent neural network. BMC Med Inform Decis Mak 17:67. https://doi.org/10.1186/s12911-017-0468-7

    Article  Google Scholar 

  47. Lu N, Wu H (2016) Exploring the impact of word-of-mouth about Physicians’ service quality on patient choice based on online health communities. BMC Med Inform Decis Mak 16:151. https://doi.org/10.1186/s12911-016-0386-0

    Article  Google Scholar 

  48. Luarn P, Lin Y-F, Chiu Y-P (2015) Influence of Facebook brand-page posts on online engagement. Online Inf Rev 39:505–519. https://doi.org/10.1108/OIR-01-2015-0029

    Article  Google Scholar 

  49. Ma Y, Xiang Z, Du Q, Fan W (2018) Effects of user-provided photos on hotel review helpfulness: an analytical approach with deep leaning. Int J Hosp Manag 71:120–131. https://doi.org/10.1016/j.ijhm.2017.12.008

    Article  Google Scholar 

  50. Majumder N, Hazarika D, Gelbukh A, Cambria E, Poria S (2018) Multimodal sentiment analysis using hierarchical fusion with context modeling. Knowl Based Syst 161:124–133. https://doi.org/10.1016/j.knosys.2018.07.041

    Article  Google Scholar 

  51. Martinez D, Ananda-Rajah MR, Suominen H, Slavin MA, Thursky KA, Cavedon L (2015) Automatic detection of patients with invasive fungal disease from free-text computed tomography (CT) scans. J Biomed Inform 53:251–260. https://doi.org/10.1016/j.jbi.2014.11.009

    Article  Google Scholar 

  52. McAllister P, Zheng H, Bond R, Moorhead A (2018) Combining deep residual neural network features with supervised machine learning algorithms to classify diverse food image datasets. Comput Biol Med 95:217–233. https://doi.org/10.1016/j.compbiomed.2018.02.008

    Article  Google Scholar 

  53. Meesala A, Paul J (2018) Service quality, consumer satisfaction and loyalty in hospitals: thinking for the future. J Retail Consum Serv 40:261–269. https://doi.org/10.1016/j.jretconser.2016.10.011

    Article  Google Scholar 

  54. Mitropoulos P, Vasileiou K, Mitropoulos I (2018) Understanding quality and satisfaction in public hospital services: a nationwide inpatient survey in Greece. J Retail Consum Serv 40:270–275. https://doi.org/10.1016/j.jretconser.2017.03.004

    Article  Google Scholar 

  55. Okike K, Peter-Bibb TK, Xie KC, Okike ON (2016) Association between physician online rating and quality of care. J Med Internet Res 18:e324. https://doi.org/10.2196/jmir.6612

    Article  Google Scholar 

  56. Orlando JI, Prokofyeva E, del Fresno M, Blaschko MB (2018) An ensemble deep learning based approach for red lesion detection in fundus images. Comput Methods Progr Biomed 153:115–127. https://doi.org/10.1016/j.cmpb.2017.10.017

    Article  Google Scholar 

  57. Paolanti M, Kaiser C, Schallner R, Frontoni E, Zingaretti P (2017) Visual and textual sentiment analysis of brand-related social media pictures using deep convolutional neural networks. In: Proceedings of the 2017 international conference on image analysis and processing, springer, image analysis and processing—ICIAP 2017, vol 10484, pp 402–413. https://doi.org/10.1007/978-3-030-03649-2_19

  58. Papacharissi Z (2019) The virtual sphere. The Internet as a public sphere. In: Stempfhuber M, Wagner E (eds) Praktiken der Überwachten. Springer, Berlin, pp 43–60

    Google Scholar 

  59. Pascanu R, Mikolov T, Bengio Y (2012) Understanding the exploding gradient problem. CoRR. arXiv:abs/121150632

  60. Poria S, Cambria E, Howard N, Huang G-B, Hussain A (2016) Fusing audio, visual and textual clues for sentiment analysis from multimodal content. Neurocomputing 174:50–59. https://doi.org/10.1016/j.neucom.2015.01.095

    Article  Google Scholar 

  61. Poria S, Peng H, Hussain A, Howard N, Cambria E (2017) Ensemble application of convolutional neural networks and multiple kernel learning for multimodal sentiment analysis. Neurocomputing 261:217–230. https://doi.org/10.1016/j.neucom.2016.09.117

    Article  Google Scholar 

  62. Portal (2018) Leading 10 US states based on the total number of active physicians as of 2018. https://www.cdc.gov/nchs/data/hus/2017/019.pdf. Accessed 5 May 2018

  63. Qian X, Li M, Ren Y, Jiang S (2019) Social media based event summarization by user–text–image co-clustering. Knowl Based Syst 164:107–121. https://doi.org/10.1016/j.knosys.2018.10.028

    Article  Google Scholar 

  64. Rajkomar A et al (2018) Scalable and accurate deep learning with electronic health records NPJ. Digit Med 1:18. https://doi.org/10.1038/s41746-018-0029-1

    Article  Google Scholar 

  65. Reddy BK, Delen D (2018) Predicting hospital readmission for lupus patients: an RNN-LSTM-based deep-learning methodology. Comput Biol Med 101:199–209. https://doi.org/10.1016/j.compbiomed.2018.08.029

    Article  Google Scholar 

  66. Rothenfluh F, Germeni E, Schulz PJ (2016) Consumer decision-making based on review websites: are there differences between choosing a hotel and choosing a physician? J Med Internet Res 18:e129. https://doi.org/10.2196/jmir.5580

    Article  Google Scholar 

  67. Saha M, Chakraborty C, Racoceanu D (2018) Efficient deep learning model for mitosis detection using breast histopathology images. Comput Med Imaging Graph 64:29–40. https://doi.org/10.1016/j.compmedimag.2017.12.001

    Article  Google Scholar 

  68. Sahu SK, Anand A (2018) Drug-drug interaction extraction from biomedical texts using long short-term memory network. J Biomed Inform 86:15–24. https://doi.org/10.1016/j.jbi.2018.08.005

    Article  Google Scholar 

  69. Sajid M, Baig M (2007) Quality of health care: an absolute necessity for public satisfaction. Int J Health Care Qual Assur 20:545–548. https://doi.org/10.1108/09526860710819477

    Article  Google Scholar 

  70. Sirinukunwattana K, Raza SEA, Tsang Y, Snead DRJ, Cree IA, Rajpoot NM (2016) Locality sensitive deep learning for detection and classification of nuclei in routine colon cancer histology images. IEEE Trans Med Imaging 35:1196–1206. https://doi.org/10.1109/TMI.2016.2525803

    Article  Google Scholar 

  71. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958

    MathSciNet  MATH  Google Scholar 

  72. Terlutter R, Bidmon S, Röttl J (2014) Who uses physician-rating websites? Differences in sociodemographic variables, psychographic variables, and health status of users and nonusers of physician-rating websites. J Med Internet Res 16:e97. https://doi.org/10.2196/jmir.3145

    Article  Google Scholar 

  73. Tutubalina E, Nikolenko S (2018) Exploring convolutional neural networks and topic models for user profiling from drug reviews. Multimed Tools Appl 77:4791–4809. https://doi.org/10.1007/s11042-017-5336-z

    Article  Google Scholar 

  74. Vedaldi A, Lenc K (2015) MatConvNet: convolutional neural networks for MATLAB. In: Paper presented at the Proceedings of the 23rd ACM international conference on Multimedia, Brisbane, Australia

  75. Wallace BC, Paul MJ, Sarkar U, Trikalinos TA, Dredze M (2014) A large-scale quantitative analysis of latent factors and sentiment in online doctor reviews. J Am Med Inform Assoc 21:1098–1103. https://doi.org/10.1136/amiajnl-2014-002711

    Article  Google Scholar 

  76. Wang J, Peng B, Zhang X (2018) Using a stacked residual LSTM model for sentiment intensity prediction. Neurocomputing 322:93–101. https://doi.org/10.1016/j.neucom.2018.09.049

    Article  Google Scholar 

  77. Xia C, Zhao D, Wang J, Liu J, Ma J (2018) ICSH 2018: LSTM based sentiment analysis for patient experience narratives in E-survey tools. In: Proceedings of the 6th international conference on smart health. Springer, Wuhan, pp 231–239. https://doi.org/10.1007/978-3-030-03649-2_23

  78. Xiao Y, Wu J, Lin Z, Zhao X (2018) A deep learning-based multi-model ensemble method for cancer prediction. Comput Methods Progr Biomed 153:1–9. https://doi.org/10.1016/j.cmpb.2017.09.005

    Article  Google Scholar 

  79. Xie J, Zhang B, Zeng D (2018) Readmission prediction using trajectory-based deep learning approach. In: Proceedings of the 6th international conference on smart health. Springer, Wuhan, pp 224–230. http://dx.doi.org/10.1007/978-3-030-03649-2_22

  80. Yang H, Guo X, Wu T (2015a) Exploring the influence of the online physician service delivery process on patient satisfaction. Decis Support Syst 78:113–121. https://doi.org/10.1016/j.dss.2015.05.006

    Article  Google Scholar 

  81. Yang H, Guo X, Wu T, Ju X (2015b) Exploring the effects of patient-generated and system-generated information on patients’ online search, evaluation and decision. Electron Commer Res Appl 14:192–203. https://doi.org/10.1016/j.elerap.2015.04.001

    Article  Google Scholar 

  82. Yu Y, Lin H, Meng J, Zhao Z (2016) Visual and textual sentiment analysis of a microblog using deep convolutional neural networks. Algorithms 9:41. https://doi.org/10.3390/a9020041

    MathSciNet  Article  MATH  Google Scholar 

  83. Zhou F, Hang R, Liu Q, Yuan X (2018) Hyperspectral image classification using spectral-spatial LSTMs. Neurocomputing. https://doi.org/10.1016/j.neucom.2018.02.105

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by the National Natural Science Foundation, People’s Republic of China (nos. 71531013, 71401047, 71729001).

Author information

Affiliations

Authors

Corresponding author

Correspondence to Adnan Muhammad Shah.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Shah, A.M., Yan, X., Shah, S.A.A. et al. Mining patient opinion to evaluate the service quality in healthcare: a deep-learning approach. J Ambient Intell Human Comput 11, 2925–2942 (2020). https://doi.org/10.1007/s12652-019-01434-8

Download citation

Keywords

  • Online doctor reviews
  • Service quality
  • Sentiment analysis
  • Multimodal fusion
  • Deep learning