Programming and Computer Software

, Volume 45, Issue 8, pp 439–447 | Cite as

Entity-Level Classification of Adverse Drug Reaction: A Comparative Analysis of Neural Network Models

  • I. S. AlimovaEmail author
  • E. V. TutubalinaEmail author


An experimental work on the analysis of effectiveness of neural network models applied to the classification of adverse drug reactions at the entity level is described. Aspect-level sentiment analysis, which aims to determine the sentimental class of a specific aspect conveyed in user opinions, has been actively studied for more than 10 years. A number of neural network architectures have been proposed. Even though the models based on these architectures have much in common, they differ in certain components. In this paper, the applicability of the neural network models developed for the aspect-level sentiment analysis to the problem of the classification of adverse drug reactions is studied. Extensive experiments on English language texts of biomedical topic, including health records, scientific literature, and social media have been conducted. The proposed models mentioned above are compared with one of the best model based on the support vector machine method and a large set of features.



This work was supported by the Russian Science Foundation, project no. 18-11-00284.


  1. 1.
    Murff, H.J., Patel, V.L., Hripcsak, G., and Bates, D.W., Detecting adverse events for patient safety research: A review of current methodologies, J. Biomed. Inform., 2003, vol. 36, nos. 1–2, pp. 131–143.CrossRefGoogle Scholar
  2. 2.
    Sarker, A., Ginn, R., Nikfarjam, A., O’Connor, K., Smith, K., Jayaraman, S., et al., Utilizing social media data for pharmacovigilance: A review, J. Biomed. Inform., 2015, vol. 54, pp. 202–212.CrossRefGoogle Scholar
  3. 3.
    Lardon, J., Abdellaoui, R., Bellet, F., Asfari, H., Souvignet, J., Texier, N., et al., Adverse drug reaction identification and extraction in social media: A scoping review, J. Med. Internet Res., 2015, vol. 17, no. 7.CrossRefGoogle Scholar
  4. 4.
    Harpaz, R., Callahan, A., Tamang, S., Low, Y., Odgers, D., Finlayson, S., et al., Text mining for adverse drug events: The promise, challenges, and state of the art, Drug Safety, 2014, vol. 37, pp. 777–790.CrossRefGoogle Scholar
  5. 5.
    Harpaz, R., DuMouchel, W., Shah, N.H., Madigan, D., Ryan, P., and Friedman, C., Novel data-mining methodologies for adverse drug event discovery and analysis, Clinical Pharmacology Therapeutics, 2012, vol. 91, no. 6, pp. 1010–1021.CrossRefGoogle Scholar
  6. 6.
    Tang, D., Qin, B., Feng, X., and Liu, T., Cited November 15, 2008.Google Scholar
  7. 7.
    Ma, D., Li, S., Zhang, X., and Wang, H., Interactive attention networks for aspect-level sentiment classification. arXiv preprint arXiv:1709 00893, 2017.Google Scholar
  8. 8.
    Tang, D., Qin, B., and Liu, T. Aspect level sentiment classification with deep memory network. arXiv preprint arXiv:1605 08900, 2016.Google Scholar
  9. 9.
    Chen, P., Sun, Z., Bing, L., and Yang, W., Recurrent attention network on memory for aspect sentiment analysis, Proc. of the 2017 Conference on Empirical Methods in Natural Language Processing, 2017, pp. 452–461.Google Scholar
  10. 10.
    Benton, A., Ungar, L., Hill, S., Hennessy, S., Mao, J., Chung, A., et al., Identifying potential adverse effects using the web: A new approach to medical hypothesis generation, J. Biomed. Inform., 2011, vol. 44, pp. 989–996.CrossRefGoogle Scholar
  11. 11.
    Yang, C.C., Yang, H., Jiang, L., and Zhang, M., Social media mining for drug safety signal detection, Proc. of the 2012 International Workshop on Smart Health and Wellbeing, 2012, pp. 33–40.Google Scholar
  12. 12.
    Liu, X. and Chen, H., AZDrugMiner: An information extraction system for mining patient-reported adverse drug events in online patient forums, Lect. Notes Comput. Sci., 2013, vol. 8040, pp. 134–150.CrossRefGoogle Scholar
  13. 13.
    Yeleswarapu, S., Rao, A., Joseph, T., Saipradeep, V.G., and Srinivasan, R., A pipeline to extract drug-adverse event pairs from multiple data sources, BMC Med. Inform. Decision Making, 2014, vol. 14, no. 13.Google Scholar
  14. 14.
    Freifeld, C.C., Brownstein, J.S., Menone, C.M., Bao, W., Filice, R., Kass-Hout, T., et al., Digital drug safety surveillance: Monitoring pharmaceutical products in Twitter, Drug Safety, 2014, vol. 37, pp. 343–350.CrossRefGoogle Scholar
  15. 15.
    O’Connor, K., Pimpalkhute, P., Nikfarjam, A., Ginn, R., Smith, K.L., and Gonzalez, G., Pharmacovigilance on Twitter? Mining tweets for adverse drug reactions, Proc. of the AMIA Annual Symposium, 2014, pp. 924–933.Google Scholar
  16. 16.
    Nikfarjam, A. and Gonzalez, G.H., Pattern mining for extraction of mentions of adverse drug reactions from user comments, Proc. of the AMIA Annual Symposium, 2011, pp. 1019–1026.Google Scholar
  17. 17.
    Na, J-C., Kyaing, W.Y.M., Khoo, C.S.G., Foo, S., Chang, Y-K., and Theng, Y-L., Sentiment classification of drug reviews using a rule-based linguistic approach, Lect. Notes Comput. Sci. 2012, vol. 7634, pp. 189–198.CrossRefGoogle Scholar
  18. 18.
    Yun Niu et al. Analysis of polarity information in medical text, Proc. of the AMIA Annual Symposium, 2005, pp. 570–574.Google Scholar
  19. 19.
    Leaman, R. et al., Towards Internet-age pharmacovigilance: Extracting adverse drug reactions from user posts to health-related social networks, Proc. of the 2010 Workshop on Biomedical Natural Language Processing, 2010, pp. 117–125.Google Scholar
  20. 20.
    Yun, N. Xiaodan, Z., et al., Predicting adverse drug events from personal health messages, Proc. of the AMIA Annual Symposium, 2011, pp. 217–226.Google Scholar
  21. 21.
    Bian, J., Topaloglu, U., and Yu, F., Towards large-scale twitter mining for drug-related adverse events, Proc. of the 2012 International Workshop on Smart Health and Wellbeing, 2012, pp. 25–32.Google Scholar
  22. 22.
    Yang, M., Wang, X., and Kiang, M.Y., Identification of consumer adverse drug reaction messages on social media, Proc. of the Pacific Asia Conference on Information Systems, 2013.Google Scholar
  23. 23.
    Sarker, A. and Gonzalez, G., Portable automatic text classification for adverse drug reaction detection via multi-corpus training, J. Biomed. Inform., 2015, vol. 53, pp. 196–207.CrossRefGoogle Scholar
  24. 24.
    Aramaki, E. et al., Extraction of adverse drug effects from clinical records, Studies Health Technol. Inform., 2010, vol. 160, no. 1, pp. 739-743.Google Scholar
  25. 25.
    Rastegar-Mojarad, M., Elayavilli, R.K., Yu, Y., and Liu, H., Detecting signals in noisy data-can ensemble classifiers help identify adverse drug reaction in tweets, Proc. of the Social Media Mining Shared Task Workshop at the Pacific Symposium on Biocomputing, 2016.Google Scholar
  26. 26.
    Sarker, A., Nikfarjam, A., and Gonzalez, G., Social media mining shared task workshop, Proc. of the Pacific Symposium on Biocomputing, 2016, pp. 581–592.Google Scholar
  27. 27.
    Sarker, A. and Gonzalez-Hernandez, G., Overview of the second social media mining for health (SMM) Shared Tasks at AMIA 2017, Proc. of the 2nd Social Media Mining for Health Research and Applications Workshop, 2017, pp. 43-48.Google Scholar
  28. 28.
    Kiritchenko, S., Mohammad, S.M., Morin, J., and de Bruijn, B., NRC-Canada at SMM4H shared task: Classifying tweets mentioning adverse drug reactions and medication intake. arXiv:1805 04558. 2018.Google Scholar
  29. 29.
    Friedrichs, J., Mahata, D., and Gupta, S. InfyNLP at SMM4H Task 2: Stacked ensemble of shallow convolutional neural networks for identifying personal medication intake from Twitter, 2018. arXiv preprint arXiv:1803 07718Google Scholar
  30. 30.
    Huynh, T., He, Y., Willis, A., and Ruger, S., Adverse drug reaction classification with deep neural networks, Proc. of the 26th International Conference on Computational Linguistics: Technical Papers, 2016, pp. 877–887.Google Scholar
  31. 31.
    Gurulingappa, H. Rajput, A.M., et al., Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports, J. Biomed. Inform., 2012, vol. 45, pp. 885–892.CrossRefGoogle Scholar
  32. 32.
    Serrano-Guerrero, J. Olivas, J.A., et al., Sentiment analysis: A review and comparative analysis of web services, Inform. Sci., 2015, vol. 311, pp. 18–38.CrossRefGoogle Scholar
  33. 33.
    Rusnachenko, N. and Loukachevitch, N., Using convolutional neural networks for sentiment attitude extraction from analytical texts, Proc. of the Third Workshop on Computational Linguistics and Language Science, Proc. of the CEUR Workshop, 2018.Google Scholar
  34. 34.
    Ivanov, V., Tutubalina, E., Mingazov, N., and Alimova, I., Extracting aspects, sentiment and categories of aspects in user reviews about restaurants and cars, Comput. Linguistics Intel. Technol. Papers from the Annual International Conference“Dialogue,” 2015, vol. 2, no. 14, pp. 22–34.Google Scholar
  35. 35.
    Solovyev, V. and Ivanov, V., Dictionary-based problem phrase extraction from user reviews, Lect. Notes Comput. Sci., 2014, vol. 8655, pp. 225–232.CrossRefGoogle Scholar
  36. 36.
    Zhang, L., Wang, S., and Liu, B., Deep learning for sentiment analysis. A survey, Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2018, vol. 8, no. 4.Google Scholar
  37. 37.
    Alimova, I. and Tutubalina, E., Automated detection of adverse drug reactions from social media posts with machine learning, Lect. Notes Comput. Sci., 2017, vol. 10716, pp. 3–15.CrossRefGoogle Scholar
  38. 38.
    Miftahutdinov, Z.S., Tutubalina, E.V., and Tropsha, A.E., Identifying disease-related expressions in reviews using conditional random fields, Comput. Linguistics Intel. Technol. Papers from the Annual International Conference“Dialogue,” 2017, vol. 1, no. 16, pp 155–166.Google Scholar
  39. 39.
    Korkontzelos, I. Nikfarjam, A., et al., Analysis of the effect of sentiment analysis on extracting adverse drug reactions from tweets and forum posts, J. Biomed. Inform., 2016, vol. 62, pp. 148–158.CrossRefGoogle Scholar
  40. 40.
    Dai, H.-J., Touray, M., Jonnagaddala, J., and Syed-Abdul, S., Feature engineering for recognizing adverse drug reactions from twitter posts, Information, 2016, vol. 7, no. 27.CrossRefGoogle Scholar
  41. 41.
    Karimi, S., Metke-Jimenez, A., Kemp, M., and Wang, C., Cadec: A corpus of adverse drug event annotations, J. Biomed. Inform., 2015, vol. 55, pp. 73–81.CrossRefGoogle Scholar
  42. 42.
    Nikfarjam, A., Sarker, A., et al., Pharmacovigilance from social media: Mining adverse drug reaction mentions using sequence labeling with word embedding cluster features, J.Amer. Med. Inform. Assoc., 2015, vol. 22, no. 3, pp. 671–681.Google Scholar
  43. 43.
    NLP challenges for detecting medication and adverse drug events from electronic health records (MADE 1.0), 2018, Massachusetts, Lowell, and Worcester, Amhers. Cited November 15, 2008.Google Scholar
  44. 44.
    Alvaro, N., Miyao, Y., and Collier, N., Twimed: Twitter and pubmed comparable corpus of drugs, diseases, symptoms, and their relations, JMIR Public Health and Surveillance, 2017, vol. 3, no. 2.CrossRefGoogle Scholar
  45. 45.
    Wilson, T., Wiebe, J., and Hoffmann, P., Recognizing contextual polarity in phrase-level sentiment analysis, Proc. of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, 2005, pp. 347–354.Google Scholar
  46. 46.
    Baccianella, S., Esuli, A., and Sebastiani, F., Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining, Proc. of the Seventh conference on International Language Resources and Evaluation, 2010, pp. 2200–2204.Google Scholar
  47. 47.
    Hu, M. and Liu, B., Mining and summarizing customer reviews, Proc, of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2004, pp. 168–177.Google Scholar

Copyright information

© Pleiades Publishing, Ltd. 2019

Authors and Affiliations

  1. 1.Kazan Federal UniversityKazanRussia

Personalised recommendations