Multi-task Learning for Extraction of Adverse Drug Reaction Mentions from Tweets

  • Shashank Gupta
  • Manish GuptaEmail author
  • Vasudeva Varma
  • Sachin Pawar
  • Nitin Ramrakhiyani
  • Girish Keshav Palshikar
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10772)


Adverse drug reactions (ADRs) are one of the leading causes of mortality in health care. Current ADR surveillance systems are often associated with a substantial time lag before such events are officially published. On the other hand, online social media such as Twitter contain information about ADR events in real-time, much before any official reporting. Current state-of-the-art in ADR mention extraction uses Recurrent Neural Networks (RNN), which typically need large labeled corpora. Towards this end, we propose a multi-task learning based method which can utilize a similar auxiliary task (adverse drug event detection) to enhance the performance of the main task, i.e., ADR extraction. Furthermore, in absence of the auxiliary task dataset, we propose a novel joint multi-task learning method to automatically generate weak supervision dataset for the auxiliary task when a large pool of unlabeled tweets is available. Experiments with \(\sim \)0.48M tweets show that the proposed approach outperforms the state-of-the-art methods for the ADR mention extraction task by \(\sim \)7.2 % in terms of F1 score.


Multi-task learning Pharmacovigilance Neural networks 


  1. 1.
    Alvaro, N., Miyao, Y., Collier, N.: TwiMed: Twitter and PubMed comparable corpus of drugs, diseases, symptoms, and their relations. JMIR Public Health Surveill. 3(2) (2017)Google Scholar
  2. 2.
    Ando, R.K., Zhang, T.: A framework for learning predictive structures from multiple tasks and unlabeled data. JMLR 6, 1817–1853 (2005)MathSciNetzbMATHGoogle Scholar
  3. 3.
    Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: NIPS, pp. 41–48 (2006)Google Scholar
  4. 4.
    Caruana, R.: Multitask learning: a knowledge-based source of inductive bias. In: ICML, pp. 41–48 (1993)Google Scholar
  5. 5.
    Cocos, A., Fiks, A.G., Masino, A.J.: Deep learning for pharmacovigilance: recurrent neural network architectures for labeling adverse drug reactions in Twitter posts. JAMIA, p. ocw180 (2017)Google Scholar
  6. 6.
    Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: ICML, pp. 160–167 (2008)Google Scholar
  7. 7.
    Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. JMLR 12(Aug), 2493–2537 (2011)zbMATHGoogle Scholar
  8. 8.
    Evgeniou, T., Pontil, M.: Regularized multi-task learning. In: KDD, pp. 109–117 (2004)Google Scholar
  9. 9.
    Fralick, S.: Learning to recognize patterns without a teacher. IEEE Trans. Inf. Theory 13(1), 57–64 (1967)CrossRefGoogle Scholar
  10. 10.
    Freifeld, C.C., Brownstein, J.S., Menone, C.M., Bao, W., Filice, R., Kass-Hout, T., Dasgupta, N.: Digital drug safety surveillance: monitoring pharmaceutical products in Twitter. Drug Saf. 37(5), 343–350 (2014)CrossRefGoogle Scholar
  11. 11.
    Godin, F., Vandersmissen, B., De Neve, W., Van de Walle, R.: Multimedia Lab@ ACL W-Nut NER shared task: named entity recognition for twitter microposts using distributed word representations. In: ACL-ICJNLP 2015, pp. 146–153 (2015)Google Scholar
  12. 12.
    Graves, A.: Sequence transduction with recurrent neural networks. CoRR abs/1211.3711 (2012)Google Scholar
  13. 13.
    Hazell, L., Shakir, S.A.: Under-reporting of adverse drug reactions: a systematic review. Pharmacoepidemiol. Drug Saf. 14, S184–S185 (2005)Google Scholar
  14. 14.
    Iosifidis, V., Ntoutsi, E.: Large scale sentiment learning with limited labels. In: KDD, pp. 1823–1832. ACM (2017)Google Scholar
  15. 15.
    Kim, Y.: Convolutional neural networks for sentence classification. In: EMNLP, pp. 1746–1751 (2014)Google Scholar
  16. 16.
    Kingma, D., Ba, J.: Adam: a method for stochastic optimization. arXiv:1412.6980 (2014)
  17. 17.
    LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)CrossRefGoogle Scholar
  18. 18.
    Liu, X., Gao, J., He, X., Deng, L., Duh, K., Wang, Y.: Representation learning using multi-task deep neural networks for semantic classification and information retrieval. In: NAACL-HLT, pp. 912–921 (2015)Google Scholar
  19. 19.
    Luong, M., Le, Q.V., Sutskever, I., Vinyals, O., Kaiser, L.: Multi-task sequence to sequence learning. CoRR abs/1511.06114 (2015)Google Scholar
  20. 20.
    Màrquez, L., Callison-Burch, C., Su, J., Pighin, D., Marton, Y. (eds.): EMNLP (2015)Google Scholar
  21. 21.
    Nikfarjam, A., Sarker, A., OConnor, K., Ginn, R., Gonzalez, G.: Pharmacovigilance from social media: mining adverse drug reaction mentions using sequence labeling with word embedding cluster features. JAMIA 22(3), 671–681 (2015)Google Scholar
  22. 22.
    Stanovsky, G., Gruhl, D., Mendes, P.N.: Recognizing mentions of adverse drug reaction in social media using knowledge-infused recurrent models. In: EACL, pp. 142–151 (2017)Google Scholar
  23. 23.
    Tsai, R.T.H., Wu, S.H., Chou, W.C., Lin, Y.C., He, D., Hsiang, J., Sung, T.Y., Hsu, W.L.: Various criteria in the evaluation of biomedical named entity recognition. BMC Bioinform. 7(1), 92 (2006)CrossRefGoogle Scholar
  24. 24.
    Vieira, H.S., da Silva, A.S., Cristo, M., de Moura, E.S.: A self-training CRF method for recognizing product model mentions in web forums. In: Hanbury, A., Kazai, G., Rauber, A., Fuhr, N. (eds.) ECIR 2015. LNCS, vol. 9022, pp. 257–264. Springer, Cham (2015). Google Scholar
  25. 25.
    Wang, Z., Zhang, J., Feng, J., Chen, Z.: Knowledge graph embedding by translating on hyperplanes. In: AAAI, pp. 1112–1119 (2014)Google Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Shashank Gupta
    • 1
  • Manish Gupta
    • 1
    Email author
  • Vasudeva Varma
    • 1
  • Sachin Pawar
    • 2
  • Nitin Ramrakhiyani
    • 2
  • Girish Keshav Palshikar
    • 2
  1. 1.International Institute of Information Technology-HyderabadHyderabadIndia
  2. 2.TCS ResearchPuneIndia

Personalised recommendations