Advertisement

Leveraging Multi-task Learning for Biomedical Named Entity Recognition

  • Tahir MehmoodEmail author
  • Alfonso Gerevini
  • Alberto Lavelli
  • Ivan Serina
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11946)

Abstract

Biomedical named entity recognition (BioNER) is the task of categorizing biomedical entities. Due to the specific characteristics of the names of biomedical entities, such as ambiguity among different concepts or different ways of referring to the same entity, the BioNER task is usually considered more challenging compared to standard named entity recognition tasks. Recent techniques based on deep learning not only significantly reduce the hand crafted feature engineering phase but also determined relevant improvements in the BioNER task. However, such systems are still facing challenges. One of them is the limited availability of annotated text data. Multi-task learning approaches tackle this problem by training different related tasks simultaneously. This enables multi-task models to learn common features among different tasks where they share some layers. To explore the advantages of the multi-task learning, we propose a model based on convolution neural networks, long-short term memories, and conditional random fields. The model we propose shows comparable results to state-of-the-art approaches. Moreover, we present an empirical analysis considering the impact of different word input representations (word embedding, character embedding, and case embedding) on the model performance.

Keywords

Biomedical named entity recognition Multi-task learning Bidirectional long short-term memory 

References

  1. 1.
    Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: Advances in Neural Information Processing Systems, pp. 41–48 (2007)Google Scholar
  2. 2.
    Bansal, T., Belanger, D., McCallum, A.: Ask the GRU: multi-task learning for deep text recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 107–114. ACM (2016)Google Scholar
  3. 3.
    Chieu, H.L., Ng, H.T.: Named entity recognition: a maximum entropy approach using global information. In: Proceedings of the 19th International Conference on Computational Linguistics-Volume 1, pp. 1–7. Association for Computational Linguistics (2002)Google Scholar
  4. 4.
    Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12(Aug), 2493–2537 (2011)zbMATHGoogle Scholar
  5. 5.
    Crichton, G., Pyysalo, S., Chiu, B., Korhonen, A.: A neural network multi-task learning approach to biomedical named entity recognition. BMC Bioinform. 18(1), 368 (2017)CrossRefGoogle Scholar
  6. 6.
    Dugas, F., Nichols, E.: DeepNNNER: applying BLSTM-CNNs and extended lexicons to named entity recognition in tweets. In: Proceedings of the 2nd Workshop on Noisy User-generated Text (WNUT), pp. 178–187 (2016)Google Scholar
  7. 7.
    Giorgi, J.M., Bader, G.D.: Transfer learning for biomedical named entity recognition with neural networks. Bioinformatics 34(23), 4087–4094 (2018)CrossRefGoogle Scholar
  8. 8.
    Gridach, M.: Character-level neural network for biomedical named entity recognition. J. Biomed. Inform. 70, 85–91 (2017)CrossRefGoogle Scholar
  9. 9.
    Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. CoRR abs/1508.01991 (2015). http://arxiv.org/abs/1508.01991
  10. 10.
    Li, Z., Hoiem, D.: Learning without forgetting. IEEE Trans. Pattern Anal. Mach. Intell. 40(12), 2935–2947 (2018).  https://doi.org/10.1109/TPAMI.2017.2773081CrossRefGoogle Scholar
  11. 11.
    Limsopatham, N., Collier, N.: Learning orthographic features in bi-directional LSTM for biomedical named entity recognition. In: Proceedings of the Fifth Workshop on Building and Evaluating Resources for Biomedical Text Mining, BioTxtM@COLING 2016, Osaka, Japan, 12 December 2016, pp. 10–19 (2016). https://aclanthology.info/papers/W16-5102/w16-5102
  12. 12.
    Luong, M., Le, Q.V., Sutskever, I., Vinyals, O., Kaiser, L.: Multi-task sequence to sequence learning. In: 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, 2–4 May 2016, Conference Track Proceedings (2016). http://arxiv.org/abs/1511.06114
  13. 13.
    Pohlert, T.: The pairwise multiple comparison of mean ranks package (PMCMR)Google Scholar
  14. 14.
    dos Santos, C., Guimaraes, V., Niterói, R., de Janeiro, R.: Boosting named entity recognition with neural character embeddings. In: Proceedings of NEWS 2015 The Fifth Named Entities Workshop, p. 25 (2015)Google Scholar
  15. 15.
    Segura-Bedmar, I., Suárez-Paniagua, V., Martínez, P.: Exploring word embedding for drug name recognition. In: Proceedings of the Sixth International Workshop on Health Text Mining and Information Analysis, Louhi@EMNLP 2015, Lisbon, Portugal, 17 September 2015, pp. 64–72 (2015).  https://doi.org/10.18653/v1/W15-2608
  16. 16.
    Sheldon, M.R., Fillyaw, M.J., Thompson, W.D.: The use and interpretation of the Friedman test in the analysis of ordinal-scale data in repeated measures designs. Physiother. Res. Int. 1(4), 221–228 (1996)CrossRefGoogle Scholar
  17. 17.
    Wang, X., et al.: Cross-type biomedical named entity recognition with deep multi-task learning. Bioinformatics 35(10), 1745–1752 (2019)CrossRefGoogle Scholar
  18. 18.
    Yang, J., Liang, S., Zhang, Y.: Design challenges and misconceptions in neural sequence labeling. In: Proceedings of the 27th International Conference on Computational Linguistics, COLING 2018, Santa Fe, New Mexico, USA, 20–26 August 2018, pp. 3879–3889 (2018). https://aclanthology.info/papers/C18-1327/c18-1327
  19. 19.
    Zimmerman, D.W., Zumbo, B.D.: Relative power of the Wilcoxon test, the Friedman test, and repeated-measures ANOVA on ranks. J. Exp. Educ. 62(1), 75–86 (1993)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Tahir Mehmood
    • 1
    • 2
    Email author
  • Alfonso Gerevini
    • 1
  • Alberto Lavelli
    • 2
  • Ivan Serina
    • 1
  1. 1.University of BresciaBresciaItaly
  2. 2.Fondazione Bruno KesslerPovoItaly

Personalised recommendations