Skip to main content
SpringerLink
Log in

Exploring Recurrent Neural Networks to Detect Named Entities from Biomedical Text

  • Conference paper
  • First Online:
Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data (CCL 2015, NLP-NABD 2015)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9427))

Abstract

Biomedical named entity recognition (bio-NER) is a crucial and basic step in many biomedical information extraction tasks. However, traditional NER systems are mainly based on complex hand-designed features which are derived from various linguistic analyses and maybe only adapted to specified area. In this paper, we construct Recurrent Neural Network to identify entity names with word embeddings input rather than hand-designed features. Our contributions mainly include three aspects: (1) we adapt a deep learning architecture Recurrent Neural Network (RNN) to entity names recognition; (2) based on the original RNNs such as Elman-type and Jordan-type model, an improved RNN model is proposed; (3) considering that both past and future dependencies are important information, we combine bidirectional recurrent neural networks based on information entropy at the top layer. The experiments conducted on the BioCreative II GM data set demonstrate RNN models outperform CRF and deep neural networks (DNN), furthermore, the improved RNN model performs better than two original RNN models and the combined method is effective.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Notes

  1. 1.

    http://deeplearning.net/tutorial/rnnslu.html.

References

  1. Chen, Y., Ouyang, Y., Li, W., Zheng, D., Zhao, T.: Using deep belief nets for chinese named entity categorization. In: Proceedings of the 2010 Named Entities Workshop, pp. 102–109 (2010)

    Google Scholar 

  2. Li, L., Fan, W., Huang, D., Dang, Y., Sun, J.: Boosting performance of gene mention tagging system by hybrid methods. J. Biomed. Inform. 45, 156–164 (2012)

    Article  Google Scholar 

  3. Lee, K.J., Hwang, Y.S., Kim, S., Rim, H.C.: Biomedical named entity recognition using two-phase model based on SVMs. J. Biomed. Inform. 37, 436–447 (2004)

    Article  Google Scholar 

  4. Saha, S.K., Sarkar, S., Mitra, P.: Feature selection techniques for maximum entropy based biomedical named entity recognition. J. Biomed. Inform. 45, 2673–2681 (2009)

    Google Scholar 

  5. Shen, D., Zhang, J., Zhou, G., Su, J., Tan, C.L.: Effective adaptation of a hidden markov model-based named entity recognizer for biomedical domain. In: Proceedings of the ACL 2003 Workshop on Natural Language Processing in Biomedicine, vol. 13, pp. 49–56 (2003)

    Google Scholar 

  6. Sun, C., Guan, Y., Wang, X., Lin, L.: Rich features based conditional random fields for biological named entities recognition. Comput. Biol. Med. 37, 1327–1333 (2007)

    Article  Google Scholar 

  7. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)

    MATH  Google Scholar 

  8. Chen, Y., Zheng, D., Zhao, T.: Exploring deep belief nets to detect and categorize chinese entities. In: Motoda, H., Wu, Z., Cao, L., Zaiane, O., Yao, M., Wang, W. (eds.) ADMA 2013, Part I. LNCS, vol. 8346, pp. 468–480. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  9. Hammerton, J.: Named entity recognition with long short-term memory. In: Proceedings of the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, vol. 4, pp. 172–175 (2003)

    Google Scholar 

  10. Mikolov, T., Karafiat, M., Burget, L., Cernoky, J., Khudanpur, S.: Recurrent neural network based language model. In: 11th Annual Conference of the International Speech Communication Association, INTERSPEECH 2010, Makuhari, Chiba, Japan, pp. 1045–1048 (2010)

    Google Scholar 

  11. Mikolov, T., Kombrink, S., Burget, L., Cernocky, J., Khudanpur, S.: Extensions of recurrent neural network language model. In: Acoustics, Speech and Signal Processing (ICASSP), pp. 5528–5531 (2011)

    Google Scholar 

  12. Mesnil, G., He, X., Deng., Bengio Y.: Investigation of recurrent neural network architectures and learning methods for spoken language understanding. In: INTERSPEECH, pp. 3771–3775 (2013)

    Google Scholar 

  13. Elman, J.L.: Finding structure in time. Cogn. Sci. 14, 179–211 (1990)

    Article  Google Scholar 

  14. Jordan, M.I.: Serial order: A parallel distributed processing approach. Adv. Psychol. 121, 471–495 (1997)

    Article  Google Scholar 

  15. Mikolov, T., Zweig, G.: Context dependent recurrent neural network language model. In: SLT, pp. 234–239 (2012)

    Google Scholar 

  16. Graves, A.: Supervised sequence labelling. In: Graves, A. (ed.) Supervised Sequence Labelling with Recurrent Neural Networks. SCI, vol. 385, pp. 5–13. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  17. Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45, 2673–2681 (1997)

    Article  Google Scholar 

  18. Turian, J., Ratinov, L., Bengio, Y.: Word representations : a simple and general method for semi-supervised learning. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 384–394 (2010)

    Google Scholar 

  19. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient Estimation of Word Representations in Vector Space. arXiv preprint arXiv:1301.3781(2013)

  20. Ratinov, L., Roth, D.: Design challenges and misconceptions in named entity recognition. In: Proceedings of the Thirteenth Conference on Computational Natural Language Learning, pp. 147–155 (2009)

    Google Scholar 

Download references

Acknowledgment

The authors gratefully acknowledge the financial support provided by the National Natural Science Foundation of China under No. 61173101, 61173100.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Dalian University of Technology, Dalian, 116024, Liaoning, China

    Lishuang Li, Liuke Jin & Degen Huang

Authors
  1. Lishuang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liuke Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Degen Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lishuang Li .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Maosong Sun

  2. Tsinghua University, Beijing, China

    Zhiyuan Liu

  3. Soochow University, Suzhou, Jiangsu, China

    Min Zhang

  4. Tsinghua University, Beijing, China

    Yang Liu

Rights and permissions

Open Access This chapter is licensed under the terms of the Creative Commons Attribution-NonCommercial 2.5 International License (http://creativecommons.org/licenses/by-nc/2.5/), which permits any noncommercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Li, L., Jin, L., Huang, D. (2015). Exploring Recurrent Neural Networks to Detect Named Entities from Biomedical Text. In: Sun, M., Liu, Z., Zhang, M., Liu, Y. (eds) Chinese Computational Linguistics and Natural Language Processing Based on Naturally Annotated Big Data. CCL NLP-NABD 2015 2015. Lecture Notes in Computer Science(), vol 9427. Springer, Cham. https://doi.org/10.1007/978-3-319-25816-4_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25816-4_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25815-7

  • Online ISBN: 978-3-319-25816-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation