Skip to main content
SpringerLink
Log in

A Multi-domain Named Entity Recognition Method Based on Part-of-Speech Attention Mechanism

  • Conference paper
  • First Online:
Computer Supported Cooperative Work and Social Computing (ChineseCSCW 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1042))

Abstract

Named entity recognition is an important and basic work in text mining. To overcome the shortcomings of existing multi-domain named entity recognition methods, a multi-domain named entity recognition method based on the part-of-speech attention mechanism, called BiLSTM-ATTENTION-CRF, was proposed in this paper. The domain dictionary was constructed to represent multi-domain semantic information and the BiLSTM network was used to capture the grammatical and syntactic features, as well as multi-domain semantic features in context information. A part-of-speech attention mechanism was designed to obtain the contribution weight of part-of-speech for entity recognition. Finally, a group of experiments were performed on the multi-domain dataset to compare various fusion strategies of multi-level entity information. The experimental results show that BiLSTM-ATTENTION-CRF has a high precision and recall rate, and can effectively recognizes the multi-domain named entities.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mikheev, A., Moens, M., Grover, C.: Named entity recognition without gazetteers. In: Ninth Conference on European Chapter of the Association for Computational Linguistics (EACL 1999), pp. 1–8. ACM Press (1999)

    Google Scholar 

  2. Chandel, A., Nagesh, P.C., Sarawagi, S.: Efficient batch top-k search for dictionary-based entity recognition. In: 22nd International Conference on Data Engineering (ICDE 2006), p. 28. ACM Press (2006)

    Google Scholar 

  3. Abacha, A.B., Zweigenbaum, P.: Medical entity recognition: a comparison of semantic and statistical methods. In: BioNLP 2011 Workshop (BioNLP 2011), pp. 56–64. ACM Press (2011)

    Google Scholar 

  4. Eftimov, T., Seljak, B.K., Korošec, P.: A rule-based named-entity recognition method for knowledge extraction of evidence-based dietary recommendations. PLoS One 12(6), e0179488 (2017)

    Article  Google Scholar 

  5. Gandhe, A., Rastrow, A., Hoffmeister, B.: Scalable language model adaptation for spoken dialogue systems. In: 2018 IEEE Spoken Language Technology Workshop (SLT 2018) (2018)

    Google Scholar 

  6. Teixeira, J., Sarmento, L., Oliveira, E.: A bootstrapping approach for training a NER with conditional random fields. In: Antunes, L., Pinto, H.S. (eds.) EPIA 2011. LNCS (LNAI), vol. 7026, pp. 664–678. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24769-9_48

    Chapter  Google Scholar 

  7. Ju, Z.F., Wang, J., Zhu, F.: Named entity recognition from biomedical text using SVM. In: 5th International Conference on Bioinformatics and Biomedical Engineering. IEEE Press (2011). https://doi.org/10.1109/icbbe.2011.5779984

  8. Morwal, S., Jahan, N., Chopra, D.: Named entity recognition using hidden markov model (HMM). Int. J. Nat. Lang. Comput. (IJNLC) 1(4), 15–23 (2012)

    Article  Google Scholar 

  9. Ding, P., Zhou, X.B., Zhang X.J., Wang, J., Lei, Z.F.: An attentive neural sequence labeling model for adverse drug reactions mentions extraction. IEEE Access 6 (2018). https://doi.org/10.1109/access.2018.2882443

  10. Lipenkova, J.: A system for fine-grained aspect-based sentiment analysis of Chinese. In: 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2015), pp. 55–60 (2015)

    Google Scholar 

  11. Liu, P.F., Joty, S., Meng, H.: Fine-grained opinion mining with recurrent neural networks and word embeddings. In: 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), pp. 1433–1443. ACL (2015)

    Google Scholar 

  12. Liu, Q., Liu, B., Zhang, Y., Kim, D.S., Gao, Z.: Improving opinion aspect extraction using semantic similarity and aspect associations. ACM SIGARCH Comput. Archit. News 44(3), 506–518 (2016)

    Article  Google Scholar 

  13. Silver, D.L., Yang, Q., Li, L.: Lifelong machine learning systems: beyond learning algorithms. In: AAAI 2013 Spring Symposium on Lifelong Machine Learning (2013)

    Google Scholar 

  14. Liu, P.F., Qiu, X.P., Chen, X.C, Wu, S.Y.: Multi-timescale long short-term memory neural network for modelling sentences and documents. In: 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), pp. 2326–2335 (2015)

    Google Scholar 

  15. Jakob, N., Gurevych, I.: Extracting opinion targets in a single- and cross-domain setting with conditional random fields. In: 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP 2010), pp. 1035–1045 (2010)

    Google Scholar 

  16. Zhao, Y.Y., Che, W.X., Guo, H.L., Qin, B, Su, Z., Liu, T.: Sentence compression for target-polarity word collocation extraction. In: 25th International Conference on Computational Linguistics (COLING 2014), pp. 1360–1369 (2014)

    Google Scholar 

  17. Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)

    Article  Google Scholar 

  18. Mikolov, T., Chen, K., Corrado, G.: Efficient estimation of word representations in vector space. Comput. Sci. (2013)

    Google Scholar 

  19. Beck, D., Cohn, T., Hardmeier, C., Specia, L.: Learning structural kernels for natural language processing. Trans. Assoc. Comput. Linguist. 3, 461–473 (2015)

    Article  Google Scholar 

  20. Shalaby, W., Zadrozny, W.: Mined semantic analysis: a new concept space model for semantic representation of textual data. In: 2017 IEEE International Conference on Big Data (Big Data 2017). IEEE Press (2017)

    Google Scholar 

  21. Ustun, V., Rosenbloom, P.S., Sagae, K., Demski, A.: Distributed vector representations of words in the sigma cognitive architecture. In: Goertzel, B., Orseau, L., Snaider, J. (eds.) AGI 2014. LNCS (LNAI), vol. 8598, pp. 196–207. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-09274-4_19

    Chapter  Google Scholar 

  22. Zilly, J.G., Srivastava, R.K., Koutník, J., Schmidhuber, J.: Recurrent highway networks. In: 34th International Conference on Machine Learning, pp. 4189–4198 (2017)

    Google Scholar 

  23. Strobelt, H., Gehrmann, S., Huber, B., Pfister, H.: LSTMVis: a tool for visual analysis of hidden state dynamics in recurrent neural networks. IEEE Trans. Visual. Comput. Graphics (2016)

    Google Scholar 

  24. Yu, Z., et al.: Using bidirectional LSTM recurrent neural networks to learn high-level abstractions of sequential features for automated scoring of non-native spontaneous speech. In: 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pp. 338–345. IEEE Press (2015)

    Google Scholar 

  25. Long, D., Zhang, R., Mao, Y.Y.: Prototypical recurrent unit. Neurocomputing 311, 146–154 (2018)

    Article  Google Scholar 

  26. Graves, A., Fernández, S., Schmidhuber, J.: Bidirectional LSTM networks for improved phoneme classification and recognition. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 799–804. Springer, Heidelberg (2005). https://doi.org/10.1007/11550907_126

    Chapter  Google Scholar 

  27. Chen, Y., et al.: Named entity recognition from Chinese adverse drug event reports with lexical feature based BiLSTM-CRF and tri-training. J. Biomed. Inform. 96 (2019). https://doi.org/10.1016/j.jbi.2019.103252

  28. Sasaki, Y., et al.: Local and global attention are mapped retinotopically in human occipital cortex. Natl. Acad. Sci. U.S.A. 98(4), 2077–2082 (2001)

    Article  Google Scholar 

  29. Quan, C.Q., Ren, F.J.: Target based review classification for fine-grained sentiment analysis. Int. J. Innov. Comput. Inf. Control 10(1), 257–268 (2016)

    Google Scholar 

  30. Huang, H.J., Li, Z.-C.: A multiclass, multicriteria logit-based traffic equilibrium assignment model under ATIS. Eur. J. Oper. Res. 176(3), 1464–1477 (2007)

    Article  Google Scholar 

  31. HanLP Tool. https://github.com/hankcs/HanLP

  32. Zhang, S., Lin, S.F., Gao, J.F., Chen, J.H.: Recognizing small-sample biomedical named entity based on contextual domain relevance. In: 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC). IEEE Press (2019)

    Google Scholar 

  33. Dong, G.C., Chen, J.H., Wang, H.Y., Zhong, N.: A narrow-domain entity recognition method based on domain relevance measurement and context information. In: 2017 IEEE/WIC/ACM International Conference on Web Intelligence (WI 2017), pp. 623–628. ACM Press (2017)

    Google Scholar 

  34. Jagannatha, A.N., Yu, H.: Structured prediction models for RNN based sequence labeling in clinical text. In: 2016 Conference on Empirical Methods in Natural Language Processing, pp. 856–865 (2016)

    Google Scholar 

  35. Miwa, M., Bansal, M.: End-to-end relation extraction using LSTMs on sequences and tree structures. In: 54th Annual Meeting of the Association for Computational Linguistics, pp. 1105–1116 (2016)

    Google Scholar 

Download references

Acknowledgment

The work received support from Science and Technology Project of Beijing Municipal Commission of Education (No. KM201710005026), National Basic Research Program of China (No. 2014CB744600), Open Foundation of Beijing Key Laboratory of MRI and Brain Informatics, Open Foundation of Beijing Key Laboratory of Multimedia and Intelligent Software (Beijing University of Technology).

Author information

Authors and Affiliations

  1. Faculty of Information Technology, Beijing University of Technology, Beijing, 100024, China

    Shun Zhang, Ying Sheng, Jiangfan Gao, Jianhui Chen, Jiajin Huang & Shaofu Lin

  2. Beijing Institute of Smart City, Beijing University of Technology, Beijing, 100024, China

    Shaofu Lin

  3. Beijing Key Laboratory of MRI and Brain Informatics, Beijing, China

    Jianhui Chen & Jiajin Huang

Authors
  1. Shun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying Sheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiangfan Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jianhui Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jiajin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shaofu Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianhui Chen .

Editor information

Editors and Affiliations

  1. Shandong University, Jinan, China

    Yuqing Sun

  2. Fudan University, Shanghai, China

    Tun Lu

  3. Kunming University of Science and Technology, Kunming, China

    Zhengtao Yu

  4. Tongji University, Shanghai, China

    Hongfei Fan

  5. University of Shanghai for Science and Technology, Shanghai, China

    Liping Gao

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, S., Sheng, Y., Gao, J., Chen, J., Huang, J., Lin, S. (2019). A Multi-domain Named Entity Recognition Method Based on Part-of-Speech Attention Mechanism. In: Sun, Y., Lu, T., Yu, Z., Fan, H., Gao, L. (eds) Computer Supported Cooperative Work and Social Computing. ChineseCSCW 2019. Communications in Computer and Information Science, vol 1042. Springer, Singapore. https://doi.org/10.1007/978-981-15-1377-0_49

Download citation

  • DOI: https://doi.org/10.1007/978-981-15-1377-0_49

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-15-1376-3

  • Online ISBN: 978-981-15-1377-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation