Technical Term Recognition with Semi-supervised Learning Using Hierarchical Bayesian Language Models

  • Ryo Fujii
  • Akito Sakurai
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7337)


To recognize technical term, term dictionaries or tagged corpora are required, but it will take much cost to compile them. Moreover, the terms may have several representations and new terms may be developed, which complicates the problem further, that is, a simple dictionary building can’t solve the problem. In this research, to reduce the cost of creating dictionaries, we aimed at building a system that learns to recognize terminology from small tagged corpus using semi-supervised learning. We solved the problem by combining a tag level language model and a character level language model based on HPYLM.

We performed experiments on recognition of biomedical terms. In supervised learning, we achived 65% F-measure which is 8% points behind the best existing system that utilizes many domain specific heuristics. In semi-supervised learning, we could keep the accuracy against reduction of supervised data better than exisiting methods.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Zhou, G.D., Su, J.: Exploring deep knowledge resources in biomedical name recognition. In: JNLPBA 2004, pp. 96–99. ACL (2004)Google Scholar
  2. 2.
    Song, Y., Kim, E., Lee, G.G., Yi, B.-K.: Posbiotm-ner in the shared task of bionlp/nlpba 2004. In: JNLPBA 2004, pp. 100–103. ACL (2004)Google Scholar
  3. 3.
    Mochihashi, D., Sumita, E.: The infinite markov model. In: NIPS 2007, pp. 1–2 (2007)Google Scholar
  4. 4.
    Teh, Y.W.: A hierarchical bayesian language model based on pitman-yor processes. In: ACL, pp. 985–992 (2006)Google Scholar
  5. 5.
    Mochihashi, D., Yamada, T., Ueda, N.: Bayesian unsupervised word segmentation with hierachical language modeling. ACL 1(36), 49 (2009)Google Scholar
  6. 6.
    Zhao, S.: Name entity recognition in biomedical text using a hmm model. In: JNLPBA 2004, pp. 84–87. ACL (2004)Google Scholar
  7. 7.
    Lee, C., Hou, W.J., Chen, H.-H.: Annotating multiple types of biomedical entities: a single word classification approach. In: JNLPBA 2004, pp. 80–83. ACL (2004)Google Scholar
  8. 8.
    Park, K.-M., Kim, S.-H., Lee, K.-J., Lee, D.-G., Rim, H.-C.: Incorporating lexical knowledge into biomedical ne recognition. In: JNLPBA 20, pp. 76–79. ACL (2004)Google Scholar
  9. 9.
    Jiao, F., Wang, S., Lee, C.-H., Greiner, R., Schuurmans, D.: Semi-supervised conditional random fields for improved sequence segmentation and labeling. In: ACL, pp. 209–216 (2006)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Ryo Fujii
    • 1
  • Akito Sakurai
    • 1
  1. 1.Keio UniversityYokohamaJapan

Personalised recommendations