Exploring Deep Belief Nets to Detect and Categorize Chinese Entities

  • Yu Chen
  • Dequan Zheng
  • Tiejun Zhao
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8346)


This paper adapts a novel model, deep belief nets (DBN), to extract entity mentions in Chinese documents. Our experiments were designed to develop entity detection system and entity categorization system applying DBN, to complete entity extraction. Our results exhibit how the depth of architecture and quantity of unit in hidden layer of DBN influence the performance. In DBN Systems, token labels are produced independently and DBN does not concerned what the labels of surrounding token are. Viterbi algorithm is a good solution to overcome this issue. It can find the most likely probability label path to make DBN to be more suitable for entity detection. Furthermore, this paper demonstrates DBN is proper model for our tasks and its results are better than Support Vector Machine (SVM), Artificial Neural Network (ANN) and Conditional Random Field (CRF).


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hinton, G.E., Osindero, S., Teh, Y.: A fast learning algorithm for deep belief nets. Neural Computation 18, 1527–1554 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  2. 2.
    Humphreys, K., Gaizauskas, R., Azzam, S., Huyck, C., Mitchell, B., Cunningham, H., Wilks, Y.: University of Sheffield: Description of the. LaSIE-II System as Used for MUC-7. In: Proceedings of 7th Message Understanding Conference, pp. 1–12 (1998)Google Scholar
  3. 3.
    Zhou, G., Su, J.: Named Entity Recognition using an HMM-based Chunk Tagger. In: Proceeding of the 40th Annual Meeting of the Association for Computation Linguistics (ACL), pp. 473–480 (2002)Google Scholar
  4. 4.
    Ratinov, L., Roth, D., Downey, D., Anderson, M.: Local and Global Algorithms for Disambiguation to Wikipedia. In: Proc. of the Annual Meeting of the Association of Computational Linguistics, ACL (2011)Google Scholar
  5. 5.
    Wiriyathammabhum, Kijssirikul, B., Takamura, H., Okumura, M.: Applying Deep Belief Networks to Word Sense Disambiguation. CoRR (2012)Google Scholar
  6. 6.
    Wang, B., Wang, X., Sun, C., Liu, B., Sun, L.: Modeling semantic relevance for question-answer pairs in web social communities. In: Proceeding of the 48th Annunnal Meeting of the Association for Computational Linguistics (2010)Google Scholar
  7. 7.
    Ratinov, L., Roth, D.: Design challenges and misconceptions in named entity recognition. In: Proceeding of the Thirteenth Conference on Computational Natural Language Learning (CoNLL), pp. 147–155 (2009)Google Scholar
  8. 8.
    Tseng, H., Chang, P., Andrew, G., Jurafsky, D.: Christopher Manning. A Conditional Random Field Word Segmenter for SighanBakeoff 2005. In: Fourth SIGHAN Workshop on Chinese Language Processing (2005)Google Scholar
  9. 9.
    Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Computation 14(8), 1711–1800 (2002)CrossRefMathSciNetGoogle Scholar
  10. 10.
    Bengio, Y., LeCun, Y.: Scaling learning algorithms towards ai. Large-Scale Kernel Machines. MIT Press (2007)Google Scholar
  11. 11.
    Hinton, G.: Products of experts. In: Proceedings of the Ninth International Conference on Artificial Neural Networks (ICANN), vol. 1, pp. 1–6 (1999)Google Scholar
  12. 12.
    Chen, Y., Ouyang, Y., Li, W., Zheng, D., Zhao, T.: Using Deep Belief Nets for Chinese Named Entity Categorization. In: Proceeding of the 2010 Named Entity Workshop (NEWS 2010), pp. 102–115 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Yu Chen
    • 1
  • Dequan Zheng
    • 1
  • Tiejun Zhao
    • 1
  1. 1.Computer Science and Technology DepartmentHarbin Institute of TechnologyChina

Personalised recommendations