Improving Text Categorization Using Domain Knowledge

  • Jingbo Zhu
  • Wenliang Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3513)


In this paper, we mainly study and propose an approach to improve document classification using domain knowledge. First we introduce a domain knowledge dictionary NEUKD, and propose two models which use domain knowledge as textual features for text categorization. The first one is BOTW model which uses domain associated terms and conventional words as textual features. The other one is BOF model which uses domain features as textual features. But due to limitation of size of domain knowledge dictionary, we study and use a machine learning technique to solve the problem, and propose a BOL model which could be considered as the extended version of BOF model. In the comparison experiments, we consider naïve Bayes system based on BOW model as baseline system. Comparison experimental results of naïve Bayes systems based on those four models (BOW, BOTW, BOF and BOL) show that domain knowledge is very useful for improving text categorization. BOTW model performs better than BOW model, and BOL and BOF models perform better than BOW model in small number of features cases. Through learning new features using machine learning technique, BOL model performs better than BOF model.


Textual Feature Domain Knowledge Domain Feature Training Corpus Topic Cluster 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ittner, D.J., Lewis, D.D., Ahn, D.D.: Text categorization of low quality images. In: Symposium on Document Analysis and Information Retrieval, Las Vegas, Las Vegas (1995)Google Scholar
  2. 2.
    Lewis, D., Schapire, R., Callan, J., Papka, R.: Training Algorithms for Linear Text Classifiers. In: Proceedings of ACM SIGIR, pp. 298–306 (1996)Google Scholar
  3. 3.
    Joachims, T.: Text categorization with Support Vector Machines: Learning with many relevant features. In: Machine Learning: ECML 1998, Tenth European Conference on Machine Learning, pp. 137–142 (1998)Google Scholar
  4. 4.
    Lewis, D.: A Comparison of Two Learning Algorithms for Text Categorization. In: Symposium on Document Analysis and IR (1994)Google Scholar
  5. 5.
    Nigam, K., Lafferty, J., McCallum, A.: Using maximum entropy for text classification. In: IJCAI 1999 Workshop on Machine Learning for Information Filtering, pp. 61–67 (1999)Google Scholar
  6. 6.
    McCallum, Nigam, K.: A Comparison of Event Models for naïve Bayes Text Classification. In: AAAI 1998 Workshop on Learning for Text Categorization (1998)Google Scholar
  7. 7.
    Scott, Matwin, S.S.: Text classification using WordNet hypernyms. In: Proceedings of the COLING/ACL Workshop on Usage of WordNet in Natural Language Processing Systems, Montreal (1998)Google Scholar
  8. 8.
    Lee, S., Shishibori, M.: Passage Segmentation Based on Topic Matter. Computer Processing of Oriental Languages 15(3), 305–340 (2002)CrossRefGoogle Scholar
  9. 9.
    Jingbo, Z., Tianshun, Y.: FIFA-based Text Classification. Journal of Chinese Information Processing 16(3) (2002) (in Chinese)Google Scholar
  10. 10.
    Wenliang, C., Jingbo, Z., Tianshun, Y.: Automatic Learning Field Words by Bootstrapping. In: Proceedings of the 7th national conference on computational linguistics, JSCL 2003 (2003) (in Chinese)Google Scholar
  11. 11.
    Yang, Y., Liu, X.: A re-examination of text categorization methods. In: Proceedings of ACM SIGIR Conference on Research and Development in Information Retrieval (1999)Google Scholar
  12. 12.
    Wenliang, C., Xingzhi, C., Huizhen, W., Jingbo, Z., Tianshun, Y.: Automatic Word Clustering for Text Categorization Using Global Information. In: AIRS 2004, Beijing (2004)Google Scholar
  13. 13.
    China Library Categorization Editorial Board. China Library Categorization, 4th ed., Beijing, Beijing Library Press (1999) (in Chinese) Google Scholar
  14. 14.
    Tianshun, Y., Jingbo, Z., li, Z., Ying, Y.: Natural Language Processing-research on making computers understand human languages, Tsinghua University Press (2002) (In Chinese)Google Scholar
  15. 15.
    Salton, G., McGill, M.J.: An introduction to modern information retrieval. McGraw-Hill, New York (1983)Google Scholar
  16. 16.
    Baker, L.D., MCallum, A.K.: Distributional clustering of words for text classification. In: Proc. 21st Int. ACM SIGIR Conf. on Research and Development in Information Retrieval, pp. 96–103 (1998)Google Scholar
  17. 17.
    Pereira, F., Tishby, N., Lee, L.: Distributional clustering of English words. In: 30th Annual Meeting of the ACL, pp. 183–190 (1993)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Jingbo Zhu
    • 1
  • Wenliang Chen
    • 1
  1. 1.Natural Language Processing Lab, Institute of Computer Software and TheoryNortheastern UniversityShenyangP.R. China

Personalised recommendations