A Highly Effective Hybrid Model for Sentence Categorization

  • Zhenhong Chen
  • Kai Yang
  • Yi CaiEmail author
  • Dongping Huang
  • Ho-fung Leung
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9645)


Sentence categorization is a task to classify sentences by their types, which is very useful for the analysis of many NLP applications. There exist grammar or syntactic rules to determine types of sentences. And keywords like negation word for negative sentences is an important feature. However, no all sentences have rules to classify. Besides, different types of sentences may contain the same keywords whose meaning may be changed by context. We address the first issue by proposing a hybrid model consisting of Decision Trees and Support Vector Machines. In addition, we design a new feature based on N-gram model. The results of the experiments conducted on the sentence categorization dataset in “Good Ideas of China” Competition 2015 show that (1) our model outperforms baseline methods and all online systems in this competition; (2) the effectiveness of our feature is higher than that of features frequently used in NLP.


Sentence categorization Hybrid model N-grams Feature 



This work is supported by National Natural Science Foundation of China (project no. 61300137), and NEMODE Network Pilot Study: A Computational Taxonomy of Business Models of the Digital Economy, P55805.


  1. 1.
    Altman, N.S.: An introduction to kernel and nearest-neighbor nonparametric regression. Am. Stat. 46(3), 175–185 (1992)MathSciNetGoogle Scholar
  2. 2.
    Billsus, D., Pazzani, M.J.: A hybrid user model for news story classification. In: Kay, J. (ed.) UM99 User Modeling. CISM International Centre for Mechanical Sciences, vol. 407, pp. 99–108. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  3. 3.
    Burrows, W.R., Benjamin, M., Beauchamp, S., Lord, E.R., McCollor, D., Thomson, B.: Cart decision-tree statistical analysis and prediction of summer season maximum surface ozone for the vancouver, montreal, and atlantic regions of canada. J. Appl. Meteorol. 34(8), 1848–1862 (1995)CrossRefGoogle Scholar
  4. 4.
    Chen, Z.C.: Research on chinese negation and speculation identification. Master’s thesis, Suzhou University (2014)Google Scholar
  5. 5.
    Comorovski, I.: Interrogative Phrases and the Syntax-Semantics Interface, vol. 59. Springer Science & Business Media, New York (2013)Google Scholar
  6. 6.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  7. 7.
    Devroye, L., Györfi, L., Lugosi, G.: A Probabilistic Theory of Pattern Recognition, vol. 31. Springer Science & Business Media, New York (2013)zbMATHGoogle Scholar
  8. 8.
    Goryachev, S., Sordo, M., Zeng, Q.T., Ngo, L.: Implementation and Evaluation of Four Different Methods of Negation Detection. DSG, Boston (2006)Google Scholar
  9. 9.
    Gupta, P., Gupta, V.: A survey of text question answering techniques. Int. J. Comput. Appl. 53(4), 1–8 (2012)CrossRefGoogle Scholar
  10. 10.
    Khashei, M., Hamadani, A.Z., Bijari, M.: A novel hybrid classification model of artificial neural networks and multiple linear regression models. Expert Syst. Appl. 39(3), 2606–2620 (2012)CrossRefGoogle Scholar
  11. 11.
    Kohavi, R.: Scaling up the accuracy of naive-bayes classifiers: a decision-tree hybrid. In: KDD, pp. 202–207. Citeseer (1996)Google Scholar
  12. 12.
    Lan, N.: Basic semantic study on wh-words. In: The Northern Forum, vol. 4, p. 013 (2005)Google Scholar
  13. 13.
    Liu, B., Zhang, L.: A survey of opinion mining and sentiment analysis. In: Aggarwal, C.C., Zhai, C. (eds.) Mining Text Data, pp. 415–463. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  14. 14.
    Na-na, T., Han, Y.L.: Mechanisms and characteristics of grammaticalization in interrogative words. Foreign Lang. Res. 5, 016 (2009)Google Scholar
  15. 15.
    Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)Google Scholar
  16. 16.
    Quinlan, J.R.: Improved use of continuous attributes in c4.5. J. Artif. Intell. Res. 4, 77–90 (1996)zbMATHGoogle Scholar
  17. 17.
    Quinlan, J.R.: C4.5: programs for machine learning. Mach. Learn. 16(3), 235–240 (2014)Google Scholar
  18. 18.
    Rowlett, P.: On the syntactic derivation of negative sentence adverbials. J. Fr. Lang. Stud. 3(01), 39–69 (1993)CrossRefGoogle Scholar
  19. 19.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach. Prentice Hall, Upper Saddle River (1995)zbMATHGoogle Scholar
  20. 20.
    Safavian, S.R., Landgrebe, D.: A survey of decision tree classifier methodology. IEEE Trans. Syst. Man Cybern. 21(3), 660–674 (1991)MathSciNetCrossRefGoogle Scholar
  21. 21.
    Shi, L.Z.: Exclamatory usages of question devices in contemporary chinese. Chin. Linguist. 4, 14–26 (2006)Google Scholar
  22. 22.
    Ultan, R.: Some general characteristics of interrogative systems. Univ. Hum. Lang. 4, 211–248 (1978)Google Scholar
  23. 23.
    Utgoff, P.E.: Incremental induction of decision trees. Mach. Learn. 4(2), 161–186 (1989)CrossRefGoogle Scholar
  24. 24.
    Wu, X., Kumar, V., Quinlan, J.R., Ghosh, J., Yang, Q., Motoda, H., McLachlan, G.J., Ng, A., Liu, B., Philip, S.Y., et al.: Top 10 algorithms in data mining. Knowl. Inf. Syst. 14(1), 1–37 (2008)CrossRefGoogle Scholar
  25. 25.
    Xu, L., Lin, H., Yang, Z.: Text orientation identification based on semantic comprehension. J. Chin. Inf. Process. 1, 015 (2007)Google Scholar
  26. 26.
    Yao, T.F., Lou, D.C.: Research on semantic orientation analysis for topics in chinese sentences. J. Chin. Inf. Process. 21(5), 73–79 (2007)Google Scholar
  27. 27.
    Zhu, Y.J.: Research on chinese language negation. Master’s thesis, Tianjin Normal University (2012)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Zhenhong Chen
    • 1
  • Kai Yang
    • 1
  • Yi Cai
    • 1
    Email author
  • Dongping Huang
    • 1
  • Ho-fung Leung
    • 2
  1. 1.School of Software EngineeringSouth China University of TechnologyGuangzhouChina
  2. 2.Department of Computer Science and EngineeringThe Chinese University of Hong KongHong KongChina

Personalised recommendations