Advertisement

Chinese Text Classification Based on Character-Level CNN and SVM

  • Huaiguang Wu
  • Daiyi LiEmail author
  • Ming Cheng
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 986)

Abstract

With the rapid development of the Internet, the high dimensional text data has increased rapidly. How to build an efficient and extensible text classification algorithm has become a hot topic in the field of data mining. Aiming at the problems of high feature dimension, sparse data and long computation time in traditional SVM classification algorithm based on TF-IDF (Term Frequency-Inverse Document Frequency), we propose a novel hybrid system for Chinese text classification: CSVM, which is independent of the hand-designed features and domain knowledge. Firstly, the encoding words are done by constructing a text vocabulary of size m for the input language, and then quantize each word using 1-of-m encoding. Secondly, we exploit the CNN (Convolutional Neural Network) to extract the morphological features of character vectors from each word, and then through large scale text material training the semantic feature of each word vectors are be obtained the semantic feature of each word vectors. Finally, the text classification is carried out with the SVM multiple classifier. Testing on a text dataset with 10 categories, the experimental results show that the CSVM algorithm is more effective than other traditional Chinese text classification algorithm.

Keywords

TF-IDF SVM Character-level CNN Text vectorization Text classification 

Notes

Acknowledgements

This research is financially supported by National Natural Science Foundation of China (Grant No. 61672470) and the National Key Research and Development Plant (Grant No. 2016YFE0100300 and 2016YFE0100600). It is also partially supported by National Natural Science Foundation of China (Grant No. 61802350), the project of the International Cooperation of Henan Province of China (Grant No. 162102410076), the Technology Tackling Key Project of Henan (Grant No. 162102310578) and the Key Scientific Research Projects of Universities in Henan (Grant No. 17A520064).

References

  1. 1.
    Li, J., Cao, Y., Wang, Y., et al.: Online learning algorithms for double-weighted least squares twin bounded support vector machines. Neural Process. Lett. 45(1), 1–21 (2016)Google Scholar
  2. 2.
    Vapnik, V.N.: Statistical Learning Theory. Wiley, Hoboken (1998)zbMATHGoogle Scholar
  3. 3.
    Wang, S., Manning, C.D.: Baselines and bigrams: simple, good sentiment and topic classification. In: Meeting of the Association for Computational Linguistics: Short Papers, pp. 90–94. Association for Computational Linguistics (2012)Google Scholar
  4. 4.
    Silva, J., Coheur, L., Mendes, A.C., et al.: From symbolic to sub-symbolic information in question classification. Artif. Intell. Rev. 35(2), 137–154 (2011)CrossRefGoogle Scholar
  5. 5.
    Ji, Y.L., Dernoncourt, F.: Sequential short-text classification with recurrent and convolutional neural networks, pp. 515–520 (2016)Google Scholar
  6. 6.
    Chatterjee, R., Negri, M., Turchi, M., et al.: Guiding neural machine translation decoding with external knowledge. In: Conference on Machine Translation, pp. 157–168 (2017)Google Scholar
  7. 7.
    Zhang, X., Zhao, J., Lecun, Y.: Character-level convolutional networks for text classification, pp. 649–657 (2015)Google Scholar
  8. 8.
    Li, J., Sun, M.: Scalable term selection for text categorization. In: Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), Prague, Czech Republic, pp. 774–782 (2015)Google Scholar
  9. 9.
    Kowalski, M., Naruniec, J., Trzcinski, T.: Deep alignment network: a convolutional neural network for robust face alignment, pp. 2034–2043 (2017)Google Scholar
  10. 10.
    Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. Comput. Sci. (2014)Google Scholar
  11. 11.
    He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition, pp. 770–778 (2015)Google Scholar
  12. 12.
    Kim, Y.: Convolutional neural networks for sentence classification. Eprint Arxiv (2014)Google Scholar
  13. 13.
    Tomar, D., Agarwal, S.: A comparison on multi-class classification methods based on least squares twin support vector machine. Knowl.-Based Syst. 81(C), 131–147 (2015)CrossRefGoogle Scholar
  14. 14.
    Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. Comput. Sci. 28(12) (2014)Google Scholar
  15. 15.
    Dong, Y., Wu, Y.: Adaptive cascade deep convolutional neural networks for face alignment. Comput. Stand. Interfaces 42(3), 105–112 (2015)CrossRefGoogle Scholar
  16. 16.
    Srivastava, N., Hinton, G., Krizhevsky, A., et al.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)MathSciNetzbMATHGoogle Scholar
  17. 17.
    Carrera-Trejo, V., Sidorov, G., Miranda-Jiménez, S., et al.: Latent Dirichlet Allocation complement in the vector space model for Multi-Label Text Classification. Cancer Biol. Ther. 7(7), 1095–1097 (2015)Google Scholar
  18. 18.
    Chanda, S., Pal, S., Pal, U.: Word-wise Sinhala Tamil and English script identification using Gaussian kernel SVM. In: International Conference on Pattern Recognition, pp. 1–4. IEEE (2008)Google Scholar
  19. 19.
    Gibson, E., Li, W., Sudre, C., et al.: NiftyNet: a deep-learning platform for medical imaging. Comput. Methods Programs Biomed. 158, 113 (2018)CrossRefGoogle Scholar
  20. 20.
    Pedregosa, F., Gramfort, A., Michel, V., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12(10), 2825–2830 (2012)MathSciNetzbMATHGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.School of Computer and Communication EngineeringZhengzhou University of Light IndustryZhengzhouChina
  2. 2.The First Affiliated Hospital of Zhengzhou UniversityZhengzhouChina

Personalised recommendations