Abstract
Feature selection is an important step. It aims to filter some irrelevant features, improve the classifier speed and also reduce the interference during text classification process. Information gain (IG) feature selection algorithm is one of the most effective feature selection algorithms. But it is easy to filter out the characteristic words which have a low IG score but have a strong ability of text type identification. Because IG algorithm only considers the number of documents of feature items in each category. Aiming at this defect, we propose an improved information gain algorithm by introducing three parameters: intra-class word frequency, inter-class separation degree and intra-class dispersion degree. Then, the improved IG algorithm is used for feature selection, and important feature words with high IG value are selected according to the threshold value. Final, the important feature words in the text are expressed as two-dimensional word vectors and input into Convolutional Neural Network (CNN) to train and classify them. Therefore, a text classification model based on improved information gain and convolutional neural network is proposed and abbreviated as “I-CNN”. Through experiments, we achieve good experimental results in THUCNews Chinese text classification corpus. Experimental results prove that the improved IG algorithm is better than the traditional feature selection algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Agnihotri, D., Verma, K., Tripathi, P.: Pattern and cluster mining on text data. In: 2014 Fourth International Conference on Communication Systems and Network Technologies, pp. 428–432. IEEE (2014)
Yang, Y., Pedersen, J.O.: A comparative study on feature selection in text categorization. In: ICML, vol. 97, pp. 35, 412–420 (1997)
Xu, J., Jiang, H.: An improved information gain feature selection algorithm for SVM text classifier. In: 2015 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, pp. 273–276. IEEE (2015)
Chen, Y., Chen, Y.: Attribute reduction algorithm based on information entropy and ant colony optimization. J. Chin. Comput. Syst. 36(3), 586–590 (2015)
Ming, H.: A text classification method based on maximum entropy model based on improved information gain feature selection. J. Southwest Normal Univ. (Nat. Sci. Ed.) 44(03), 119–124 (2019)
Kim, Y.: Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882 (2014)
Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)
Chang, L., Deng, X.M., Zhou, M.Q., et al.: Convolution neural network in image understanding. Acta Automatica Sinica 42(9), 1300–1312 (2016)
Sainath, T.N., Mohamed, A., Kingsbury, B., et al.: Deep convolutional neural networks for LVCSR. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8614–8618. IEEE (2013)
Lecun, Y., Bottou, L., Bengio, Y., et al.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Zeiler, M.D., Fergus, R.: Stochastic pooling for regularization of deep convolutional neural networks. arXiv preprint arXiv:1301.3557 (2013)
Mikolov, T., Sutskever, I., Chen, K., et al.: Distributed representations of words and phrases and their compositionality. In: Advances in Neural Information Processing Systems, pp. 3111–3119 (2013)
O’Shea, K., Nash, R.: An introduction to convolutional neural networks. arXiv preprint arXiv:1511.08458 (2015)
LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436 (2015)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
Hinton, G.E., Srivastava, N., Krizhevsky, A., et al.: Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Dong, M., Xu, H., Xu, Q. (2020). Text Classification Based on Improved Information Gain Algorithm and Convolutional Neural Network. In: Gao, H., Li, K., Yang, X., Yin, Y. (eds) Testbeds and Research Infrastructures for the Development of Networks and Communications. TridentCom 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 309. Springer, Cham. https://doi.org/10.1007/978-3-030-43215-7_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-43215-7_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-43214-0
Online ISBN: 978-3-030-43215-7
eBook Packages: Computer ScienceComputer Science (R0)