Improved Document Feature Selection with Categorical Parameter for Text Classification

  • Fen Wang
  • Xiaoxuan LiEmail author
  • Xiaotao Huang
  • Ling Kang
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10026)


Social network develops rapidly and thousands of new data appears on the Internet every day. Classification technology is the key to organize big data. Feature Selection (FS) is a direct way to improve classification efficiency. FS can reduce the size of the feature subset and ensure classification accuracy based on features’ score, which is calculated by FS methods. Most previous studies of FS emphasized on precision while time-efficiency was commonly ignored. In our study, we proposed a method named CDFDC at first. It combines both CDF and Category-Frequency. Secondly, we compared DF, CDF, CHI, IG, CDFP_VM and CDFDC to figure out the relationships among algorithm complexity, time efficiency and classification accuracy. The experiment is implemented with 20-news-group data set and NB classifier. The performance of the FS methods evaluated by seven aspects: precision, Micro F1, Macro F1, feature-selection-time, documents-conversion-time, training-time and classification-time. The result shows that the proposed method performs well on efficiency and accuracy when the size of feature subset is greater than 3,000. And it is also discovered that FS algorithm’s complexity is unrelated to accuracy but complexity can ensure time stability and predictability.


Feature selection Measurement Comparison Time efficiency Experimentation 



I feel much indebted to many people who have instructed me in writing this paper. I would like to express my heartfelt gratitude to my tutor, Prof. Wang, for her warm-heart encouragement and most valuable advice, especially for her insightful comments and suggestions on the draft of this paper. Without her help, encouragement and guidance, I could not have completed this paper.

And I would like to express my thanks to my family and my friends for their valuable encouragement and spiritual support during my study.


  1. 1.
    Basu, T., Murthy, C.A.: Effective text classification by a supervised feature selection approach. In: 12th IEEE International Conference Data Mining Workshops (ICDMW), pp. 918–925. IEEE Press, New York (2012)Google Scholar
  2. 2.
    Li, Q., He, L., Lin, X.: Improved categorical distribution difference feature selection for Chinese document categorization. In: 8th International Conference on Ubiquitous Information Management and Communication, pp. 102:1–102:7. IEEE Press, New York (2014)Google Scholar
  3. 3.
    Sharma, A., Dey, S.: A comparative study of feature selection and machine learning techniques for sentiment analysis. In: ACM Research in Applied Computation Symposium, pp. 1–7. IEEE Press, New York (2012)Google Scholar
  4. 4.
    Wang, Z., Chen, S., Liu, J., Zhang, D.: Pattern representation in feature extraction and classifier design: matrix versus vector. J. IEEE Trans. Neural Netw. 19, 758–769 (2008)CrossRefGoogle Scholar
  5. 5.
    Tariq, A., Karim, A.: Fast supervised feature extraction by term discrimination information pooling. In: 20th ACM International Conference on Information and Knowledge Management, pp. 2233–2236. IEEE Press, New York (2011)Google Scholar
  6. 6.
    Van, M., Kang, H.-J.: Bearing-fault diagnosis using non-local means algorithm and empirical mode decomposition-based feature extraction and two-stage feature selection. J. IET Sci. Measur. Technol. 9, 671–680 (2015)CrossRefGoogle Scholar
  7. 7.
    Somol, P., Novovicova, J.: Evaluating stability and comparing output of feature selectors that optimize feature subset cardinality. J. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1921–1939 (2010)CrossRefGoogle Scholar
  8. 8.
    Meng, J., Lin, H.: A two-stage feature selection method for text categorization. In: Seventh International Conference Fuzzy Systems and Knowledge Discovery (FSKD), pp. 1492–1496. IEEE Press, New York (2010)Google Scholar
  9. 9.
    Zhang, W., Yoshida, T., Tang, X.: A comparative study of TF*IDF, LSI and multi-words for text classification. J. Exp. Syst. Appl. 38, 1492–1496 (2011)Google Scholar
  10. 10.
    Kadhim, A.I., Cheah, Y.N., Ahamed, N.H., Salman, L.A.: Feature extraction for co-occurrence-based cosine similarity score of text documents. In: IEEE Student Conference Research and Development (SCOReD), pp. 1–4. IEEE Press, New York (2014)Google Scholar
  11. 11.
    Li, Y., Algarni, A., Albathan, M., Shen, Y., Bijaksana, M.A.: Relevance feature discovery for text mining. In: IEEE Transactions on Knowledge and Data Engineering, pp. 1656–1669. IEEE Press, New York (2015)Google Scholar
  12. 12.
    Li, Y., Algarni, A., Albathan, M., Shen, Y., Bijaksana, M.A.: Relevance feature discovery for text mining. J. IEEE Trans. Knowl. Data Eng. 27, 1656–1669 (2015)CrossRefGoogle Scholar
  13. 13.
    Song, S.J., Heo, G.E., Kim, H.J., Jung, H.J., Kim, Y.H., Song, M.: Grounded feature selection for biomedical relation extraction by the combinative approach. In: ACM 8th International Workshop on Data and Text Mining in Bioinformatics, pp. 29–32. IEEE Press, New York (2014)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Fen Wang
    • 1
  • Xiaoxuan Li
    • 1
    Email author
  • Xiaotao Huang
    • 1
  • Ling Kang
    • 2
  1. 1.Department of Computer Science and TechnologyHuazhong University of Science and TechnologyHubeiChina
  2. 2.Department of Hydropower and Information EngineeringHuazhong University of Science and TechnologyHubeiChina

Personalised recommendations