Advertisement

WordNet-Based Text Categorization Using Convolutional Neural Networks

  • K. PremchanderEmail author
  • S. S. V. N. Sarma
  • K. Vaishali
  • P. Vijaypal Reddy
  • M. Anjaneyulu
  • S. Nagaprasad
Conference paper
Part of the Lecture Notes in Networks and Systems book series (LNNS, volume 34)

Abstract

Text Categorization is a task of assigning documents to a fixed number of predefined categories. Concept is the grouping of semantically related items under a unique name. Dimensionality space and sparsity of the document representation can be reduced using concept generation. Conceptual representation of a text can be generated using WordNet. In this paper, an empirical evolution using Convolutional Neural Networks (CNN) for text categorization has been performed. The Convolutional Neural Networks exploit the one-dimensional structures of the text such as words, concepts, word embeddings, and concept embeddings to improve the categorical label prediction. The Reuter’s dataset is evaluated with Convolutional Neural Networks on four categories of data. The representation of a text with word embeddings and concept embeddings together results to a better classification performance using CNN compared with word embeddings and concept embeddings individually.

Keywords

Text categorization Convolutional Neural Networks Word embeddings Concept embeddings WordNet 

References

  1. 1.
    Thorsten: TC with SVM and Learn relevant features, ECML (1998)Google Scholar
  2. 2.
    Yang, E.T.: Semi supervised RNN classification of text with word embedding. JMLR Res. 5, 361–397 (2004)Google Scholar
  3. 3.
    Dai, A.M., Le, Q.V.: Semi-supervised sequence learning. Adv. Neural Inf. Process. Syst. 3079–3087 (2015)Google Scholar
  4. 4.
    Zhang, X., Zhao, J., LeCun, Y.: Character-level convolutional networks for text classification. Adv. Neural Inf. Process. Syst. 649–657 (2015)Google Scholar
  5. 5.
    Johnson, R., Zhang, T.: Semi-supervised convolutional neural networks for text categorization via region embedding. Adv. Neural Inf. Process. Syst. 919–927 (2015)Google Scholar
  6. 6.
    Aggarwal, C.C., Zhai, C.: A survey of text classification algorithms. Mining text data, 163–222 (2012)Google Scholar
  7. 7.
    Dinu, G.: Predict a systematic compare of context counting using context predict semantic vector. ACL, 238–247 (2012)Google Scholar
  8. 8.
    Vincent, P.: ANN probabilistic model of a language. JMLR 3, 1137, 1155 (2003)Google Scholar
  9. 9.
    Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)CrossRefGoogle Scholar
  10. 10.
    Bottou, L.: Learning of gradient in networks using CNN. In: Proceedings on Neuro-Nımes, vol. 91 (1999)Google Scholar
  11. 11.
    Bloehdorn, S., Hotho, A.: Boosting for text classification with semantic features. In: WebKDD, pp. 149–166 (2004)Google Scholar
  12. 12.
    Johnson, M.: Maxent discriminative re-ranking and Coarse-to-fine n-best parsing. In: Association for Computational Linguistics, pp. 173–180 (2005)Google Scholar
  13. 13.
    Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.: Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12, 2493–2537 (2011)Google Scholar
  14. 14.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley (2012)Google Scholar
  15. 15.
    Glänzel, Wolfgang, Thijs, Bart: Using ‘core documents’ for detecting and labelling new emerging topics. Scientometrics 91(2), 399–416 (2012)CrossRefGoogle Scholar
  16. 16.
    Hinton, G.E, Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science, 313(5786), 504–507 (2006)MathSciNetCrossRefGoogle Scholar
  17. 17.
    Huang, E.H., Socher, R., Manning, C.D., Ng, A.Y.: Improving word representations via global context and multiple word prototypes. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers, vol. 1, pp. 873–882. Association for Computational Linguistics (2012)Google Scholar
  18. 18.
    Kalchbrenner, N., Blunsom, P.: Recurrent convolutional neural networks for discourse compositionality. arXiv preprint arXiv:1306.3584 (2013)
  19. 19.
    Klementiev, A., Titov, I., Bhattarai, B.: Inducing crosslingual distributed representations of words (2012)Google Scholar
  20. 20.
    Mikonos, T.: Distributed representations of sentences and docs. ICML (2014)Google Scholar
  21. 21.
    Sutskever, I.: Distributional representations of words and phrases and their composite. NIPS, 3111–3119 (2013)Google Scholar
  22. 22.
    Mikolov, T., Yih, W., Zweig, G.: Linguistic regularities in continuous space word representations. In hlt-Naacl 13, 746–751 (2013)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  • K. Premchander
    • 1
    Email author
  • S. S. V. N. Sarma
    • 2
  • K. Vaishali
    • 3
  • P. Vijaypal Reddy
    • 4
  • M. Anjaneyulu
    • 1
  • S. Nagaprasad
    • 5
  1. 1.Department of Computer ScienceDravidian UniversityKuppamIndia
  2. 2.Department of CSEVaagdevi College of EngineeringWarangalIndia
  3. 3.Department of CSEJyothismathi Institute of Technology and SciencesKarimnagarIndia
  4. 4.Department of CSEMatrusri Engineering CollegeHyderabadIndia
  5. 5.S.R.R. Government Arts & Science CollegeKarimnagarIndia

Personalised recommendations