Comparison of Support Vector Machines With and Without Latent Semantic Analysis for Document Classification

  • Vaibhav KhatavkarEmail author
  • Parag Kulkarni
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 808)


Document Classification is a key technique in Information Retrieval. Various techniques have been developed for document classification. Every technique aims for higher accuracy and greater speed. Its performance depends on various parameters like algorithms, size, and type of dataset used. Support Vector Machine (SVM) is a prominent technique used for classifying large datasets. This paper attempts to study the effect of Latent Semantic Analysis (LSA) on SVM. LSA is used for dimensionality reduction. The performance of SVM is studied on reduced dataset generated by LSA.


Document classification Support vector machine Latent semantic analysis Dimensionality reduction Singular value decomposition Context vector 



We acknowledge the help extended by Mr. Shubham Gatkal, Mr. Sandesh Gupta and Mr. Prathamesh Ingle for experimentation. We would also like to acknowledge the support and encouragements received from the authorities of College  of Engineering, Pune. (COEP).


  1. 1.
    Camastra, F., Ciaramella, A., Placitelli, A., & Staiano, A. (2015). Machine learning-based web documents categorization by semantic graphs. ResearchGate Publisher.
  2. 2.
    Xu, J., & Croft, W. B. (2017). Query expansion using local and global document analysis. In ACM SIGIR Forum (Vol. 51, No. 2), July 2017.Google Scholar
  3. 3.
    Khatavkar, V., & Kulkarni, P. (2016, December). Context vector machine for information retrieval. International Conference on Communication and Signal Processing.Google Scholar
  4. 4.
    Khatavkar, V., & Kulkarni, P. (2017). Document context identification using latent semantic analysis. Presented in 3rd International Conference on Computing, Communication, Control and Automation, August 17–18, 2017, Pune, MS, India. (To be published on IEEE).Google Scholar
  5. 5.
    Sheikh, I., Illina, I., Fohr, D., & Linar, G. (2016). Document level semantic context for retrieving OOV proper names. ICASSP.Google Scholar
  6. 6.
    Wissner-Gross, A. (2016). Datasets over algorithms, Retrieved January 8, 2016.Google Scholar
  7. 7.
    The BBC dataset available at:
  8. 8.
    The 20 Newsgroup dataset available at
  9. 9.
  10. 10.
    Manning, C. D., Raghavan, P., & Schutze, H. (2008). Scoring, term weighting, and the vector space model. In Introduction to information retrieval (PDF) (p. 100). ISBN 978-0-511-80907-1.
  11. 11.
    Robertson, S. (2004). Understanding inverse document frequency: On theoretical arguments for IDF. Journal of Documentation, 60(5), 503–520. Scholar
  12. 12.
    Onal Suzek, T. (2017). Using latent semantic analysis for automated keyword extraction from large document corpora. Turkish Journal of Electrical Engineering & Computer Sciences.Google Scholar
  13. 13.
    Rajkumar, K., & Karthik, K. (2017). Contextual plagiarism detection using latent semantic analysis. International Research Journal of Advanced Engineering and Science, 2(1), 214–217.Google Scholar
  14. 14.
    Marcolin, C. B., & Luiz Becker, J. (2016). Exploring latent semantic analysis in a big data (base). In Twenty-second Americas Conference on Information Systems, San Diego.Google Scholar
  15. 15.
    Hofmann, T. (2017). Probabilistic latent semantic indexing. In ACM SIGIR Forum (Vol. 51, No. 2), July 2017.Google Scholar
  16. 16.
  17. 17.
    Dumais, S. T. (2005). Latent semantic analysis. Annual Review of Information Science and Technology.Google Scholar
  18. 18.
    DeAngelis, G. C., Ohzawa, I., & Freeman, R. D. (1995). Receptive-field dynamics in the central visual pathways. Trends in Neuroscience, 18(10), 451–458. PMID 8545912.
  19. 19.
    Information about LSA and SVD is available at
  20. 20.
    Fatima, S., & Srinivasu, B. Dr. (2017, February). Text document categorization using support vector machine. International Research Journal of Engineering and Technology (IRJET).Google Scholar
  21. 21.
    Cortes, C., & Vapnik, V. (2003). Support-vector networks. Machine Learning, 20(3), 273–297.zbMATHGoogle Scholar
  22. 22.
    Joachims, T. (1991). Text categorization with support vector machines: Learning with many relevant features. University at Dortmund Informatik LS8, Baroper Str. 30144221 Dortmund, Germany.Google Scholar
  23. 23.
    Abdiansah, A., & Wardoyo, R. (2015). Time complexity analysis of support vector machines (SVM) in LibSVM. International Journal of Computer Applications.Google Scholar
  24. 24.
    The python implementation of LSA was taken from
  25. 25.
    The python implementation for SVM was taken from
  26. 26.
  27. 27.
    The python implementation of LSA was taken from
  28. 28.

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Department of Computer Engineering and Information TechnologyCollege of Engineering, PuneShivajinagar, PuneIndia

Personalised recommendations