Comparison of Support Vector Machines With and Without Latent Semantic Analysis for Document Classification
Document Classification is a key technique in Information Retrieval. Various techniques have been developed for document classification. Every technique aims for higher accuracy and greater speed. Its performance depends on various parameters like algorithms, size, and type of dataset used. Support Vector Machine (SVM) is a prominent technique used for classifying large datasets. This paper attempts to study the effect of Latent Semantic Analysis (LSA) on SVM. LSA is used for dimensionality reduction. The performance of SVM is studied on reduced dataset generated by LSA.
KeywordsDocument classification Support vector machine Latent semantic analysis Dimensionality reduction Singular value decomposition Context vector
We acknowledge the help extended by Mr. Shubham Gatkal, Mr. Sandesh Gupta and Mr. Prathamesh Ingle for experimentation. We would also like to acknowledge the support and encouragements received from the authorities of College of Engineering, Pune. (COEP).
- 1.Camastra, F., Ciaramella, A., Placitelli, A., & Staiano, A. (2015). Machine learning-based web documents categorization by semantic graphs. ResearchGate Publisher. https://doi.org/10.1007/978-3-319-18164-6_8.
- 2.Xu, J., & Croft, W. B. (2017). Query expansion using local and global document analysis. In ACM SIGIR Forum (Vol. 51, No. 2), July 2017.Google Scholar
- 3.Khatavkar, V., & Kulkarni, P. (2016, December). Context vector machine for information retrieval. International Conference on Communication and Signal Processing.Google Scholar
- 4.Khatavkar, V., & Kulkarni, P. (2017). Document context identification using latent semantic analysis. Presented in 3rd International Conference on Computing, Communication, Control and Automation, August 17–18, 2017, Pune, MS, India. (To be published on IEEE).Google Scholar
- 5.Sheikh, I., Illina, I., Fohr, D., & Linar, G. (2016). Document level semantic context for retrieving OOV proper names. ICASSP.Google Scholar
- 6.Wissner-Gross, A. (2016). Datasets over algorithms, Edge.com. Retrieved January 8, 2016.Google Scholar
- 7.The BBC dataset available at: http://mlg.ucd.ie/datasets/bbc.html.
- 8.The 20 Newsgroup dataset available at http://qwone.com/~jason/20Newsgroups/.
- 9.The article about Tokenization available at https://nlp.stanford.edu/IR-book/html/htmledition/tokenization-1.html.
- 10.Manning, C. D., Raghavan, P., & Schutze, H. (2008). Scoring, term weighting, and the vector space model. In Introduction to information retrieval (PDF) (p. 100). https://doi.org/10.1017/cbo9780511809071.007. ISBN 978-0-511-80907-1.
- 12.Onal Suzek, T. (2017). Using latent semantic analysis for automated keyword extraction from large document corpora. Turkish Journal of Electrical Engineering & Computer Sciences.Google Scholar
- 13.Rajkumar, K., & Karthik, K. (2017). Contextual plagiarism detection using latent semantic analysis. International Research Journal of Advanced Engineering and Science, 2(1), 214–217.Google Scholar
- 14.Marcolin, C. B., & Luiz Becker, J. (2016). Exploring latent semantic analysis in a big data (base). In Twenty-second Americas Conference on Information Systems, San Diego.Google Scholar
- 15.Hofmann, T. (2017). Probabilistic latent semantic indexing. In ACM SIGIR Forum (Vol. 51, No. 2), July 2017.Google Scholar
- 16.Zhu, Z. (2017). Latent semantic analysis, July 18, 2017. Available at http://www.professorbray.net/Teaching/89s-MOU/2017-summerTerm2/Papers/third\%20submissions/ZHUfiPaper2fiLSA.pdf10.
- 17.Dumais, S. T. (2005). Latent semantic analysis. Annual Review of Information Science and Technology.Google Scholar
- 18.DeAngelis, G. C., Ohzawa, I., & Freeman, R. D. (1995). Receptive-field dynamics in the central visual pathways. Trends in Neuroscience, 18(10), 451–458. https://doi.org/10.1016/0166-2236(95)94496-r. PMID 8545912.
- 19.Information about LSA and SVD is available at http://web.mit.edu/be.400/www/SVD/SingularfiValuefiDecomposition.htm.
- 20.Fatima, S., & Srinivasu, B. Dr. (2017, February). Text document categorization using support vector machine. International Research Journal of Engineering and Technology (IRJET).Google Scholar
- 22.Joachims, T. (1991). Text categorization with support vector machines: Learning with many relevant features. University at Dortmund Informatik LS8, Baroper Str. 30144221 Dortmund, Germany.Google Scholar
- 23.Abdiansah, A., & Wardoyo, R. (2015). Time complexity analysis of support vector machines (SVM) in LibSVM. International Journal of Computer Applications.Google Scholar
- 24.The python implementation of LSA was taken from http://mccormickml.com/2016/03/25/lsa-for-text-classification-tutorial/.
- 25.The python implementation for SVM was taken from https://pythonprogramming.net/linear-svc-example-scikit-learn-svm-python/.
- 26.The python implementation of LSA was taken from http://blog.josephwilk.net/projects/latent-semantic-analysis-in-python.html.
- 27.The python implementation of LSA was taken from https://gist.github.com/vgoklani/1267632.
- 28.The python implementation of LSA was taken from https://technowiki.wordpress.com/2011/08/27/latent-semantic-analysis-lsa-tutorial/.