Abstract
The problem of overfitting arises frequently in text mining due to high dimensional feature spaces, making the task of the learning algorithms difficult. Moreover, in such spaces visualization is not feasible. We focus on supervised text classification by presenting an approach that uses prior information about training labels, manifold learning and Support Vector Machines (SVM). Manifold learning is herein used as a pre-processing step, which performs nonlinear dimension reduction in order to tackle the curse of dimensionality that occurs. We use Isomap (Isometric Mapping) which allows text to be embedded in a low dimensional space, while enhancing the geometric characteristics of data by preserving the geodesic distance within the manifold. Finally, kernel-based machines can be used with benefits for final text classification in this reduced space. Results on a real-world benchmark corpus from Reuters demonstrate the visualization capabilities of the method in the severely reduced space. Furthermore we show the method yields performances comparable to those obtained with single kernel-based machines.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sebastiani, F.: Classification of Text, Automatic. In: Brown, K. (ed.) The Encyclopedia of Language and Linguistics, 2nd edn., vol. 14. Elsevier, Amsterdam (2006)
Eyheramendy, S., Genkin, A., Ju, W., Lewis, D., Madigan, D.: Sparse Bayesian Classifiers for Text Classification. Journal of Intelligence Community R&D (2003)
Tenenbaum, J., Silva, V., Langford, J.: A global geometric framework for nonlinear dimensionality reduction. Science 290(5500), 2319–2323 (2000)
Urtasun, R., Darrell, T.: Discriminative Gaussian Process Latent Variable Models for Classification. In: International Conference on Machine Learning (2007)
Kim, H., Park, H., Zha, H.: Distance Preserving Dimension Reduction for Manifold Learning. In: International Conference on Data Mining, pp. 1147–1151 (2007)
Navarro, D.J., Lee, M.D.: Spatial Visualization of Document Similarity. In: Defence Human Factors. Special Interest Group Meeting (2001)
Zhang, D., Chen, X., Lee, W.S.: Text Classification with Kernels on the Multinomial Manifold. In: SIGIR 2005, pp. 266–273 (2005)
Vapnik, V.: The Nature of Statistical Learning Theory, 2nd edn. Springer, Heidelberg (1999)
Tipping, M.: Sparse Bayesian Learning and the Relevance Vector Machine. Journal of Machine Learning Research I, 211–214 (2001)
Jolliffe, I.T.: Principal Component Analysis. Springer, New York (1986)
Comon, P.: Independent Component Analysis: a New Concept? Signal Processing 36(3), 287–314 (1994)
Cox, T., Cox, M.: Multidimensional Scaling. Chapman & Hall, London (1994)
Duraiswami, R., Raykar, V.: The Manifolds of Spatial Hearing. In: International Conference on Acoustics, Speech and Signal Processing, pp. 285–288 (2005)
Elgammal, A., Lee, C.: Inferring 3D body pose from silhouettes using activity manifold learning. IEEE Computer Vision and Pattern Recognition 2(27), 681–688 (2004)
Geng, X., Zhan, D.-C., Zhou, Z.-H.: Supervised Nonlinear Dimensionality Reduction for Visualization and Classification. IEEE Transactions on Systems, Man, and Cybernetics 35(6), 1098–1107 (2005)
Specht, D.: A General Regression Neural Network. IEEE Transactions on Neural Networks 2(6), 568–576 (1991)
van Rijsbergen, C.: Information Retrieval, 2nd edn. Butterworths, London (1979)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Silva, C., Ribeiro, B. (2008). Text Classification on Embedded Manifolds. In: Geffner, H., Prada, R., Machado Alexandre, I., David, N. (eds) Advances in Artificial Intelligence – IBERAMIA 2008. IBERAMIA 2008. Lecture Notes in Computer Science(), vol 5290. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88309-8_28
Download citation
DOI: https://doi.org/10.1007/978-3-540-88309-8_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88308-1
Online ISBN: 978-3-540-88309-8
eBook Packages: Computer ScienceComputer Science (R0)