Support Vector Machines for Visualization and Dimensionality Reduction

Abstract

Discriminant functions calculated by Support Vector Machines (SVMs) define in a computationally efficient way projections of high-dimensional data on a direction perpendicular to the discriminating hyperplane. These projections may be used to estimate and display posterior probability densities . Additional directions for visualization and dimensionality reduction are created by repeating the linear discrimination process in a space orthogonal to already defined projections. This process allows for an efficient reduction of dimensionality and visualization of data, at the same time improving classification accuracy of a single discriminant function. Visualization of real and artificial data shows that transformed data may not be separable and thus linear discrimination will completely fail, but the nearest neighbor or rule-based methods in the reduced space may still provide simple and accurate solutions.