Fast Algorithms for DR Approximation

Abstract

In nonlinear dimensionality reduction, the kernel dimension is the square of the vector number in the data set. In many applications, the number of data vectors is very large. The spectral decomposition of a large dimensioanl kernel encounters difficulties in at least three aspects: large memory usage, high computational complexity, and computational instability. Although the kernels in some nonlinear DR methods are sparse matrices, which enable us to overcome the difficulties in memory usage and computational complexity partially, yet it is not clear if the instability issue can be settled. In this chapter, we study some fast algorithms that avoid the spectral decomposition of large dimensional kernels in DR processing, dramatically reducing memory usage and computational complexity, as well as increasing numerical stability. In Section 15.1, we introduce the concepts of rank revealings. In Section 15.2, we present the randomized low rank approximation algorithms. In Section 15.3, greedy lank-revealing algorithms (GAT) and randomized anisotropic transformation algorithms (RAT), which approximate leading eigenvalues and eigenvectors of DR kernels, are introduced. Numerical experiments are shown in Section 15.4 to illustrate the validity of these algorithms. The justification of RAT algorithms is included in Section 15.5.