Optimal Landmark Selection for Nyström Approximation

Abstract

The Nyström method is an efficient technique for large-scale kernel learning. It provides a low-rank matrix approximation to the full kernel matrix. The quality of Nyström approximation largely depends on the choice of landmark points. While standard method uniformly samples columns of the kernel matrix, improved sampling techniques have been proposed based on ensemble learning [1] and clustering [2]. These methods are focused on minimizing the approximation error for the original kernel. In this paper, we take a different perspective by minimizing the approximation error for the input vectors instead. We show under some restrictive condition that the new formulation is equivalent to the standard Nyström solution. This leads to a novel approach for optimizing landmark points for the Nyström approximation. Experimental results demonstrate the superior performance of the proposed landmark optimization method compared to existing Nyström methods in terms of lower approximation errors obtained.