Deep Neural Networks for Out-of-sample Classification of Nonlinear Manifolds

Abstract

In information processing domains, high dimensional data poses several challenges in terms of storage, visualization, and retrieval. There are several instances where, even though the data points are of high dimension, the data actually resides on a lower dimensional space. Dimensionality reduction methods attempt to find meaningful representation of data which is present in the lower dimension leading to better visualization, removal of noisy features and redundant information. Traditional linear dimensionality reduction techniques are incapable of dealing with nonlinear datasets. Nonlinear dimensionality reduction methods like Isomap work well for such datasets, but suffer from the issue of out-of-sample extension. In this paper, a solution for out-of-sample problem of Isomap method and extended Isomap method is put forward by employing neural networks. Out-of-sample extensions of Isomap and extended Isomap using deep neural network (DNN) are proposed. The proposed method is tested using AT&T face database, Yale face database, and MNIST handwritten digit database. The proposed technique is compared with the existing out-of-sample extension method using general regression neural network (GRNN).