Abstract
We introduce a novel framework, called sparse embedding (SE), for simultaneous dimensionality reduction and dictionary learning. We formulate an optimization problem for learning a transformation from the original signal domain to a lower-dimensional one in a way that preserves the sparse structure of data. We propose an efficient optimization algorithm and present its non-linear extension based on the kernel methods. One of the key features of our method is that it is computationally efficient as the learning is done in the lower-dimensional space and it discards the irrelevant part of the signal that derails the dictionary learning process. Various experiments show that our method is able to capture the meaningful structure of data and can perform significantly better than many competitive algorithms on signal recovery and object classification tasks.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Stone, C.J.: Optimal global rates of convergence for nonparametric regression. The Annals of Statistics 10, 1040–1053 (1982)
Beyer, K., Goldstein, J., Ramakrishnan, R., Shaft, U.: When Is “Nearest Neighbor” Meaningful? In: Beeri, C., Bruneman, P. (eds.) ICDT 1999. LNCS, vol. 1540, pp. 217–235. Springer, Heidelberg (1998)
Lee, J.A., Verleysen, M.: Nonlinear dimensionality reduction. Information Science and Statistics. Springer (2006)
Elad, M.: Sparse and redundant representations. In: From Theory to Applications in Signal and Image Processing. Springer, New York (2010)
Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Processing 15(12), 3736–3745 (2006)
Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. PAMI 31(2), 210–227 (2009)
Ramírez, I., Sprechmann, P., Sapiro, G.: Classification and clustering via dictionary learning with structured incoherence and shared features. In: CVPR, pp. 3501–3508. IEEE (2010)
Gkioulekas, I., Zickler, T.: Dimensionality reduction using the sparse linear model. In: Advances in Neural Information Processing Systems, NIPS (2011)
Zhang, L., Yang, M., Feng, Z., Zhang, D.: On the dimensionality reduction for sparse representation based face recognition. In: ICPR, pp. 1237–1240. IEEE (2010)
Qi, H., Hughes, S.: Using the kernel trick in compressive sensing: Accurate signal recovery from fewer measurements. In: IEEE ICASSP, pp. 3940–3943 (May 2011)
Nguyen, H.V., Patel, V.M., Nasrabadi, N.M., Chellappa, R.: Kernel dictionary learning. In: IEEE Int. Conference on Acoustics, Speech and Signal Processing (2012)
Mairal, J., Bach, F., Ponce, J.: Task-driven dictionary learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 34(4), 791–804 (2012)
Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In: IEEE CVPR, vol. 2, pp. 2169–2178 (2006)
Tuzel, O., Porikli, F., Meer, P.: Region Covariance: A Fast Descriptor for Detection and Classification. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006, Part II. LNCS, vol. 3952, pp. 589–600. Springer, Heidelberg (2006)
Aharon, M., Elad, M., Bruckstein, A.M.: The K-SVD: an algorithm for designing of overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)
Engan, K., Aase, S.O., Husoy, J.H.: Multi-frame compression: Theory and design. Signal Processing 80(10), 2121–2140 (2000)
Pati, Y.C., Rezaiifar, R., Krishnaprasad, P.S.: Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In: 27th Asilomar Conference on Signals, Systems and Computers, pp. 40–44 (1993)
Zhang, H., Berg, A., Maire, M., Malik, J.: SVM-KNN: Discriminative nearest neighbor classification for visual category recognition. In: IEEE CVPR (2006)
Griffin, G., Holub, A., Perona, P.: Caltech-256 object category dataset. Technical Report (2007)
Boiman, O., Shechtman, E., Irani, M.: In defense of nearest-neighbor based image classification. In: IEEE CVPR, pp. 1–8 (June 2008)
Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: IEEE CVPR, pp. 1794–1801 (June 2009)
Wang, J., Yang, J., Yu, K., Lv, F., Huang, T., Gong, Y.: Locality-constrained linear coding for image classification. In: IEEE CVPR, pp. 3360–3367 (June 2010)
Zhang, Q., Li, B.: Discriminative K-SVD for dictionary learning in face recognition. In: IEEE CVPR, pp. 2691–2698 (June 2010)
Jiang, Z., Lin, Z., Davis, L.: Learning a discriminative dictionary for sparse coding via label consistent K-SVD. In: CVPR, pp. 1697–1704 (June 2011)
van Gemert, J.C., Geusebroek, J.-M., Veenman, C.J., Smeulders, A.W.M.: Kernel Codebooks for Scene Categorization. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part III. LNCS, vol. 5304, pp. 696–709. Springer, Heidelberg (2008)
Kulkarni, N., Li, B.: Discriminative affine sparse codes for image classification. In: IEEE CVPR, pp. 1609–1616 (June 2011)
Perona, P., Fergus, R., Li, F.F.: Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In: Workshop on Generative Model Based Vision, p. 178 (2004)
Gehler, P., Nowozin, S.: On feature combination for multiclass object classification. In: 2009 IEEE 12th International Conference on Computer Vision, September 29-October 2, pp. 221–228 (2009)
Jain, P., Kulis, B., Grauman, K.: Fast image search for learned metrics. In: IEEE CVPR 2008, pp. 1–8 (June 2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nguyen, H.V., Patel, V.M., Nasrabadi, N.M., Chellappa, R. (2012). Sparse Embedding: A Framework for Sparsity Promoting Dimensionality Reduction. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7577. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33783-3_30
Download citation
DOI: https://doi.org/10.1007/978-3-642-33783-3_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33782-6
Online ISBN: 978-3-642-33783-3
eBook Packages: Computer ScienceComputer Science (R0)