Generalized Dictionary Learning for Symmetric Positive Definite Matrices with Application to Nearest Neighbor Retrieval

  • Suvrit Sra
  • Anoop Cherian
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6913)


We introduce Generalized Dictionary Learning (GDL), a simple but practical framework for learning dictionaries over the manifold of positive definite matrices. We illustrate GDL by applying it to Nearest Neighbor (NN) retrieval, a task of fundamental importance in disciplines such as machine learning and computer vision. GDL distinguishes itself from traditional dictionary learning approaches by explicitly taking into account the manifold structure of the data. In particular, GDL allows performing “sparse coding” of positive definite matrices, which enables better NN retrieval. Experiments on several covariance matrix datasets show that GDL achieves performance rivaling state-of-the-art techniques.


Face Recognition Covariance Matrice Near Neighbor Online Algorithm Geodesic Distance 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Alexander, D., Pierpaoli, C., Basser, P., Gee, J.: Spatial transformations of diffusion tensor magnetic resonance images. IEEE Tran. Med. Imaging 20(11), 1131–1139 (2002)CrossRefGoogle Scholar
  2. 2.
    Arsigny, V., Fillard, P., Pennec, X., Ayache, N.: Log-Euclidean metrics for fast and simple calculus on diffusion tensors. Magnetic Resonance in Medicine 56(2), 411–421 (2006)CrossRefGoogle Scholar
  3. 3.
    Arya, S., Mount, D., Netanyahu, N., Silverman, R., Wu, A.: An optimal algorithm for approximate nearest neighbor searching fixed dimensions. Journal of the ACM (JACM) 45(6), 891–923 (1998)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Birgin, E., Martínez, J., Raydan, M.: Nonmonotone spectral projected gradient methods on convex sets. SIAM Journal on Optimization 10(4), 1196–1211 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Cai, J., Candes, E., Shen, Z.: A singular value thresholding algorithm for matrix completion. Arxiv preprint arXiv:0810.3286 (2008)Google Scholar
  6. 6.
    Candes, E., Plan, Y.: Matrix completion with noise. Proceedings of the IEEE 98(6), 925–936 (2010)CrossRefGoogle Scholar
  7. 7.
    Chaudhry, R., Ivanov, Y.: Fast Approximate Nearest Neighbor Methods for Non-Euclidean Manifolds with Applications to Human Activity Analysis in Videos. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 735–748. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  8. 8.
    Ciaccia, P., Patella, M., Zezula, P.: M-tree: An Efficient Access Method for Similarity Search in Metric Spaces. In: Proceedings of the 23rd VLDB Conference, Athens, Greece, pp. 426–435 (1997)Google Scholar
  9. 9.
    Dana, K., Van Ginneken, B., Nayar, S., Koenderink, J.: Reflectance and texture of real-world surfaces. ACM Transactions on Graphics (TOG) 18(1), 1–34 (1999)CrossRefGoogle Scholar
  10. 10.
    Elad, M., Aharon, M.: Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries. IEEE Tran. Image Processing 15(12), 3736–3745 (2006)CrossRefMathSciNetGoogle Scholar
  11. 11.
    Porikli, F., Tuzel, O.: Covariance tracker. In: CVPR (2006)Google Scholar
  12. 12.
    Forstner, W., Moonen, B.: A metric for covariance matrices. Qua vadis geodesia, pp. 113–128 (1999)Google Scholar
  13. 13.
    Gaivoronski, A.A.: Convergence properties of backpropagation for neural nets via theory of stochastic gradient methods. Part 1. Optimization Methods and Software 4(2), 117–134 (1994)CrossRefGoogle Scholar
  14. 14.
    Gionis, A., Indyk, P., Motwani, R.: Similarity search in high dimensions via hashing. In: Proceedings of the 25th International Conference on Very Large Data Bases, pp. 518–529 (1999)Google Scholar
  15. 15.
    Indyk, P.: On approximate nearest neighbors in non-euclidean spaces. In: Proceedings of the 39th Annual Symposium on Foundations of Computer Science, p. 148 (1998)Google Scholar
  16. 16.
    Indyk, P., Motwani, R.: Approximate nearest neighbors: towards removing the curse of dimensionality. In: Proceedings of the Thirtieth Annual ACM Symposium on Theory of Computing, pp. 604–613 (1998)Google Scholar
  17. 17.
    Kim, D., Sra, S., Dhillon, I.: A non-monotonic method for large-scale non-negative least squares. Preprint on: Optimization Online (2011)Google Scholar
  18. 18.
    Kleinberg, J.: Two algorithms for nearest-neighbor search in high dimensions. In: Proceedings of the Twenty-Ninth Annual ACM Symposium on Theory of Computing, p. 608 (1997)Google Scholar
  19. 19.
    Knuth, D.: The art of computer programming. Sorting and Searching, vol. 3. Addison-Wesley, Reading (1973)zbMATHGoogle Scholar
  20. 20.
    Kulis, B., Grauman, K.: Kernelized locality-sensitive hashing for scalable image search. In: ICCV (2009)Google Scholar
  21. 21.
    Kushilevitz, E., Ostrovsky, R., Rabani, Y.: Efficient search for approximate nearest neighbor in high dimensional spaces. In: Proceedings of the Thirtieth Annual ACM Symposium on Theory of Computing, p. 623 (1998)Google Scholar
  22. 22.
    Lang, S.: Fundamentals of differential geometry. Graduate Texts in Mathematics, vol. 191 (1999)Google Scholar
  23. 23.
    Lepore, N., Brun, C., Chou, Y., Chiang, M., Dutton, R., Hayashi, K., Luders, E., Lopez, O., Aizenstein, H., Toga, A., et al.: Generalized tensor-based morphometry of HIV/AIDS using multivariate statistics on deformation tensors. IEEE Tran. Med. Imaging 27(1), 129–141 (2007)CrossRefGoogle Scholar
  24. 24.
    Liu, C.: Gabor-based kernel PCA with fractional power polynomial models for face recognition. IEEE PAMI 26(5), 572–581 (2004)CrossRefGoogle Scholar
  25. 25.
    Liu, D., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Mathematical Programming 45(1), 503–528 (1989)CrossRefzbMATHMathSciNetGoogle Scholar
  26. 26.
    Liu, Z., Vandenberghe, L.: Interior-point method for nuclear norm approximation with application to system identification. SIAM Journal on Matrix Analysis and Applications 31(3), 1235–1256 (2009)CrossRefzbMATHMathSciNetGoogle Scholar
  27. 27.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online dictionary learning for sparse coding. In: Proceedings of the 26th Annual International Conference on Machine Learning, pp. 689–696. ACM, New York (2009)Google Scholar
  28. 28.
    Mehlhorn, K.: Data structures and algorithms 3: multi-dimensional searching and computational geometry. Springer-Verlag New York, Inc., New York (1984)CrossRefzbMATHGoogle Scholar
  29. 29.
    Murray, J., Kreutz-Delgado, K.: Sparse image coding using learned overcomplete dictionaries. Machine Learning for Signal Processing, 579–588 (September 2004)Google Scholar
  30. 30.
    Tuzel, O., Porikli, F., Meer, P.: Covariance Tracking using Model Update Based on Lie Algebra. In: CVPR (2006)Google Scholar
  31. 31.
    Pang, Y., Yuan, Y., Li, X.: Gabor-based region covariance matrices for face recognition. IEEE Tran. Circuits and Sys. for Video Tech. 18(7), 989–993 (2008)CrossRefGoogle Scholar
  32. 32.
    Phillips, P., Moon, H., Rizvi, S., Rauss, P.: The FERET evaluation methodology for face-recognition algorithms. Pattern Analysis and Machine Intelligence 22(10), 1090–1104 (2000)CrossRefGoogle Scholar
  33. 33.
    Phillips, P., Wechsler, H., Huang, J., Rauss, P.: The FERET database and evaluation procedure for face-recognition algorithms. Image and Vision Computing 16(5), 295–306 (1998)CrossRefGoogle Scholar
  34. 34.
    Shen, C., Welsh, A., Wang, L.: PSDBoost: Matrix-generation Linear Programming for Positive Semidefinite Matrices Learning. In: Advances Neural Information Processing Systems (2008)Google Scholar
  35. 35.
    Shen, C., Kim, J., Wang, L.: Scalable large-margin mahalanobis distance metric learning. Neural Networks 21(9), 1524–1530 (2010)CrossRefGoogle Scholar
  36. 36.
    Sivalingam, R., Boley, D., Morellas, V., Papanikolopoulos, N.: Tensor sparse coding for region covariances. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 722–735. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  37. 37.
    Turaga, P., Chellappa, R.: Nearest-neighbor search algorithms on non-Euclidean manifolds for computer vision applications. In: Indian Conf. Comp. Vis. Graph. and Img. Proc., pp. 282–289 (2010)Google Scholar
  38. 38.
    Wang, C., Blei, D., Fei-Fei, L.: Simultaneous image classification and annotation. In: Computer Vision and Pattern Recognition (2010)Google Scholar
  39. 39.
    Weiss, Y., Torralba, A., Fergus, R.: Spectral hashing. Advances in Neural Information Processing Systems 21, 1753–1760 (2009)Google Scholar
  40. 40.
    Wright, J., Ma, Y., Mairal, J., Spairo, G., Huang, T., Yan, S.: Sparse representation for computer vision and pattern recognition. In: CVPR (2009)Google Scholar
  41. 41.
    Yuan, C., Hu, W., Li, X., Maybank, S., Luo, G.: Human action recognition under log-euclidean riemannian metric. In: ACCV, pp. 343–353 (2010)Google Scholar
  42. 42.
    Zhang, H., Berg, A., Maire, M., Malik, J.: SVM-KNN: Discriminative nearest neighbor classification for visual category recognition. In: Computer Vision and Pattern Recognition, vol. 2, pp. 2126–2136. IEEE, Los Alamitos (2006)Google Scholar
  43. 43.
    Zhu, H., Zhang, H., Ibrahim, J., Peterson, B.: Statistical analysis of diffusion tensors in diffusion-weighted magnetic resonance imaging data. Journal of the American Statistical Association 102(480), 1085–1102 (2007)CrossRefzbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Suvrit Sra
    • 1
  • Anoop Cherian
    • 2
  1. 1.MPI for Intelligent SystemsTübingenGermany
  2. 2.University of MinnesotaTwin CitiesUSA

Personalised recommendations