Advertisement

Efficient Highly Over-Complete Sparse Coding Using a Mixture Model

  • Jianchao Yang
  • Kai Yu
  • Thomas Huang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6315)

Abstract

Sparse coding of sensory data has recently attracted notable attention in research of learning useful features from the unlabeled data. Empirical studies show that mapping the data into a significantly higher-dimensional space with sparse coding can lead to superior classification performance. However, computationally it is challenging to learn a set of highly over-complete dictionary bases and to encode the test data with the learned bases. In this paper, we describe a mixture sparse coding model that can produce high-dimensional sparse representations very efficiently. Besides the computational advantage, the model effectively encourages data that are similar to each other to enjoy similar sparse representations. What’s more, the proposed model can be regarded as an approximation to the recently proposed local coordinate coding (LCC), which states that sparse coding can approximately learn the nonlinear manifold of the sensory data in a locally linear manner. Therefore, the feature learned by the mixture sparse coding model works pretty well with linear classifiers. We apply the proposed model to PASCAL VOC 2007 and 2009 datasets for the classification task, both achieving state-of-the-art performances.

Keywords

Gaussian Mixture Model Sparse Representation Local Descriptor Sparse Code Descriptor Space 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381 (1996)Google Scholar
  2. 2.
    Olshausen, B.A., Field, D.J.: Sparse coding with an overcomplete basis set: A strategy emplyed by v1?. Vision Research (1997)Google Scholar
  3. 3.
    Donoho, D.L.: For most large underdetermined systems of linear equations, the minimal ℓ1-nomr solution is also the sparest solution. Comm. on Pure and Applied Math. (2006)Google Scholar
  4. 4.
    Aharon, M., Elad, M., Bruckstein, A.: K-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing (2006)Google Scholar
  5. 5.
    Yang, J., Wright, J., Huang, T., Ma, Y.: Image super-resolution as sparse representation of raw image patches. In: CVPR (2008)Google Scholar
  6. 6.
    Bradley, D.M., Bagnell, J.A.: Differential sparse coding. In: NIPS (2008)Google Scholar
  7. 7.
    Yu, K., Zhang, T., Gong, Y.: Nonlinear learning using local coordinate coding. In: Advances in Neural Information Processing Systems, vol. 22 (2009)Google Scholar
  8. 8.
    Cheng, B., Yang, J., Yan, S., Huang, T.: Learning with ℓ1 graph for image analysis. IEEE Transactions on Image Processing (2010)Google Scholar
  9. 9.
    Cevher, V., Sankaranarayanan, A., Duarte, M.F., Reddy, D., Baraniuk, R.G., Chellappa, R.: Compressive sensing for background subtraction. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 155–168. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. 10.
    Wright, J., Yang, A., Ganesh, A., Satry, S., Ma, Y.: Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) (Feburay 2009)Google Scholar
  11. 11.
    Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2009)Google Scholar
  12. 12.
    Yang, J., Yu, K., Huang, T.: Supervised translation-invariant sparse coding. In: IEEE Conference on Computer Vision and Pattern Recognition (2010)Google Scholar
  13. 13.
    Lee, H., Battle, A., Raina, R., Ng, A.Y.: Efficient sparse coding algorithms. In: NIPS (2006)Google Scholar
  14. 14.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research 11, 19–60 (2010)Google Scholar
  15. 15.
    Mairal, J., Bach, F., Ponce, J., Sapiro, G.: Supervised dictionary learning. In: NIPS (2008)Google Scholar
  16. 16.
    Ranzato, M.A., Boureau, Y.L., LeCun, Y.: Sparse feature learning for deep belief networks. In: NIPS (2007)Google Scholar
  17. 17.
    Raina, R., Battle, A., Lee, H., Packer, B., Ng, A.Y.: Self-taught learning: Transfer learning from unlabeled dataGoogle Scholar
  18. 18.
    Kavukcuoglu, K., Ranzato, M., Fergus, R., LeCun, Y.: Learing invariant features through topographic filter maps. In: IEEE Conference on Computer Vison and Pattern Recognition (2009)Google Scholar
  19. 19.
    Lazebnik, S., Schmid, C., Ponce, J.: Beyonad bags of features: spatial pyramid matching for recognizing natural scene categories. In: IEEE Conference on Computer Vision and Pattern Recognition (2006)Google Scholar
  20. 20.
    Everingham, M., Gool, L.V., Williams, C., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge, VOC 2007 Results (2007), http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html
  21. 21.
    Bishop, C.M.: Pattern recognition and machine learning. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  22. 22.
    Zhou, X., Cui, N., Li, Z., Liang, F., Huang, T.: Hierarchical gaussianization for image classification. In: IEEE International Conference on Computer Vision (ICCV) (2009)Google Scholar
  23. 23.
    Everingham, M., Gool, L.V., Williams, C., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge, VOC 2009 Results (2009), http://www.pascal-network.org/challenges/VOC/voc2009/workshop/index.html
  24. 24.
    Marszalek, M., Schmid, C., Harzallah, H., van de Weijer, J.: Learning representations for visual object class recognition. In: PASCAL Visual Object Class Challenge (VOC) workshop (2007)Google Scholar
  25. 25.
    Wang, J., Yang, J., Yu, K., Lv, F.: Locality-constrained linear coding for image classification. In: IEEE Conference on Computer Vision and Pattern Classificatoin (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Jianchao Yang
    • 1
  • Kai Yu
    • 2
  • Thomas Huang
    • 1
  1. 1.Beckman InstituteUniversity of Illinois at Urbana Champaign
  2. 2.NEC Laboratories AmericaCupertino

Personalised recommendations