Graph Regularized ICA for Over-Complete Feature Learning

  • Yanhui Xiao
  • Zhenfeng Zhu
  • Yao Zhao
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7633)


Independent Component Analysis with a soft Reconstruction cost (RICA) has been recently presented to learn highly over-complete sparse features even on unwhitened data. However, RICA failed to consider the geometrical structure of the data space, which has been shown essential for classification problems. To address this problem, we propose a graph regularized ICA model with Reconstruction constraint for image classification, called gRICA. In particular, we construct an affinity graph to encode the geometrical information, and thereby learn a graph regularized over-complete basis which makes sparse representations respect the graph structure. Experiments conducted on several datasets show the effectiveness of gRICA for classification.


Independent Component Analysis Training Image Sparse Representation Independent Component Analysis Sparse Code 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Hyvärinen, A., Karhunen, J., Oja, E.: Independent component analysis, vol. 26. Wiley-Interscience (2001)Google Scholar
  2. 2.
    Hyvärinen, A., Hurri, J., Hoyer, P.: Natural image statistics, vol. 1. Springer (2009)Google Scholar
  3. 3.
    Le, Q., Ngiam, J., Chen, Z., Chia, D., Koh, P., Ng, A.: Tiled convolutional neural networks. In: NIPS, vol. 23 (2010)Google Scholar
  4. 4.
    Coates, A., Lee, H., Ng, A.: An analysis of single-layer networks in unsupervised feature learning. In: AISTATS, vol. 1001 (2010)Google Scholar
  5. 5.
    Bengio, Y., Lamblin, P., Popovici, D., Larochelle, H.: Greedy layer-wise training of deep networks, vol. 19 (2007)Google Scholar
  6. 6.
    Hinton, G., Osindero, S., Teh, Y.: A fast learning algorithm for deep belief nets. Neural Computation 18(7) (2006)Google Scholar
  7. 7.
    Le, Q., Karpenko, A., Ngiam, J., Ng, A.: Ica with reconstruction cost for efficient overcomplete feature learning. In: NIPS (2011)Google Scholar
  8. 8.
    Cai, D., He, X., Han, J., Huang, T.: Graph regularized nonnegative matrix factorization for data representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(8) (2011)Google Scholar
  9. 9.
    Zheng, M., Bu, J., Chen, C., Wang, C., Zhang, L., Qiu, G., Cai, D.: Graph regularized sparse coding for image representation. IEEE Transactions on Image Processing 20(5) (2011)Google Scholar
  10. 10.
    Chung, F.: Spectral graph theory, vol. 92. Amer. Mathematical Society (1997)Google Scholar
  11. 11.
    Schmidt, M.: minfunc. (2005)Google Scholar
  12. 12.
    Fei-Fei, L., Fergus, R., Perona, P.: Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. Computer Vision and Image Understanding 106(1) (2007)Google Scholar
  13. 13.
    Yang, J., Yu, K., Gong, Y., Huang, T.: Linear spatial pyramid matching using sparse coding for image classification. In: CVPR. IEEE (2009)Google Scholar
  14. 14.
    Jiang, Z., Lin, Z., Davis, L.: Learning a discriminative dictionary for sparse coding via label consistent k-svd. In: CVPR. IEEE (2011)Google Scholar
  15. 15.
    Yu, K., Zhang, T.: Improved local coordinate coding using local tangents. In: ICML (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Yanhui Xiao
    • 1
  • Zhenfeng Zhu
    • 1
  • Yao Zhao
    • 1
  1. 1.Institute of Information ScienceBeijing Jiaotong UniversityBeijingChina

Personalised recommendations