What Is the Role of Independence for Visual Recognition?

  • Nuno Vasconcelos
  • Gustavo Carneiro
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2350)


Independent representations have recently attracted significant attention from the biological vision and cognitive science communities. It has been 1) argued that properties such as sparseness and independence play a major role in visual perception, and 2) shown that imposing such properties on visual representations originates receptive fields similar to those found in human vision. We present a study of the impact of feature independence in the performance of visual recognition architectures. The contributions of this study are of both theoretical and empirical natures, and support two main conclusions. The first is that the intrinsic complexity of the recognition problem (Bayes error) is higher for independent representations. The increase can be significant, close to 10% in the databases we considered. The second is that criteria commonly used in independent component analysis are not sufficient to eliminate all the dependencies that impact recognition. In fact, “independent components” can be less independent than previous representations, such as principal components or wavelet bases.


Feature Space Discrete Cosine Transform Independent Component Analysis Recognition Accuracy Independent Component Analysis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    A. Bell and T. Sejnowski. The independent components of natural scenes are edge filters. Vision Research, 37(23):3327–3328, December 1997.Google Scholar
  2. 2.
    J. Bergen and E. Adelson. Early Vision and Texture Perception. Nature, 333(6171):363–364, 1988.CrossRefGoogle Scholar
  3. 3.
    J. Bergen and M. Landy. Computational Modeling of Visual Texture Segregation. In M. Landy and J. Movshon, editors, Computational Models of Visual Processing. MIT Press, 1991.Google Scholar
  4. 4.
    J. Cardoso. Blind Signal Separation: Statistical Principles. Proceedings of the IEEE, 90(8):2009–20026, October 1998.Google Scholar
  5. 5.
    P. Comon. Independent Component Analysis, A New concept? Signal Processing, 36:287–314, 1994.zbMATHCrossRefGoogle Scholar
  6. 6.
    L. Devroye, L. Gyorfi, and G. Lugosi. A Probabilistic Theory of Pattern Recognition. Springer-Verlag, 1996.Google Scholar
  7. 7.
    D. Field. What is the goal of sensory coding? Neural Computation, 6(4):559–601, January 1989.Google Scholar
  8. 8.
    I. Fogel and D. Sagi. Gabor Filters as Texture Discriminators. Biol. Cybern., 61:103–113, 1989.CrossRefGoogle Scholar
  9. 9.
    D. Hubel and T. Wiesel. Brain Mechanisms of Vision. Scientific American, September 1979.Google Scholar
  10. 10.
    A. Hyvarinen and E. Oja. Independent Component Analysis: Algorithms and Applications. Neural Networks, 13:411–430, 2000.CrossRefGoogle Scholar
  11. 11.
    N. Jayant and P. Noll. Digital Coding of Waveforms: Principles and Applications to Speech and Video. Prentice Hall, 1984.Google Scholar
  12. 12.
    D. Knill and W. Richards. Perception as Bayesian Inference. Cambridge Univ. Press, 1996.Google Scholar
  13. 13.
    J. Malik and P. Perona. Preattentive Texture Discrimination with Early Vision Mechanisms. Journal of the Optical Society of America, 7(5):923–932, May 1990.Google Scholar
  14. 14.
    S. Mallat. A Theory for Multiresolution Signal Decomposition: the Wavelet Representation. IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 11:674–693, July 1989.Google Scholar
  15. 15.
    B. Olshausen and D. Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381:607–609, 1996.CrossRefGoogle Scholar
  16. 16.
    M. Porat and Y. Zeevi. Localized Texture Processing in Vision: Analysis and Synthesis in the Gaborian Space. IEEE Trans. on Biomedical Engineering, 36(1):115–129, January 1989.Google Scholar
  17. 17.
    J. Portilla and E. Simoncelli. Texture Modeling and Synthesis using Joint Statistics of Complex Wavelet Coefficients. In IEEE Workshop on Statistical and Computational Theories of Vision, Fort Collins, Colorado, 1999.Google Scholar
  18. 18.
    D. Sagi. The Psychophysics of Texture Segmentation. In T. Papathomas, editor, Early Vision and Beyond, chapter 7. MIT Press, 1996.Google Scholar
  19. 19.
    A. Sutter, J. Beck, and N. Graham. Contrast and Spatial Variables in Texture Segregation: testing a simple spatial-frequency channels model. Perceptual Psychophysics, 46:312–332, 1989.Google Scholar
  20. 20.
    N. Vasconcelos. Bayesian Models for Visual Information Retrieval. PhD thesis, Massachusetts Institute of Technology, 2000.Google Scholar
  21. 21.
    N. Vasconcelos and A. Lippman. A Probabilistic Architecture for Content-based Image Retrieval. In Proc. IEEE Computer Vision and Pattern Recognition Conf., Hilton Head, North Carolina, 2000.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Nuno Vasconcelos
    • 1
  • Gustavo Carneiro
    • 2
  1. 1.Compaq Computer CorporationCambridge Research LaboratoryUK
  2. 2.Department of Computer ScienceUniversity of TorontoCanada

Personalised recommendations