Maximizing Causal Information of Natural Scenes in Motion


Abstract Natural scenes contain a huge amount of information if counted by spatial pixels and temporal frames. However, most of the information is redundant because the pixels and the frames are highly correlated. The optical flow, generated by motions of the objects and the observer, contributes significantly to the statistical regularity of such spatiotemporal correlations. The visual system of an animal such as the human is highly adapted to this statistical regularity such that the visual sensitivity follows the same contours as the spatiotemporal correlations of natural scenes in motion, in particular, along two axes: space and motion instead of space and time. Furthermore, vision is an active process, during which eye movements interact with visual scenes and select images that arrive on the retina: pursuits and fixations on objects significantly alter the image velocity distributions on the fovea and the periphery, which lead to the dependence of the visual sensitivity on the retinal eccentricity; saccades between objects change the natural scene statistics dynamically, which lead to the dependence of the visual sensitivity on the time relative to saccades. All of these can be accounted for by the proposed ecological theory that the visual system maximizes the causal information of the natural visual input.


Visual Input Temporal Frequency Visual Scene Natural Scene Contrast Signal 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The author wishes to thank Dr. Theodore Weyand for many interesting discussions about the LGN function and to thank Dr. Anna Kashina for her critical reading of the manuscript. This work was supported in part by FAU under the grant No. RIA-25, by NIMH under the grant No. MH019116, and by NSF under the grant No. PHY99-07949.


  1. 1..
    Atick JJ (1992) Could information theory provide an ecological theory of sensory processing? Network-Compu. Neural. 3, 213–251.CrossRefGoogle Scholar
  2. 2.
    Atick JJ, Redlich AN (1990) Towards a theory of early visual processing. Neural Comp. 2, 308–320.CrossRefGoogle Scholar
  3. 3.
    Atick JJ, Redlich AN (1992) What does the retina know about natural scenes? Neural Comp. 4, 196–210.CrossRefGoogle Scholar
  4. 4.
    Atick JJ, Li Z, Redlich AN (1993) What does post-adaptation color appearance reveal about cortical color representation. Vision Res. 33, 123–129.PubMedCrossRefGoogle Scholar
  5. 5.
    Attneave F (1954) Some informational aspects of visual perception. Psychol Rev 61, 183–193.PubMedCrossRefGoogle Scholar
  6. 6.
    Barlow HB (1961) Possible principles underlying the transformation of sensory messages. In: Sensory Communication (Rosenblith WA, ed, MIT Press, Cambridge) 217–234.Google Scholar
  7. 7.
    Brenner N, Bialek W, VanSteveninck RR (2000) Adaptive Rescaling Maximizes Information Transmission. Neuron 26, 695–702.PubMedCrossRefGoogle Scholar
  8. 8.
    Buisseret P (1995) Influence of extraocular-muscle proprioception on vision. Physiological Reviews 75, 323–338.PubMedGoogle Scholar
  9. 9.
    Burton GJ, Moorhead IR (1987) Color and spatial structure in natural scenes. Applied Optics 26, 157–170.PubMedCrossRefGoogle Scholar
  10. 10.
    Cafforio C, Rocca F (1975) Methods for measuring small displacements of television images. IEEE Info. Theo. 22, 573–579.CrossRefGoogle Scholar
  11. 11.
    Cai DQ, DeAngelis GC, Freeman RD (1997) Spatiotemporal receptive-field organization in the lateral geniculate-nucleus of cats and kittens. J of Neurophysiol 78, 1045–1061.Google Scholar
  12. 12.
    Carandini M, Ferster D (1998) A tonic hyperpolarization underlying contrast adaptation in cat visual cortex. Science 276, 949–952.CrossRefGoogle Scholar
  13. 13.
    Dan Y, Atick JJ, Reid RC (1996) Efficient coding of natural scenes in the lateral geniculate nucleus: Experimental test of a computational theory. J Neurosci 16, 3351–3362.PubMedGoogle Scholar
  14. 14.
    Dastjerdi M (2007) Efficient representation of natural visual input in the thalamus. Ph D thesis, University Microfilms International, Ann Arbor, MI.Google Scholar
  15. 15.
    Dastjerdi M, Weyand TG, Dong DW (2003) The spatiotemoral receptive field (STRF) properties of the lateral geniculate nucleus (LGN) in the awake cats during free-viewing natural time-varying images. Society for Neuroscience Abstract 29, 229.3.Google Scholar
  16. 16.
    Ditchburn RW, Ginsborg BL (1952) Vision with a stabilized retinal image. Nature 170, 36–37.PubMedCrossRefGoogle Scholar
  17. 17.
    Donaldson IML, Dixon RA (1980) Excitation of units in the lateral geniculate and contiguous nuclei of the cat by stretch of extrinsic ocular muscles. Exp. Brain Res. 38, 245–255.PubMedCrossRefGoogle Scholar
  18. 18.
    Dong DW (1994) Associative decorrelation dynamics: a theory of self-organization and optimization in feedback networks. In: Advances in Neural Information Processing Systems 7 (Tesauro G, Touretzky DS, Leen TK, eds, MIT Press, Cambridge, MA), 925–932.Google Scholar
  19. 19.
    Dong DW (2001a) Spatiotemporal inseparability of natural images and visual sensitivities. In: Computational, neural & ecological constraints of visual motion processing (Zanker JM, Zeil J, eds, Springer Verlag, Berlin Heidelberg New York), 371–380.Google Scholar
  20. 20.
    Dong DW (2001b) Effects of eye movements on information processing: Visual input statistics during free-viewing natural time-varying images. Invest ophth & Vis Sci 42, S3347.Google Scholar
  21. 21.
    Dong DW, Atick JJ (1995a) Statistics of natural time-varying images. Network-Compu. Neural. 6, 345–358.CrossRefGoogle Scholar
  22. 22.
    Dong DW, Atick JJ (1995b) Temporal decorrelation: a theory of lagged and nonlagged responses in the lateral geniculate nucleus. Network-Compu. Neural. 6, 159–178.CrossRefGoogle Scholar
  23. 23.
    Dragoi V, Sharma J, Sur M (2000) Adaptation-induced plasticity of orientation tuning in adult visual cortex. Neuron 28, 287–298.PubMedCrossRefGoogle Scholar
  24. 24.
    Eckert MP, Buchsbaum G (1993) Efficient coding of natural time varying images in the early visual system. Phil. Trans. R. Soc. Lond. B 339, 385–395.CrossRefGoogle Scholar
  25. 25.
    Field DJ (1987) Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A 4, 2379–2394.PubMedCrossRefGoogle Scholar
  26. 26.
    Field DJ (1994) What is the goal of sensory coding. Neural Computation 6, 559–601.CrossRefGoogle Scholar
  27. 27.
    Guido W, Weyand TG (1995) Burst responses in thalamic relay cells of the awake, behaving cat. J. Neurophysiol. 74, 1782–1786.PubMedGoogle Scholar
  28. 28.
    Horn BKP, Schunk BG (1981) Determining optical flow. Artificial Intelligence 17, 185–203.CrossRefGoogle Scholar
  29. 29.
    Hoyer PO, Hyvarinen A (2000) Independent component analysis applied to feature extraction from colour and stereo images. Network-Compu. Neural. 3, 191–210.CrossRefGoogle Scholar
  30. 30.
    Hubel DH (1988) Eye, Brain, and Vision. (Freeman and Company, New York).Google Scholar
  31. 31.
    Jain JR, Jain AK (1981) Displacement measurement and its application in interframe image coding. IEEE T. Commun. 29, 1799–1808.CrossRefGoogle Scholar
  32. 32.
    Kelly DH (1961) Visual responses to time-dependent stimuli.1. amplitude sensitivity measurements. Journal of the Optical Society of America 51, 422–429.PubMedCrossRefGoogle Scholar
  33. 33.
    Kelly DH (1979) Motion and vision. II. Stabilized spatio-temporal threshold surface. J. Opt. Soc. Am. 69, 1340–1349.PubMedCrossRefGoogle Scholar
  34. 34.
    Koenderink JJ, VanDoorn AJ (1987) Facts on Optical Flow. Bio. Cybern. 56, 247–254.CrossRefGoogle Scholar
  35. 35.
    Lee D, Malpeli JG (1998) Effects of saccades on the activity of neurons in the cat lateral geniculate nucleus. J. Neurophysiol. 79, 922–936.PubMedGoogle Scholar
  36. 36.
    Lal R, Friedlander MJ (1990a) Effect of eye position changes on retinogeniculate transmission in the cat. J. Neurophysiol. 63, 502–522.PubMedGoogle Scholar
  37. 37.
    Lal R, Friedlander MJ (1990b) Effect of passive eye movement on retinogeniculate transmission in the cat. J. Neurophysiol. 63, 523–538.PubMedGoogle Scholar
  38. 38.
    Li Z, Atick JJ (1994) Efficient stereo coding in the multiscale representation. Network-Compu. Neural. 5, 1–18.CrossRefGoogle Scholar
  39. 39.
    Linsker R (1988) Self-organization in a perceptual network. Computer 21, 105–117.CrossRefGoogle Scholar
  40. 40.
    Linsker R (1989) An application of the principle of maximum information preservation to linear systems. In: Advances in Neural Information Processing Systems 1 (Touretzky DS, ed, Morgan Kaufman, San Mateo, CA), 186–194.Google Scholar
  41. 41.
    Muller JR, Metha AB, Krauskopf J, Lennie P (1999) Rapid Adaptation in Visual Cortex to the Structure of Images. Science 285, 1405–1408.PubMedCrossRefGoogle Scholar
  42. 42.
    Olshausen BA, Field DJ (1996) Emergence of simple-cell receptive-field properties by learning a sparse code for natural images. Nature 381, 607–609.PubMedCrossRefGoogle Scholar
  43. 43.
    Parraga CA, Brelstaff G, Troscianko T (1998) Color and luminance information in natural scenes. J Opt Soc Am 15, 563–569.CrossRefGoogle Scholar
  44. 44.
    Penev PS, Atick JJ (1996) Local feature analysis: A general statistical theory for object representation. Network-Compu. Neural. 7, 477–500.CrossRefGoogle Scholar
  45. 45.
    Ramcharan EJ, Gnadt JW, Sherman SM (2001) The effects of saccadic eye movements on the activity of geniculate relay neurons in the monkey. Visual Neuroscience 18, 253–258.PubMedCrossRefGoogle Scholar
  46. 46.
    Reppas JB, Usrey WM, Reid RC (2002) Saccadic eye movements modulate visual responses in the lateral geniculate nucleus. Neuron 35, 961–974.PubMedCrossRefGoogle Scholar
  47. 47.
    Riggs LA, Ratliff F (1952) The effects of counteracting the normal movements of the eye. J. Opt. Soc. Am. 42, 872–873.Google Scholar
  48. 48.
    Rucci M, Edelman GM, Wray J (2000) Modeling LGN responses during free-viewing: A possible role of microscopic eye movements in the refinement of cortical orientation selectivity. J Neurosci 20, 4708–4720.PubMedGoogle Scholar
  49. 49.
    Ruderman DL, Bialek W (1994) Statistics of natural images: scaling in the woods. Phy. Rev. Let. 73(6), 814–817.CrossRefGoogle Scholar
  50. 50.
    Ruderman DL, Cronin TW, Chiao CC (1998) Statistics of cone responses to natural images: implications for visual coding. J Opt Soc Am 15, 2036–2045.CrossRefGoogle Scholar
  51. 51.
    Schmidt M (1996) Neurons in the cat pretectum that project to the dorsal lateral geniculate nucleus are activated during saccades. J. Neurophysiol. 76, 2907–2918.PubMedGoogle Scholar
  52. 52.
    Shannon, CE, Weaver W (1949) A mathematical theory of communication. Univ. of Illinois Press, Urbana.Google Scholar
  53. 53.
    Simoncelli EP, Olshausen BA (2001) Natural image statistics and neural representation. Ann. Rev. Neurosci. 24, 1193–1216.PubMedCrossRefGoogle Scholar
  54. 54.
    Singer W (1977) Control of thalamic transmission by corticofugal and ascending reticular pathways in the visual system. Phys. Reviews 57, 386–420.Google Scholar
  55. 55.
    Smirnakis SM, Berry MJ, Warland DK, Bialek W, Meister M (1997) Adaptation of retinal processing to image contrast and spatial scale. Nature 386, 69–73.PubMedCrossRefGoogle Scholar
  56. 56.
    Srinivasan MV, Laughlin SB, Dubs A (1982) Predictive coding - a fresh view of inhibition in the retina. Proc. R. Soc. B 216, 427–459.CrossRefGoogle Scholar
  57. 57.
    Sterling P (1998) Retina. In: The Synaptic Organization of the Brain (Shepherd GM, ed, Oxford, New York) 205–253.Google Scholar
  58. 58.
    Swadlow HA, Weyand TG (1987) Corticogeniculate neurons, corticotectal neurons, and suspected interneurons in visual cortex of awake rabbits: Receptive field properties, axonal properties, and effects of EEG arousal. J. Neurophysiol. 57, 977–1001.PubMedGoogle Scholar
  59. 59.
    Tailor DR, Finkel LH, Buchsbaum G (2000) Color-opponent receptive fields derived from independent component analysis of natural images. Vision Research 40, 2671–2676.PubMedCrossRefGoogle Scholar
  60. 60.
    Truccolo WA, Dong DW (2001) Dynamic temporal decorrelation: information theoretic and biophysical model of the functional role of lateral geniculate nucleus. Neurocomputing 38–40, 993–1001.CrossRefGoogle Scholar
  61. 61.
    VanHateren JH (1992) Theoretical predictions of spatiotemporal receptive fields of fly LMCs, and experimental validation. J. Comp. Physiol. A 171, 157–170.Google Scholar
  62. 62.
    VanHateren JH, Ruderman DL (1998) Independent component analysis of natural image sequences yields spatio-temporal filters similar to simple cells in primary visual cortex. P Roy Soc Lond B Bio 265, 2315–2320.CrossRefGoogle Scholar
  63. 63.
    Verri A, Poggi T (1989) Motion Field and Optical Flow: qualitative properties. IEEE T. Pat. Ana. Mac. Intel. 11, 490–498.CrossRefGoogle Scholar
  64. 64.
    Victor JD (1999) Temporal aspects of neural coding in the retina and lateral geniculate. Network-Comp. Neural. 10, R1–R66CrossRefGoogle Scholar
  65. 65.
    Wandell BA (1995) Foundations of Vision (Sinauer, Sunderland).Google Scholar
  66. 66.
    Webster MA, Mollon JD (1997) Adaptation and the color statistics of natural images. Vision Res. 37, 3283–3298.PubMedCrossRefGoogle Scholar
  67. 67.
    Webster MA, Miyahara E (1997) Contrast adaptation and the spatial structure of natural images. J Opt Soc Am A 14, 2355–2366.CrossRefGoogle Scholar
  68. 68.
    Wehmeier U, Dong DW, Koch C, VanEssen DC (1989) Modeling the mammalian visual system. In: Methods in Neuronal Modeling: from Synapses to Networks (Koch C, Segev I, eds, MIT Press, Cambridge), 335–360.Google Scholar
  69. 69.
    Yarbus AL (1967) Eye Movements and Vision (Plenum, New York).Google Scholar
  70. 70.
    Zhaoping L (2002) Optimal Sensory Encoding. In: The Handbook of Brain Theory and Neural Networks (Second Edition) (Arbib MA, ed, MIT Press, Cambridge, MA), 815–819.Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Center for Complex Systems and Brain SciencesFlorida Atlantic UniversityBoca RatonUSA

Personalised recommendations