Qualitative Spatiotemporal Analysis Using an Oriented Energy Representation

  • Richard P. Wildes
  • James R. Bergen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1843)


This paper presents an approach to representing and analyzing spatiotemporal information in support of making qualitative, yet semantically meaningful distinctions at the earliest stages of processing. A small set of primitive classes of spatiotemporal structure are proposed that correspond to categories of stationary, coherently moving, incoherently moving, flickering, scintillating and “too unstructured to support further inference”. It is shown how these classes can be represented and distinguished in a uniform fashion in terms of oriented energy signatures. Further, empirical results are presented that illustrate the use of the approach in application to natural imagery. The importance of the described work is twofold: (i) From a theoretical point of view a semantically meaningful decomposition of spatiotemporal information is developed. (ii) From a practical point of view, the developed approach has the potential to impact real world image understanding and analysis applications. As examples: The approach could be used to support early focus of attention and cueing mechanisms that guide subsequent activities by an intelligent agent; the approach could provide the representational substrate for indexing video and other spatiotemporal data.


Coherent Motion Spatiotemporal Data Spatiotemporal Information Spatiotemporal Structure Spatial Axis 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Adelson, E., Bergen, J.: Spatiotemporal energy models for the perception of motion. JOSA A 2 (1985) 284–299Google Scholar
  2. 2.
    Anderson, C., Burt, P., van der Wal, G.: Change detection and tracking using pyramid transform techniques. Proc. SPIE Conf. on Intell. Rob. and Comp. Vis. (1985) 300–305Google Scholar
  3. 3.
    Beauchemin, S., Barron, J.: The computation of optical flow. ACM Comp. Surv. 27 (1995) 433–467CrossRefGoogle Scholar
  4. 4.
    Bergen, J., Wilson, H.: Prediction of flicker sensitivities from temporal three pulse data. Vis. Res. (1985) 284–299Google Scholar
  5. 5.
    Black, M., Yacoob, Y., Jepson, A., Fleet, D.: Learning parameterized models of image motion. Proc. IEEE CVPR (1997) 561–567Google Scholar
  6. 6.
    Bruce, V., Green, B., Georgeson, M.: Visual Perception. East Sussex: Earlbaum (1996)Google Scholar
  7. 7.
    Chou, G.: A model of figure-ground segregation from kinetic occlusion. Proc. ICCV (1995) 1050–1057Google Scholar
  8. 8.
    Fleet, D., Jepson, A.: A Cascaded Approach to the Construction of Velocity Selective Mechanisms. RBCV Tech. Rep., TR-85-6, Dept. of Comp. Sci., University of Toronto (1985)Google Scholar
  9. 9.
    Fleet, D., Black, M., Jepson, A.: Motion feature extraction using steerable flow fields. Proc. IEEE CVPR (1998) 274–281Google Scholar
  10. 10.
    Freeman, A., Adelson, E.: The design and use of steerable filters. IEEE PAMI 13 (1991) 891–906Google Scholar
  11. 11.
    Granlund, G., Knutsson, H.: Signal Processing for Computer Vision. Boston: Kluwer (1995)Google Scholar
  12. 12.
    Grzywacz, N., Yuille, A.: A model for the estimation of local velocity by cells in the visual cortex. Proc. Roy. Soc. Lond. B 239 (1990) 129–161Google Scholar
  13. 13.
    Heeger, D.: A model for the extraction of image flow. JOSA A 4, (1997) 1455–1471Google Scholar
  14. 14.
    Jähne, B.: Digital Image Processing. Berlin: Springer-Verlag (1993)Google Scholar
  15. 15.
    Knutsson, H., Wilson, R., Granlund, G.: Anisotropic non-stationary image estimation and its applications-part I: Restoration of noisy images. IEEE TC 31 (1983) 388–397CrossRefGoogle Scholar
  16. 16.
    Koenderink, J.: Scale-time. Bio. Cyb. 58 (1988) 159–162zbMATHMathSciNetGoogle Scholar
  17. 17.
    Nelson, R., Polana, R.: Qualitative recognition of motion using temporal texture. CVGIP-IU 56 (1992) 78–89zbMATHCrossRefGoogle Scholar
  18. 18.
    van Ness, R., Bouman, M.: Spatial modulation transfer in the human eye. JOSA 57 (1967) 401–406Google Scholar
  19. 19.
    Niyogi, S.: Detecting kinetic occlusion. Proc. ICCV (1995) 1044–1049Google Scholar
  20. 20.
    Movshon, J., Thompson, I., Tolhurst, D.: Receptive field organization of complex cells in the cat’s striate cortex. J. Physiol. Lond. 283 (1978) 79–99Google Scholar
  21. 21.
    Reichardt, W.: Autocorrelation, a principle for the evaluation of sensory information by the central nervous system. In W. Rosenblith (Ed.) Sensory Communication, NY: Wiley (1961)Google Scholar
  22. 22.
    Reichardt, W., Poggio, T.: Figure-ground discrimination by relative movement in the visual system of the fly. Bio. Cyb. 35 (1979) 81–100CrossRefGoogle Scholar
  23. 23.
    van Santen, J., Sperling, G.: Temporal covariance model of human motion perception. JOSA A 1 (1984) 451–473Google Scholar
  24. 24.
    Stromeyer, C., Kronauer, R., Madsen, J., Klein, S.: Opponent mechanisms in human vision. JOSA A 1 (1984) 876–884Google Scholar
  25. 25.
    Thompson, P.: The coding of the velocity of movement in the human visual system. Vis. Res. 24 (1984) 41–45CrossRefGoogle Scholar
  26. 26.
    Thompson, W., Kearney, J.: Inexact vision. Proc. Workshop on Motion Rep. and Anal. (1986) 15–22Google Scholar
  27. 27.
    Verri, A., Poggio, T.: Against quantitative optical flow. IEEE PAMI 9 (1987) 171–180Google Scholar
  28. 28.
    Watson, A., Ahumada, A.: Model of human motion sensing. JOSA A 2 (1985) 322–341CrossRefGoogle Scholar
  29. 29.
    Watson, A., Robson, J.: Discrimination at threshold: Labelled detectors in human vision. Vis. Res. 21 (1981) 1115–1122CrossRefGoogle Scholar
  30. 30.
    Wildes, R.: A measure of motion salience. Proc. IEEE ICIP (1988) 183–187Google Scholar
  31. 31.
    Witkin, A.: Scale-space filtering. Proc. IJCAI (1983) 1019–1021Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2000

Authors and Affiliations

  • Richard P. Wildes
    • 1
  • James R. Bergen
    • 1
  1. 1.Sarnoff CorporationPrincetonUSA

Personalised recommendations