Abstract
We introduced an algorithm for sequence alignment, based on maximizing local space-time correlations. Our algorithm aligns sequences of the same action performed at different times and places by different people, possibly at different speeds, and wearing different clothes. Moreover, the algorithm offers a unified approach to the problem of sequence alignment for a wide range of scenarios (e.g., sequence pairs taken with stationary or jointly moving cameras, with the same or different photometric properties, with or without moving objects). Our algorithm is applied directly to the dense space-time intensity information of the two sequences (or to filtered versions of them). This is done without prior segmentation of foreground moving objects, and without prior detection of corresponding features across the sequences. Examples of challenging sequences with complex actions are shown, including ballet dancing, actions in the presence of other complex scene dynamics (clutter), as well as multi-sensor sequence pairs.
Chapter PDF
References
Burt, P., Hingorani, R., Kolczynski, R.: Mechanisms for isolating component patterns in the sequential analysis of multiple motion. In: Workshop on Visual Motion (1991)
Carlsson, S.: Recognizing walking people. IJRR 22, 359–370 (2003)
Caspi, Y., Irani, M.: Aligning non-overlapping sequences. IJCV 48 (2002)
Caspi, Y., Irani, M.: Spatio-temporal alignment of sequences. T-PAMI (2002)
Caspi, Y., Simakov, D., Irani, M.: Feature-based sequence-to-sequence matching. In: VMODS (2002)
Giese, M.A., Poggio, T.: Synthesis and recognition of biological motion patterns based on linear superposition of prototypical motion sequences. In: IEEE Workshop on Multi-View Modeling and Analysis of Visual Scenes (1999)
Gonzalez, R.C., Woods, R.E.: Digital Image Processing. Addison-Wesley, Reading (1993)
Irani, M., Anandan, P.: Robust multi-sensor image alignment. In: ICCV (1998)
Press, W., Flannery, B., Teukolsky, S., Vetterling, W.: Numerical Recipes in C. Cambridge University Press, Cambridge (1988)
Rao, C., Gritai, A., Shah, M., Syeda-Mahmood, T.: View-invariant alignment and matching of video sequences. In: ICCV, pp. 939–945 (2004)
Shechtman, E., Irani, M.: Space-time behavior based correlation. In: CVPR (2005)
Thomas, G., Finney, R.: Calculus and Analytic Geometry, 9th edn. Addison-Wesley, Reading (1996)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ukrainitz, Y., Irani, M. (2006). Aligning Sequences and Actions by Maximizing Space-Time Correlations. In: Leonardis, A., Bischof, H., Pinz, A. (eds) Computer Vision – ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3953. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11744078_42
Download citation
DOI: https://doi.org/10.1007/11744078_42
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33836-9
Online ISBN: 978-3-540-33837-6
eBook Packages: Computer ScienceComputer Science (R0)