Abstract
In this work we present a method to synchronize video sequences of events that are acquired via uncalibrated cameras at unknown and dynamically varying temporal offsets. Unlike existing methods that synchronize videos of similar events (i.e., videos related to each other through the motion in the scene) up to an integer alignment, we establish sub-frame video synchronization. While contemporary synchronization algorithms implement a unidirectional alignment which biases the results towards a single reference sequence, we adopt a bi-directional or symmetrical alignment approach that results in a more optimal synchronization. To this end, we propose a novel symmetric transfer error which is dynamically minimized, and reduces the propagation of error from feature extraction and spatial mapping into temporal synchronization. The advantages of our approach are validated by tests conducted on (publicly available) real and synthetic sequences. We present qualitative and quantitative comparisons with another state-of-the-art algorithm. A unique application of this work in generating high-resolution 4D MRI data from multiple low-resolution MRI scans is described.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full chapter text
Chapter PDF
References
Caspi, Y., Irani, M.: Spatio-temporal alignment of sequences. PAMI 24, 1409–1424 (2002)
Caspi, Y., Simakov, D., Irani, M.: Feature-based sequence-to-sequence matching. IJCV 68, 53–64 (2006)
Singh, M., Basu, A., Mandal, M.: Event dynamics based temporal registration. IEEE Transactions on Multimedia 9, 1004–1015 (2007)
Lee, L., Romano, R., Stein, G.: Monitoring activities from multiple video streams: establishing a common coordinate frame. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 758–767 (2000)
Hess, R., Fern, A.: Improved video registration using non-distinctive local image features. In: CVPR, pp. 1–8 (2007)
Rao, C., Gritai, A., Shah, M., Mahmood, T.F.S.: View-invariant alignment and matching of video sequences. In: Proc. ICCV, pp. 939–945 (2003)
Rao, C., Shah, M., Syeda-Mahmood, T.: Invariance in motion analysis of videos. In: Proceedings of the ACM Int. Conf. on Multimedia, pp. 518–527 (2003)
Keogh, E., Pazzani, M.: Derivative dynamic time warping. In: SIAM International Conference on Data Mining (2001)
Dai, C., Zheng, Y., Li, X.: Subframe video synchronization via 3d phase correlation. In: ICIP, pp. 501–504 (2006)
Tuytelaars, T., VanGool, L.J.: Synchronizing video sequences. In: CVPR, pp. I: 762–768 (2004)
Tresadern, P., Reid, I.: Synchronizing image sequences of non-rigid objects. In: Proc. BMVC, vol. 2, pp. 629–638 (2003)
Carceroni, R.L., Padua, F.L.C., Santos, G.A.M.R., Kutulakos, K.N.: Linear sequence-to-sequence alignment. CVPR, I: 746–753 (2004)
Lei, C., Yang, Y.H.: Tri-focal tensor-based multiple video synchronization with subframe optimization. IEEE Trans. on IP 15, 2473–2480 (2006)
Wolf, L., Zomet, A.: Wide baseline matching between unsynchronized video sequences. IJCV 68, 43–52 (2006)
Pooley, D.W., Brooks, M.J., van den Hengel, A.J., Chojnacki, W.: A voting scheme for estimating the synchrony of moving-camera videos. In: ICIP, vol. 1, pp. I–413–16 (September 14-17, 2003)
Perperidis, D., Mohiaddin, R., Rueckert, D.: Spatio-temporal free-form registration of cardiac mr image sequences. Medical Image Analysis 9, 441–456 (2005)
Giese, M.A., Poggio, T.: Morphable models for the analysis and synthesis of complex motion patterns. Int. J. Comput. Vision 38, 59–73 (2000)
Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Gool, L.V.: A comparison of affine region detectors. Int. J. Comput. Vision 65, 43–72 (2005)
Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proceedings of British Machine Vision Conference, September 2002, vol. I, pp. 384–393 (2002)
Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Singh, M., Cheng, I., Mandal, M., Basu, A. (2008). Optimization of Symmetric Transfer Error for Sub-frame Video Synchronization. In: Forsyth, D., Torr, P., Zisserman, A. (eds) Computer Vision – ECCV 2008. ECCV 2008. Lecture Notes in Computer Science, vol 5303. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88688-4_41
Download citation
DOI: https://doi.org/10.1007/978-3-540-88688-4_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88685-3
Online ISBN: 978-3-540-88688-4
eBook Packages: Computer ScienceComputer Science (R0)