Optimization of Symmetric Transfer Error for Sub-frame Video Synchronization

  • Meghna Singh
  • Irene Cheng
  • Mrinal Mandal
  • Anup Basu
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5303)


In this work we present a method to synchronize video sequences of events that are acquired via uncalibrated cameras at unknown and dynamically varying temporal offsets. Unlike existing methods that synchronize videos of similar events (i.e., videos related to each other through the motion in the scene) up to an integer alignment, we establish sub-frame video synchronization. While contemporary synchronization algorithms implement a unidirectional alignment which biases the results towards a single reference sequence, we adopt a bi-directional or symmetrical alignment approach that results in a more optimal synchronization. To this end, we propose a novel symmetric transfer error which is dynamically minimized, and reduces the propagation of error from feature extraction and spatial mapping into temporal synchronization. The advantages of our approach are validated by tests conducted on (publicly available) real and synthetic sequences. We present qualitative and quantitative comparisons with another state-of-the-art algorithm. A unique application of this work in generating high-resolution 4D MRI data from multiple low-resolution MRI scans is described.


Video Sequence Synchronization Algorithm Synthetic Sequence Feature Trajectory Frame Alignment 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Caspi, Y., Irani, M.: Spatio-temporal alignment of sequences. PAMI 24, 1409–1424 (2002)CrossRefGoogle Scholar
  2. 2.
    Caspi, Y., Simakov, D., Irani, M.: Feature-based sequence-to-sequence matching. IJCV 68, 53–64 (2006)CrossRefGoogle Scholar
  3. 3.
    Singh, M., Basu, A., Mandal, M.: Event dynamics based temporal registration. IEEE Transactions on Multimedia 9, 1004–1015 (2007)CrossRefGoogle Scholar
  4. 4.
    Lee, L., Romano, R., Stein, G.: Monitoring activities from multiple video streams: establishing a common coordinate frame. IEEE Transactions on Pattern Analysis and Machine Intelligence 22, 758–767 (2000)CrossRefGoogle Scholar
  5. 5.
    Hess, R., Fern, A.: Improved video registration using non-distinctive local image features. In: CVPR, pp. 1–8 (2007)Google Scholar
  6. 6.
    Rao, C., Gritai, A., Shah, M., Mahmood, T.F.S.: View-invariant alignment and matching of video sequences. In: Proc. ICCV, pp. 939–945 (2003)Google Scholar
  7. 7.
    Rao, C., Shah, M., Syeda-Mahmood, T.: Invariance in motion analysis of videos. In: Proceedings of the ACM Int. Conf. on Multimedia, pp. 518–527 (2003)Google Scholar
  8. 8.
    Keogh, E., Pazzani, M.: Derivative dynamic time warping. In: SIAM International Conference on Data Mining (2001)Google Scholar
  9. 9.
    Dai, C., Zheng, Y., Li, X.: Subframe video synchronization via 3d phase correlation. In: ICIP, pp. 501–504 (2006)Google Scholar
  10. 10.
    Tuytelaars, T., VanGool, L.J.: Synchronizing video sequences. In: CVPR, pp. I: 762–768 (2004)Google Scholar
  11. 11.
    Tresadern, P., Reid, I.: Synchronizing image sequences of non-rigid objects. In: Proc. BMVC, vol. 2, pp. 629–638 (2003)Google Scholar
  12. 12.
    Carceroni, R.L., Padua, F.L.C., Santos, G.A.M.R., Kutulakos, K.N.: Linear sequence-to-sequence alignment. CVPR, I: 746–753 (2004)Google Scholar
  13. 13.
    Lei, C., Yang, Y.H.: Tri-focal tensor-based multiple video synchronization with subframe optimization. IEEE Trans. on IP 15, 2473–2480 (2006)Google Scholar
  14. 14.
    Wolf, L., Zomet, A.: Wide baseline matching between unsynchronized video sequences. IJCV 68, 43–52 (2006)CrossRefGoogle Scholar
  15. 15.
    Pooley, D.W., Brooks, M.J., van den Hengel, A.J., Chojnacki, W.: A voting scheme for estimating the synchrony of moving-camera videos. In: ICIP, vol. 1, pp. I–413–16 (September 14-17, 2003)Google Scholar
  16. 16.
    Perperidis, D., Mohiaddin, R., Rueckert, D.: Spatio-temporal free-form registration of cardiac mr image sequences. Medical Image Analysis 9, 441–456 (2005)CrossRefGoogle Scholar
  17. 17.
    Giese, M.A., Poggio, T.: Morphable models for the analysis and synthesis of complex motion patterns. Int. J. Comput. Vision 38, 59–73 (2000)CrossRefzbMATHGoogle Scholar
  18. 18.
    Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Gool, L.V.: A comparison of affine region detectors. Int. J. Comput. Vision 65, 43–72 (2005)CrossRefGoogle Scholar
  19. 19.
    Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proceedings of British Machine Vision Conference, September 2002, vol. I, pp. 384–393 (2002)Google Scholar
  20. 20.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)CrossRefzbMATHGoogle Scholar
  21. 21.

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Meghna Singh
    • 1
  • Irene Cheng
    • 2
  • Mrinal Mandal
    • 1
  • Anup Basu
    • 2
  1. 1.Department of Electrical and Computer EngineeringCanada
  2. 2.Department of Computing ScienceUniversity of Alberta, EdmontonAlbertaCanada

Personalised recommendations