Space-Time-Scale Registration of Dynamic Scene Reconstructions

  • Kemal E. Ozden
  • Kurt Cornelis
  • Luc Van Gool
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3954)


The paper presents a method for multi-dimensional registration of two video streams. The sequences are captured by two hand-held cameras moving independently with respect to each other, both observing one object rigidly moving apart from the background. The method is based on uncalibrated Structure-from-Motion (SfM) to extract 3D models for the foreground object and the background, as well as for their relative motion. It fixes the relative scales between the scene parts within and between the videos. It also provides the registration between all partial 3D models, and the temporal synchronization between the videos. The crux is that not a single point on the foreground or background needs to be in common between both video streams. Extensions to more than two cameras and multiple foreground objects are possible.


  1. 1.
    Avidan, S., Shashua, A.: Trajectory triangulation: 3D reconstruction of moving points from a monocular image sequence. PAMI 22(4), 348–357 (2000)CrossRefGoogle Scholar
  2. 2.
    Brand, M.: Morphable 3d models from video. In: CVPR, vol. 2, pp. 456–463 (2001)Google Scholar
  3. 3.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3d shape from image streams. In: CVPR, pp. 2690–2696 (2000)Google Scholar
  4. 4.
    Caspi, Y., Irani, M.: Spatio-temporal Alignment of Sequences. PAMI 24(11), 1409–1424 (2002)CrossRefGoogle Scholar
  5. 5.
    Caspi, Y., Irani, M.: Alignment of Non-Overlapping Sequences. In: ICCV, pp. 76–83 (2001)Google Scholar
  6. 6.
    Costeira, J., Kanade, T.: A Multi-Body Factorization Method for Motion Analysis. In: ICCV, pp. 1071–1076 (1995)Google Scholar
  7. 7.
    Dornaika, F., Chung, R.: Self-calibration of a stereo rig without stereo correspondence. Vision Interface, pp. 264–271 (1999)Google Scholar
  8. 8.
    Faugeras, O.D., Hebert, M.: The representation, recognition and locating of 3D objects. International Journal of Robotics Research 5(3), 27–52 (1986)CrossRefGoogle Scholar
  9. 9.
    Fitzgibbon, A.W., Zisserman, A.: Multibody structure and motion: 3-D reconstruction of independently moving objects. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 891–906. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  10. 10.
    Horaud, R., Dornaika, F.: Hand-Eye Calibration. International Journal of Robotics Research 14(3), 195–210 (1995)CrossRefGoogle Scholar
  11. 11.
    Hartley, R., Zisserman, A.: Multiple View Geometry. Cambridge University Press, Cambridge (2000)zbMATHGoogle Scholar
  12. 12.
    Machline, M., Zelnik-Manor, L., Irani, M.: Multi-body segmentation: Revisiting motion consistency. In: ECCV Workshop on Vision and Modeling of Dynamic Scenes, Copenhagen (2002)Google Scholar
  13. 13.
    Ozden, K.E., Cornelis, K., Van Eycken, L., Van Gool, L.: Reconstructing 3D independent motions using non-accidentalness. In: CVPR, vol. 1, pp. 819–825 (2004)Google Scholar
  14. 14.
    Ozden, K.E., Van Gool, L.: Background recognition in dynamic scenes with motion constraints. In: CVPR, vol. 1, pp. 250–255 (2005)Google Scholar
  15. 15.
    Ozden, K.E., Cornelis, K., Van Gool, L.: Reconstructing 3D trajectories of independently moving objects using generic constraints. In: CVIU, December 2004, pp. 453–471 (2004)Google Scholar
  16. 16.
    Shi, J., Malik, J.: Motion segmentation and tracking using normalized cuts. In: ICCV, pp. 1154–1160 (1998)Google Scholar
  17. 17.
    Shiu, Y.C., Ahmad, S.: Calibration of wrist mounted robotic sensors by solving homogenous transform equations of the form AX=XB. IEEE J. Robot. Automation 5(1), 16–19 (1989)CrossRefGoogle Scholar
  18. 18.
    Sinha, S.N., Pollefeys, M.: Synchronization and Calibration of Camera Networks from Silhouettes. In: ICPR, vol. 1, pp. 116–119 (2004)Google Scholar
  19. 19.
    Tsai, R.Y.: A versatile camera calibration technique for high accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Automotion RA-3(4), 323–344 (1987)CrossRefGoogle Scholar
  20. 20.
    Torr, P.H.S.: Geometric motion segmentation and model selection. Phil. Trans. Royal Society of London A 356(1740), 1321–1340 (1998)MathSciNetCrossRefzbMATHGoogle Scholar
  21. 21.
    Vidal, R., Soatto, S., Ma, Y., Sastry, S.: Segmentation of dynamic scenes from the multibody fundamental matrix. In: ECCV Workshop on Vision and Modeling of Dynamic Scenes, Copenhagen (2002)Google Scholar
  22. 22.
    Vidal, R., Soatto, S., Ma, Y., Sastry, S.: A Factorization Method for 3D Multi-body Motion Estimation and Segmentation. Technical Report (2002)Google Scholar
  23. 23.
    Wolf, L., Shashua, A.: On projection matrices P^k → P^2, k=3,...,6, and their applications in computer vision. In: ICCV, pp. 412–419 (2001)Google Scholar
  24. 24.
    Wolf, L., Zomet, A.: Sequence-to-Sequence Self Calibration. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2351, pp. 370–382. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  25. 25.
    Wolf, L., Zomet, A.: Correspondence-free synchronization and reconstruction in a non-rigid scene. In: ECCV Workshop on Vision and Modeling of Dynamic Scenes, Copenhagen (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Kemal E. Ozden
    • 1
  • Kurt Cornelis
    • 1
  • Luc Van Gool
    • 1
    • 2
  1. 1.ESAT/PSI, Computer Vision Lab.K.U. LeuvenBelgium
  2. 2.BIWIETHZurichSwitzerland

Personalised recommendations