Abstract
The paper presents a method for multi-dimensional registration of two video streams. The sequences are captured by two hand-held cameras moving independently with respect to each other, both observing one object rigidly moving apart from the background. The method is based on uncalibrated Structure-from-Motion (SfM) to extract 3D models for the foreground object and the background, as well as for their relative motion. It fixes the relative scales between the scene parts within and between the videos. It also provides the registration between all partial 3D models, and the temporal synchronization between the videos. The crux is that not a single point on the foreground or background needs to be in common between both video streams. Extensions to more than two cameras and multiple foreground objects are possible.
Chapter PDF
Similar content being viewed by others
References
Avidan, S., Shashua, A.: Trajectory triangulation: 3D reconstruction of moving points from a monocular image sequence. PAMI 22(4), 348–357 (2000)
Brand, M.: Morphable 3d models from video. In: CVPR, vol. 2, pp. 456–463 (2001)
Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3d shape from image streams. In: CVPR, pp. 2690–2696 (2000)
Caspi, Y., Irani, M.: Spatio-temporal Alignment of Sequences. PAMI 24(11), 1409–1424 (2002)
Caspi, Y., Irani, M.: Alignment of Non-Overlapping Sequences. In: ICCV, pp. 76–83 (2001)
Costeira, J., Kanade, T.: A Multi-Body Factorization Method for Motion Analysis. In: ICCV, pp. 1071–1076 (1995)
Dornaika, F., Chung, R.: Self-calibration of a stereo rig without stereo correspondence. Vision Interface, pp. 264–271 (1999)
Faugeras, O.D., Hebert, M.: The representation, recognition and locating of 3D objects. International Journal of Robotics Research 5(3), 27–52 (1986)
Fitzgibbon, A.W., Zisserman, A.: Multibody structure and motion: 3-D reconstruction of independently moving objects. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1842, pp. 891–906. Springer, Heidelberg (2000)
Horaud, R., Dornaika, F.: Hand-Eye Calibration. International Journal of Robotics Research 14(3), 195–210 (1995)
Hartley, R., Zisserman, A.: Multiple View Geometry. Cambridge University Press, Cambridge (2000)
Machline, M., Zelnik-Manor, L., Irani, M.: Multi-body segmentation: Revisiting motion consistency. In: ECCV Workshop on Vision and Modeling of Dynamic Scenes, Copenhagen (2002)
Ozden, K.E., Cornelis, K., Van Eycken, L., Van Gool, L.: Reconstructing 3D independent motions using non-accidentalness. In: CVPR, vol. 1, pp. 819–825 (2004)
Ozden, K.E., Van Gool, L.: Background recognition in dynamic scenes with motion constraints. In: CVPR, vol. 1, pp. 250–255 (2005)
Ozden, K.E., Cornelis, K., Van Gool, L.: Reconstructing 3D trajectories of independently moving objects using generic constraints. In: CVIU, December 2004, pp. 453–471 (2004)
Shi, J., Malik, J.: Motion segmentation and tracking using normalized cuts. In: ICCV, pp. 1154–1160 (1998)
Shiu, Y.C., Ahmad, S.: Calibration of wrist mounted robotic sensors by solving homogenous transform equations of the form AX=XB. IEEE J. Robot. Automation 5(1), 16–19 (1989)
Sinha, S.N., Pollefeys, M.: Synchronization and Calibration of Camera Networks from Silhouettes. In: ICPR, vol. 1, pp. 116–119 (2004)
Tsai, R.Y.: A versatile camera calibration technique for high accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Automotion RA-3(4), 323–344 (1987)
Torr, P.H.S.: Geometric motion segmentation and model selection. Phil. Trans. Royal Society of London A 356(1740), 1321–1340 (1998)
Vidal, R., Soatto, S., Ma, Y., Sastry, S.: Segmentation of dynamic scenes from the multibody fundamental matrix. In: ECCV Workshop on Vision and Modeling of Dynamic Scenes, Copenhagen (2002)
Vidal, R., Soatto, S., Ma, Y., Sastry, S.: A Factorization Method for 3D Multi-body Motion Estimation and Segmentation. Technical Report (2002)
Wolf, L., Shashua, A.: On projection matrices P^k → P^2, k=3,...,6, and their applications in computer vision. In: ICCV, pp. 412–419 (2001)
Wolf, L., Zomet, A.: Sequence-to-Sequence Self Calibration. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2351, pp. 370–382. Springer, Heidelberg (2002)
Wolf, L., Zomet, A.: Correspondence-free synchronization and reconstruction in a non-rigid scene. In: ECCV Workshop on Vision and Modeling of Dynamic Scenes, Copenhagen (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ozden, K.E., Cornelis, K., Van Gool, L. (2006). Space-Time-Scale Registration of Dynamic Scene Reconstructions. In: Leonardis, A., Bischof, H., Pinz, A. (eds) Computer Vision – ECCV 2006. ECCV 2006. Lecture Notes in Computer Science, vol 3954. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11744085_14
Download citation
DOI: https://doi.org/10.1007/11744085_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-33838-3
Online ISBN: 978-3-540-33839-0
eBook Packages: Computer ScienceComputer Science (R0)