Abstract
Lens distortions in off-the-shelf or wide-angle cameras block the road to high accuracy Structure and Motion Recovery (SMR) from video sequences. Neglecting lens distortions introduces a systematic error buildup which causes recovered structure and motion to bend and inhibits turntable or other loop sequences to close perfectly. Locking back onto previously reconstructed structure can become impossible due to the large drift caused by the error buildup. Bundle adjustments are widely used to perform an ultimate post-minimization of the total reprojection error. However, the initial recovered structure and motion needs to be close to optimal to avoid local minima. We found that bundle adjustments cannot remedy the error buildup caused by ignoring lens distortions. The classical approach to distortion removal involves a preliminary distortion estimation using a calibration pattern, known geometric properties of perspective projections or only 2D feature correspondences. Often the distortion is assumed constant during camera usage and removed from the images before applying SMR algorithms. However, lens distortions can change by zooming, focusing and temperature variations. Moreover, when only the video sequence is available preliminary calibration is often not an option. This paper addresses all fore-mentioned problems by sequentially recovering lens distortions together with structure and motion from video sequences without tedious pre-calibrations and allowing lens distortions to change over time. The devised algorithms are fairly simple as they only use linear least squares techniques. The unprocessed video sequence forms the only input and no severe restrictions are placed on viewed scene geometry. Therefore, the accurate recovery of structure and motion is fully automated and widely applicable. The experiments demonstrate the necessity of modeling lens distortions to achieve high accuracy in recovered structure and motion.
Kurt Cornelis and Marc Pollefeys are respectively research assistant and postdoctoral fellow of the Fund for Scientific Research - Flanders(Belgium)(F.W.O. - Vlaanderen)
Chapter PDF
Similar content being viewed by others
References
K. Cornelis, M. Pollefeys, and L. Van Gool. Tracking based structure and motion recovery for augmented video productions. Proceedings of the ACM Symposium on Virtual Reality Software and Technology VRST2001, pages 17–24, November 2001.
J. Batista, H. Araújo, and A. Almeida. Iterative multi-step explicit camera calibration. IEEE Transactions on Robotics and Automation, 15(5), October 1999.
H. A. Beyer. Accurate calibration of ccd cameras. In Proceedings of the International Conference on Computer Vision and Pattern Recognition, Urbana Champaign, Illinois, USA, pages 96–101, June 1992.
F. Devernay and O. Faugeras. Automatic calibration and removal of distortion from scenes of structured environments. In Proceedings of SPIE Conference, San Diego, CA, 2567:62–72, July 1995.
Andrew W. Fitzgibbon. Simultaneous linear estimation of multiple view geometry and lens distortion. CVPR, Kauai, Hawaii,, I:125–132, December 2001.
R. Haralick, H. Joo, C. Lee, X. Zhuang, V. Vaidya, and M. Kim. Pose estimation from corresponding point data. IEEE Transactions on Systems, Man and Cybernetics, 19(6):1426–1446, November/December 1989.
E. Marchand and F. Chaumette. A new formulation for non-linear camera calibration using virtual visual servoing. Rapport de Recherche IRISA, No 1366, January 2001.
M. A. Penna. Camera calibration: A quick and easy way to determine the scale factor. in IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(12):1240–1245, December 1991.
G. Stein. Internal camera calibration using rotation and geometric shapes. Master’s Thesis, Massachussets Institute of Technology. Artificial Intelligence Laboratory, 1993.
G. Stein. Lens distortion calibration using point correspondences. In Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition, pages 143–148, June 1997.
Stevenson, Daniel, and M. Fleck. Nonparametric correction of distortion. TR 95-07, Comp. Sci., University of Iowa, 1995.
R. Swaminathan and S. Nayar. Non-metric calibration of wide angle lenses. In Proceedings of the 1998 DARPA Image Understanding Workshop, Monterey, California, November 1998.
R. Y. Tsai. A versatile camera calibration technique for high accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses. IEEE Journal of Robotics and Automation, 3(4):323–344, August 1987.
G.-Q. Wei and S. Ma. Implicit and explicit camera calibration: Theory and experiments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(5):469–480, May 1994.
J. Weng, P. Cohen, and M. Herniou. Camera calibration with distortion models and accuracy evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(10):965–980, October 1992.
R. Willson. Modeling and Calibration of Automated Zoom Lenses. Ph.D. thesis, Department of Electrical and Computer Engineering, Carnegie Mellon University, January 1994.
Z. Zhang. On the epipolar geometry between two images with lens distortion. In the Proceedings of the International Conference on Pattern Recognition (ICPR), I:407–411, August 1996.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cornelis, K., Pollefeys, M., Van Gool, L. (2002). Lens Distortion Recovery for Accurate Sequential Structure and Motion Recovery. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds) Computer Vision — ECCV 2002. ECCV 2002. Lecture Notes in Computer Science, vol 2351. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-47967-8_13
Download citation
DOI: https://doi.org/10.1007/3-540-47967-8_13
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-43744-4
Online ISBN: 978-3-540-47967-3
eBook Packages: Springer Book Archive