Lens Distortion Recovery for Accurate Sequential Structure and Motion Recovery

  • Kurt Cornelis
  • Marc Pollefeys
  • Luc Van Gool
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2351)


Lens distortions in off-the-shelf or wide-angle cameras block the road to high accuracy Structure and Motion Recovery (SMR) from video sequences. Neglecting lens distortions introduces a systematic error buildup which causes recovered structure and motion to bend and inhibits turntable or other loop sequences to close perfectly. Locking back onto previously reconstructed structure can become impossible due to the large drift caused by the error buildup. Bundle adjustments are widely used to perform an ultimate post-minimization of the total reprojection error. However, the initial recovered structure and motion needs to be close to optimal to avoid local minima. We found that bundle adjustments cannot remedy the error buildup caused by ignoring lens distortions. The classical approach to distortion removal involves a preliminary distortion estimation using a calibration pattern, known geometric properties of perspective projections or only 2D feature correspondences. Often the distortion is assumed constant during camera usage and removed from the images before applying SMR algorithms. However, lens distortions can change by zooming, focusing and temperature variations. Moreover, when only the video sequence is available preliminary calibration is often not an option. This paper addresses all fore-mentioned problems by sequentially recovering lens distortions together with structure and motion from video sequences without tedious pre-calibrations and allowing lens distortions to change over time. The devised algorithms are fairly simple as they only use linear least squares techniques. The unprocessed video sequence forms the only input and no severe restrictions are placed on viewed scene geometry. Therefore, the accurate recovery of structure and motion is fully automated and widely applicable. The experiments demonstrate the necessity of modeling lens distortions to achieve high accuracy in recovered structure and motion.


Structure from motion calibration lens distortion recovery high accuracy sequential 


  1. 1.
    K. Cornelis, M. Pollefeys, and L. Van Gool. Tracking based structure and motion recovery for augmented video productions. Proceedings of the ACM Symposium on Virtual Reality Software and Technology VRST2001, pages 17–24, November 2001.Google Scholar
  2. 2.
    J. Batista, H. Araújo, and A. Almeida. Iterative multi-step explicit camera calibration. IEEE Transactions on Robotics and Automation, 15(5), October 1999.Google Scholar
  3. 3.
    H. A. Beyer. Accurate calibration of ccd cameras. In Proceedings of the International Conference on Computer Vision and Pattern Recognition, Urbana Champaign, Illinois, USA, pages 96–101, June 1992.Google Scholar
  4. 4.
    F. Devernay and O. Faugeras. Automatic calibration and removal of distortion from scenes of structured environments. In Proceedings of SPIE Conference, San Diego, CA, 2567:62–72, July 1995.Google Scholar
  5. 5.
    Andrew W. Fitzgibbon. Simultaneous linear estimation of multiple view geometry and lens distortion. CVPR, Kauai, Hawaii,, I:125–132, December 2001.Google Scholar
  6. 6.
    R. Haralick, H. Joo, C. Lee, X. Zhuang, V. Vaidya, and M. Kim. Pose estimation from corresponding point data. IEEE Transactions on Systems, Man and Cybernetics, 19(6):1426–1446, November/December 1989.Google Scholar
  7. 7.
    E. Marchand and F. Chaumette. A new formulation for non-linear camera calibration using virtual visual servoing. Rapport de Recherche IRISA, No 1366, January 2001.Google Scholar
  8. 8.
    M. A. Penna. Camera calibration: A quick and easy way to determine the scale factor. in IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(12):1240–1245, December 1991.Google Scholar
  9. 9.
    G. Stein. Internal camera calibration using rotation and geometric shapes. Master’s Thesis, Massachussets Institute of Technology. Artificial Intelligence Laboratory, 1993.Google Scholar
  10. 10.
    G. Stein. Lens distortion calibration using point correspondences. In Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition, pages 143–148, June 1997.Google Scholar
  11. 11.
    Stevenson, Daniel, and M. Fleck. Nonparametric correction of distortion. TR 95-07, Comp. Sci., University of Iowa, 1995.Google Scholar
  12. 12.
    R. Swaminathan and S. Nayar. Non-metric calibration of wide angle lenses. In Proceedings of the 1998 DARPA Image Understanding Workshop, Monterey, California, November 1998.Google Scholar
  13. 13.
    R. Y. Tsai. A versatile camera calibration technique for high accuracy 3d machine vision metrology using off-the-shelf tv cameras and lenses. IEEE Journal of Robotics and Automation, 3(4):323–344, August 1987.Google Scholar
  14. 14.
    G.-Q. Wei and S. Ma. Implicit and explicit camera calibration: Theory and experiments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 16(5):469–480, May 1994.Google Scholar
  15. 15.
    J. Weng, P. Cohen, and M. Herniou. Camera calibration with distortion models and accuracy evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(10):965–980, October 1992.Google Scholar
  16. 16.
    R. Willson. Modeling and Calibration of Automated Zoom Lenses. Ph.D. thesis, Department of Electrical and Computer Engineering, Carnegie Mellon University, January 1994.Google Scholar
  17. 17.
    Z. Zhang. On the epipolar geometry between two images with lens distortion. In the Proceedings of the International Conference on Pattern Recognition (ICPR), I:407–411, August 1996.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Kurt Cornelis
    • 1
  • Marc Pollefeys
    • 1
  • Luc Van Gool
    • 1
  1. 1.K. U. Leuven, ESAT-PSILeuven-HeverleeBelgium

Personalised recommendations