Bundle Adjustment in the Large

  • Sameer Agarwal
  • Noah Snavely
  • Steven M. Seitz
  • Richard Szeliski
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6312)


We present the design and implementation of a new inexact Newton type algorithm for solving large-scale bundle adjustment problems with tens of thousands of images. We explore the use of Conjugate Gradients for calculating the Newton step and its performance as a function of some simple and computationally efficient preconditioners. We show that the common Schur complement trick is not limited to factorization-based methods and that it can be interpreted as a form of preconditioning. Using photos from a street-side dataset and several community photo collections, we generate a variety of bundle adjustment problems and use them to evaluate the performance of six different bundle adjustment algorithms. Our experiments show that truncated Newton methods, when paired with relatively simple preconditioners, offer state of the art performance for large-scale bundle adjustment. The code, test problems and detailed performance data are available at .


Conjugate Gradient Levenberg Marquardt Newton Step Bundle Adjustment Leaf Image 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Snavely, N., Seitz, S.M., Szeliski, R.: Skeletal graphs for efficient structure from motion. In: CVPR, pp. 1–8 (2008)Google Scholar
  2. 2.
    Li, X., Wu, C., Zach, C., Lazebnik, S., Frahm, J.: Modeling and recognition of landmark image collections using iconic scene graphs. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part I. LNCS, vol. 5302, pp. 427–440. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  3. 3.
    Agarwal, S., Snavely, N., Simon, I., Seitz, S.M., Szeliski, R.: Building Rome in a day. In: ICCV (2009)Google Scholar
  4. 4.
    Triggs, B., McLauchlan, P., Hartley, R.I., Fitzgibbon, A.: Bundle Adjustment - A modern synthesis. In: Vision Algorithms 1999, pp. 298–372 (1999)Google Scholar
  5. 5.
    Lourakis, M., Argyros, A.A.: SBA: A software package for generic sparse bundle adjustment. TOMS 36, 2 (2009)CrossRefGoogle Scholar
  6. 6.
    Mouragnon, E., Lhuillier, M., Dhome, M., Dekeyser, F., Sayd, P.: Generic and real-time structure from motion using local bundle adjustment. Image and Vision Computing 27, 1178–1193 (2009)CrossRefGoogle Scholar
  7. 7.
    Steedly, D., Essa, I.: Propagation of innovative information in non-linear least-squares structure from motion. In: ICCV, pp. 223–229 (2001)Google Scholar
  8. 8.
    Ni, K., Steedly, D., Dellaert, F.: Out-of-core bundle adjustment for large-scale 3d reconstruction. In: ICCV (2007)Google Scholar
  9. 9.
    Steedly, D., Essa, I., Dellaert, F.: Spectral partitioning for structure from motion. In: ICCV, pp. 996–1003 (2003)Google Scholar
  10. 10.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)Google Scholar
  11. 11.
    Nocedal, J., Wright, S.: Numerical optimization. Springer, Heidelberg (2000)Google Scholar
  12. 12.
    More, J.: The Levenberg-Marquardt algorithm: implementation and theory. Lecture Notes in Math. 630, 105–116 (1977)CrossRefMathSciNetGoogle Scholar
  13. 13.
    Björck, A.: Numerical methods for least squares problems. SIAM, Philadelphia (1996)zbMATHGoogle Scholar
  14. 14.
    Madsen, K., Nielsen, H., Tingleff, O.: Methods for non-linear least squares problems (2004)Google Scholar
  15. 15.
    Brown, D.C.: A solution to the general problem of multiple station analytical stereotriangulation. Technical Report 43, Patrick Airforce Base, Florida (1958)Google Scholar
  16. 16.
    Trefethen, L., Bau, D.: Numerical linear algebra. SIAM, Philadelphia (1997)zbMATHGoogle Scholar
  17. 17.
    Chen, Y., Davis, T., Hager, W., Rajamanickam, S.: Algorithm 887: CHOLMOD, Supernodal Sparse Cholesky Factorization and Update/Downdate. TOMS 35 (2008)Google Scholar
  18. 18.
    Wright, S.J., Holt, J.N.: An inexact Levenberg-Marquardt method for large sparse nonlinear least squares. J. Austral. Math. Soc. Ser. B 26, 387–403 (1985)zbMATHCrossRefMathSciNetGoogle Scholar
  19. 19.
    Saad, Y.: Iterative methods for sparse linear systems. SIAM, Philadelphia (2003)zbMATHGoogle Scholar
  20. 20.
    Elsner, L.: A note on optimal block-scaling of matrices. Numer. Math. 44, 127–128 (1984)zbMATHCrossRefMathSciNetGoogle Scholar
  21. 21.
    Mandel, J.: On block diagonal and Schur complement preconditioning. Numer. Math. 58, 79–93 (1990)zbMATHCrossRefMathSciNetGoogle Scholar
  22. 22.
    Mathew, T.: Domain decomposition methods for the numerical solution of partial differential equations. Springer, Heidelberg (2008)zbMATHCrossRefGoogle Scholar
  23. 23.
    Dennis Jr., J., Gay, D., Welsch, R.: Algorithm 573: NL2SOLan adaptive nonlinear least-squares algorithm [E4]. TOMS 7, 369–383 (1981)CrossRefGoogle Scholar
  24. 24.
    Goto, K., Van De Geijn, R.: High-performance implementation of the level-3 blas. TOMS 35, 1–14 (2008)zbMATHCrossRefGoogle Scholar
  25. 25.
    Snavely, N., Seitz, S.M., Szeliski, R.: Photo Tourism: Exploring photo collections in 3D. TOG 25, 835–846 (2006)CrossRefGoogle Scholar
  26. 26.
    Engels, C., Stewenius, H., Nister, D.: Bundle adjustment rules. Photogrammetric Computer Vision 2 (2006)Google Scholar
  27. 27.
    Lourakis, M., Argyros, A.: Is Levenberg-Marquardt the most efficient optimization algorithm for implementing bundle adjustment. In: ICCV, pp. 1526–1531 (2005)Google Scholar
  28. 28.
    Byröd, M., Åström, K., Lund, S.: Bundle adjustment using conjugate gradients with multiscale preconditioning (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Sameer Agarwal
    • 1
  • Noah Snavely
    • 2
  • Steven M. Seitz
    • 3
  • Richard Szeliski
    • 4
  1. 1.Google Inc 
  2. 2.Cornell University 
  3. 3.Google Inc. & University of Washington 
  4. 4.Microsoft Research 

Personalised recommendations