Damped Techniques for the Limited Memory BFGS Method for Large-Scale Optimization

  • Mehiddin Al-Baali
  • Lucio Grandinetti
  • Ornella Pisacane


This paper is aimed to extend a certain damped technique, suitable for the Broyden–Fletcher–Goldfarb–Shanno (BFGS) method, to the limited memory BFGS method in the case of the large-scale unconstrained optimization. It is shown that the proposed technique maintains the global convergence property on uniformly convex functions for the limited memory BFGS method. Some numerical results are described to illustrate the important role of the damped technique. Since this technique enforces safely the positive definiteness property of the BFGS update for any value of the steplength, we also consider only the first Wolfe–Powell condition on the steplength. Then, as for the backtracking framework, only one gradient evaluation is performed on each iteration. It is reported that the proposed damped methods work much better than the limited memory BFGS method in several cases.


Large-scale optimization The limited memory BFGS method Damped technique Line search framework 


  1. 1.
    Fletcher, R.: Practical Methods of Optimization, 2nd edn. Wiley, Chichester (1987). Reprinted in 2000 MATHGoogle Scholar
  2. 2.
    Al-Baali, M.: Convergence analysis of a class of damped quasi-Newton methods for nonlinear optimization. Research Report, DOMAS 11/2, Sultan Qaboos University, Oman (2011) Google Scholar
  3. 3.
    Powell, M.J.D.: Algorithms for nonlinear constraints that use Lagrange functions. Math. Program. 14, 224–248 (1978) MATHCrossRefGoogle Scholar
  4. 4.
    Nocedal, J.: Updating quasi-Newton matrices with limited storage. Math. Comput. 35, 773–782 (1980) MATHMathSciNetCrossRefGoogle Scholar
  5. 5.
    Al-Baali, M., Grandinetti, L.: On practical modifications of the quasi-Newton BFGS method. Adv. Model. Optim. 11, 63–76 (2009) MATHMathSciNetGoogle Scholar
  6. 6.
    Al-Baali, M.: Quasi-Wolfe conditions for quasi-Newton methods for large-scale optimization. In: 40th Workshop on Large Scale Nonlinear Optimization, Erice, Italy, June 22–July 1 (2004) Google Scholar
  7. 7.
    Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Program. 45, 503–528 (1989) MATHMathSciNetCrossRefGoogle Scholar
  8. 8.
    Kelley, C.T.: Iterative Methods for Optimization, 2nd edn. Siam, Philadelphia (1999) MATHCrossRefGoogle Scholar
  9. 9.
    Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, London (1999) MATHCrossRefGoogle Scholar
  10. 10.
    Byrd, R.H., Nocedal, J.: A tool for the analysis of quasi-Newton methods with application to unconstrained minimization. SIAM J. Numer. Anal. 26, 727–739 (1989) MATHMathSciNetCrossRefGoogle Scholar
  11. 11.
    Al-Baali, M.: Variational quasi-Newton methods for unconstrained optimization. J. Optim. Theory Appl. 77, 127–143 (1993) MATHMathSciNetCrossRefGoogle Scholar
  12. 12.
    Al-Baali, M.: Improved Hessian approximations for the limited memory BFGS method. Numer. Algorithms 22, 99–112 (1999) MATHMathSciNetCrossRefGoogle Scholar
  13. 13.
    Nash, S.G., Nocedal, J.: A numerical study of the limited memory BFGS method and the truncated-Newton method for large scale optimization. SIAM J. Optim. 1, 358–372 (1991) MATHMathSciNetCrossRefGoogle Scholar
  14. 14.
    Moré, J.J., Garbow, B.S., Hillstrom, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7, 17–41 (1981) MATHCrossRefGoogle Scholar
  15. 15.
    Toint, P.L.: Test problems for partially separable optimization and results for the routine PSPMIN. Technical Report No. 83/4, Department of Mathematics, Faculties University de Namur, Namur, Belgium (1983) Google Scholar
  16. 16.
    Grandinetti, L.: Some investigations in a new algorithm for nonlinear optimization based on conic models of the objective function. J. Optim. Theory Appl. 43, 1–21 (1984) MATHMathSciNetGoogle Scholar
  17. 17.
    Siegel, D.: Implementing and modifying Broyden class updates for large scale optimization. Technical Report NA12, Department of Applied Mathematics and Theoretical Physics, Cambridge University, England (1992) Google Scholar
  18. 18.
    Toint, Ph.L.: Some numerical results using a sparse matrix updating formula in unconstrained optimization. Math. Comput. 32, 839–851 (1978) MATHMathSciNetCrossRefGoogle Scholar
  19. 19.
    Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. Program. 91, 201–213 (2002) MATHMathSciNetCrossRefGoogle Scholar
  20. 20.
    Malmedy, V., Toint, Ph.L.: Approximating Hessians in unconstrained optimization arising from discretized problems. Comput. Optim. Appl. 50, 1–22 (2011) MATHMathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  • Mehiddin Al-Baali
    • 1
  • Lucio Grandinetti
    • 2
  • Ornella Pisacane
    • 3
  1. 1.Department of Mathematics and StatisticsSultan Qaboos UniversityMuscatOman
  2. 2.Department of Computer Engineering, Electronics and SystemsUniversity of CalabriaRendeItaly
  3. 3.Information Engineering DepartmentPolytechnic University of MarcheAnconaItaly

Personalised recommendations