A comparative study of several general convergence conditions for algorithms modeled by point-to-set maps

  • S. Tishyadhigama
  • E. Polak
  • R. Klessig
Part of the Mathematical Programming Studies book series (MATHPROGRAMM, volume 10)


A general structure is established that allows the comparison of various conditions that are sufficient for convergence of algorithms that can be modeled as the recursive application of a point-to-set map. This structure is used to compare several earlier sufficient conditions as well as three new sets of sufficient conditions. One of the new sets of conditions is shown to be the most general in that all other sets of conditions imply this new set. This new set of conditions is also extended to the case where the point-to-set map can change from iteration to iteration.

Key words

Optimization Algorithms Convergence Conditions Point-to-set Maps Nonlinear Programming Comparative Study 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    J.W. Daniel, “Convergent step sizes for gradient-like feasible direction algorithms for constrained optimization,” in J.B. Rosen, O.L. Mangasarian and K. Ritter, eds., Nonlinear programming (Academic Press, New York, 1970) pp. 245–274.Google Scholar
  2. [2]
    J. Dubois, “Theorems of convergence for improved nonlinear programming algorithms”, Operations Research 21 (1) (1973) 328–332.zbMATHCrossRefMathSciNetGoogle Scholar
  3. [3]
    B.C. Eaves and W.I. Zangwill, “Generalized cutting plane algorithms”, Siam Journal on Control 9 (4) (1971) 529–542.CrossRefMathSciNetGoogle Scholar
  4. [4]
    R.M. Elkin, “Convergence theorems for Gauss-Seidel and other minimization algorithms”, Computer Science Center Tech. Rept. 68-59, University of Maryland, College Park, MD (1968).Google Scholar
  5. [5]
    W.W. Hogan, “Point-to-set maps in mathematical programming”, SIAM Review 15 (3) (1973) 591–603.zbMATHCrossRefMathSciNetGoogle Scholar
  6. [6]
    W.W. Hogan, “Applications of general convergence theory for outer approximation algorithms”, Mathematical Programming 5 (1973) 151–168.zbMATHCrossRefMathSciNetGoogle Scholar
  7. [7]
    P. Huard, “Optimization algorithms and point-to-set maps”, Mathematical Programming 8 (3) (1975) 308–331.zbMATHCrossRefMathSciNetGoogle Scholar
  8. [8]
    R. Klessig, “A general theory of convergence for constrained optimization algorithms that use antizigzagging provisions”, SIAM Journal on Control 12 (4) (1974) 598–608.zbMATHCrossRefMathSciNetGoogle Scholar
  9. [9]
    R. Klessig and E. Polak, “An adaptive precision gradient method for optimal control”, SIAM Journal on Control 11 (1) (1973) 80–93.zbMATHCrossRefMathSciNetGoogle Scholar
  10. [10]
    D.G. Luenberger, Introduction to linear and nonlinear programming (Addison-Wesley, Reading, MA, 1973).zbMATHGoogle Scholar
  11. [11]
    M.L. Lenard, “Practical convergence conditions for unconstrained optimization”, Mathematical Programming 4 (1973) 309–323.zbMATHCrossRefMathSciNetGoogle Scholar
  12. [12]
    G.G.L. Meyer and E. Polak, “Abstract models for the synthesis of optimization algorithms”, SIAM Journal on Control 9 (1971) 547–560.zbMATHCrossRefMathSciNetGoogle Scholar
  13. [13]
    G.G.L. Meyer, “Convergence conditions for a type of algorithm model”, Tech. Rept. 75-14, The Johns Hopkins University, Baltimore, MD (1975).Google Scholar
  14. [14]
    G.G.L. Meyer, “Algorithm model for penalty-type iterative procedures”, Journal of Computer and Systems Sciences 9 (1974) 20–30.zbMATHGoogle Scholar
  15. [15]
    R.R. Meyer, “Sufficient conditions for the convergence of monotonic mathematical programming algorithms”, Journal of Computer and Systems Sciences 12 (1976) 108–121.zbMATHGoogle Scholar
  16. [16]
    R.R. Meyer, “A comparison of the forcing functions and point-to-set mapping approaches to convergence analysis”, SIAM Journal on Control 15 (4) (1977) 699–715.zbMATHCrossRefMathSciNetGoogle Scholar
  17. [17]
    R.R. Meyer, “A convergence theory for a class of anti-jamming strategies”, Tech. Rept. 1481, Math. Research Center, University of Wisconsin, Madison, WI. (1974).Google Scholar
  18. [18]
    H. Mukai and E. Polak, “On the use of approximations in algorithms for optimization problems with equality and inequality constraints”, SIAM Journal on Numerical Analysis 15 (4) (1978) 674–693.zbMATHCrossRefMathSciNetGoogle Scholar
  19. [19]
    J.M. Ortega and W.C. Rheinboldt, Iterative solution of nonlinear equations in several variables (Academic Press, New York, 1970).zbMATHGoogle Scholar
  20. [20]
    J.M. Ortega and W.C. Rheinboldt, “A general convergence result for unconstrained minimization methods”, SIAM Journal on Numerical Analysis 9 (1) (1972) 40–43.zbMATHCrossRefMathSciNetGoogle Scholar
  21. [21]
    E. Polak, “On the convergence of optimization algorithms”, Revue Française d’Automatique, Informatique et Recherche Opérationnelle 16(R1) (1969) 17–34.MathSciNetGoogle Scholar
  22. [21a]
    E. Polak, Computational methods in optimization: a unified approach (Academic Press, New York, 1971).Google Scholar
  23. [22]
    E. Polak, “On the implementation of conceptual algorithms”, in J.B. Rosen, O.L. Mangasarian and K. Ritter, eds., Nonlinear programming (Academic Press, New York, 1970) pp. 275–291.Google Scholar
  24. [23]
    E. Polak, R.W.H. Sargent and D.J. Sebastian, “On the convergence of sequential minimization algorithms”, Journal of Optimization Theory and Applications 14 (1974) 439–442.zbMATHCrossRefMathSciNetGoogle Scholar
  25. [24]
    B.T. Polyak, “Gradient methods for the minimization of functionals”, U.S.S.R. Computational Mathematics and Mathematical Physics 3 (1963) 864–878.CrossRefGoogle Scholar
  26. [25]
    S.W. Rauch, “A convergence theory for a class of nonlinear programming problems”, SIAM Journal on Numerical Analysis 10 (1) (1973) 207–228.zbMATHCrossRefMathSciNetGoogle Scholar
  27. [26]
    S. Tishyadhigama, “General convergence theorems: Their relationships and applications”, Ph.D. Thesis, Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA, (1977).Google Scholar
  28. [27]
    D.M. Topkis and A.F. Veinott, “On the convergence of some feasible direction algorithms for nonlinear programming”, SIAM Journal on Control 5 (2) (1967) 268–279.zbMATHCrossRefMathSciNetGoogle Scholar
  29. [28]
    P. Wolfe, “Convergence conditions for ascene methods”, SIAM Review 11 (2) (1969) 226–235.zbMATHCrossRefMathSciNetGoogle Scholar
  30. [29]
    W.I. Zangwill, Nonlinear programming: a unified approach (Prentice-Hall, Englewood Cliffs, NJ., 1969).zbMATHGoogle Scholar
  31. [30]
    W.I. Zangwill, “Convergence conditions for nonlinear programming algorithms”, Management Science 16 (1) (1969) 1–13.zbMATHCrossRefMathSciNetGoogle Scholar
  32. [31]
    J.L. Kelley, General topology (Van Nostrand, Princeton, NJ., 1955).zbMATHGoogle Scholar
  33. [32]
    L. Armijo, “Minimization of functions having continuous partial derivatives”, Pacific Journal of Mathematics 16 (1966) 1–3.zbMATHMathSciNetGoogle Scholar

Copyright information

© The Mathematical Programming Society 1979

Authors and Affiliations

  • S. Tishyadhigama
    • 1
  • E. Polak
    • 1
  • R. Klessig
    • 2
  1. 1.University of CaliforniaBerkeleyUSA
  2. 2.Bell LaboratoriesHolmdelUSA

Personalised recommendations