Advertisement

Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

A cutting-plane algorithm with linear and geometric rates of convergence

  • 162 Accesses

  • 16 Citations

Abstract

This paper presents a cutting-plane algorithm for nonlinear programming which, under suitable conditions, exhibits a linear or geometric global rate of convergence. Other known rates of convergence for cutting-plane algorithms are no better than arithmetic for problems not satisfying a Haar condition. The feature responsible for this improved rate of convergence is the addition at each iteration of a new cut for each constraint, rather than adding only one new cut corresponding to the most violated constraint as is typically the case. Certain cuts can be dropped at each iteration, and there is a uniform upper bound on the number of old cuts retained. Geometric convergence is maintained if the subproblems at each iteration are approximated, rather than solved exactly, so the algorithm is implementable. The algorithm is flexible with respect to the point used to generate new cuts.

This is a preview of subscription content, log in to check access.

References

  1. 1.

    Kelley, J. E., Jr.,The Cutting-Plane Method for Solving Convex Programs, Journal of the Society for Industrial and Applied Mathematics, Vol. 8, pp. 703–712, 1960.

  2. 2.

    Dinkel, J. J., Elliott, W. H., andKochenberger, G. A.,Computational Aspects of Cutting-Plane Algorithms for Geometric Programming Problems, Mathematical Programming, Vol. 13, pp. 200–220, 1977.

  3. 3.

    Wolfe, P.,Convergence Theory in Nonlinear Programming, Integer and Nonlinear Programming, Edited by J. Abadie, North-Holland Publishing Company, Amsterdam, Holland, 1970.

  4. 4.

    Veinott, A. F., Jr.,The Supporting Hyperplane Method for Unimodal Programming, Operations Research, Vol. 15, pp. 147–152, 1967.

  5. 5.

    Topkis, D. M.,Cutting-Plane Methods without Nested Constraint Sets, Operations Research, Vol. 18, pp. 404–413, 1970.

  6. 6.

    Topkis, D. M.,A Note on Cutting-Plane Methods without Nested Constraint Sets, Operations Research, Vol. 18, pp. 1216–1220, 1970.

  7. 7.

    Wolfe, P.,Accelerating the Cutting Plane Method for Nonlinear Programming, Journal of the Society for Industrial and Applied Mathematics, Vol. 9, pp. 481–488, 1961.

  8. 8.

    Kuhn, H. W., andTucker, A. W.,Nonlinear Programming, Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, Edited by J. Neyman, University of California Press, Berkeley, California, 1951.

  9. 9.

    Cheney, E. W., andGoldstein, A. A.,Newton's Method for Convex Programming and Tchebycheff Approximation, Numerische Mathematik, Vol. 1, pp. 253–268, 1959.

  10. 10.

    Eaves, B. C., andZangwill, W. I.,Generalized Cutting Plane Algorithms, SIAM Journal on Control, Vol. 9, pp. 529–542, 1971.

  11. 11.

    Oettli, W.,Solving Optimization Problems with Many Constraints by a Sequence of Subproblems Containing Only Two Constraints, Mathematische Nachrichten, Vol. 71, pp. 143–145, 1976.

  12. 12.

    Blankenship, J. W., andFalk, J. E.,Infinitely Constrained Optimization Problems, Journal of Optimization Theory and Applications, Vol. 19, pp. 261–281, 1976.

  13. 13.

    Gehner, K. R.,Optimization Problems with an Infinite Number of Constraints and Applications to Constrained Approximation Problems, University of Wisconsin, PhD Thesis, 1971.

  14. 14.

    Afriat, S. N.,The Progrssive Support Method for Convex Programming, SIAM Journal on Numerical Analysis, Vol. 7, pp. 447–457, 1970.

  15. 15.

    Blum, E., andOettli, W.,Mathematische Optimierung, Springer-Verlag, Berlin, Germany, 1975.

  16. 16.

    Elzinga, J., andMoore, T. G.,A Central Cutting Plane Algorithm for the Convex Programming Problem, Mathematical Programming, Vol. 8, pp. 134–145, 1975.

  17. 17.

    Gonzaga, C., andPolak, E.,On Constraint Dropping Schemes and Optimality Functions for a Class of Outer Approximations Algorithms, SIAM Journal on Control and Optimization, Vol. 17, pp. 477–493, 1979.

  18. 18.

    Hogan, W. W.,Applications of a General Convergence Theory for Outer Approximation Algorithms, Mathematical Programming, Vol. 5, pp. 151–168, 1973.

  19. 19.

    Ortega, J. M., andRheinboldt, W. C.,Iterative Solution of Nonlinear Equations in Several Variables, Academic Press, New York, New York, 1970.

  20. 20.

    Kapur, K. C.,On Cutoff Optimization Methods in Infinite-Dimensional Spaces and Applications, Journal of Optimization Theory and Applications, Vol. 12, pp. 16–31, 1973.

  21. 21.

    Levitin, E. S., andPolyak, B. T.,Constrained Minimization Methods, USSR Computational Mathematics and Mathematical Physics, Vol. 6, pp. 1–50, 1966.

  22. 22.

    Remes, E.,Sur un Procédé Convergent d'Approximations Successives pour Déterminer les Polynomes d'Approximation, Comptes Rendus Hebdomadaires des Séances de l'Académie des Sciences, Paris, France, Vol. 198, pp. 2063–2065, 1934.

  23. 23.

    Remes, E.,Sur le Calcul Effectif des Polynomes d'Approximation de Tchebichef, comptes Rendus Hebdomadaires des Séances de l'Académic des Sciences, Paris, France, Vol. 199, pp. 337–340, 1934.

  24. 24.

    Remes, E.,General Computational Methods of Chebyshev Approximation, Academy of Sciences of the Ukrainian SSR, Kiev, Ukraine, 1957 (United States Atomic Energy Commission Translation Series, AEC-TR-4491).

  25. 25.

    Veidinger, L.,On the Numerical Determination of the Best Approximations in the Chebyshev Sense, Numerische Mathematik, Vol. 2, pp. 99–105, 1960.

  26. 26.

    Cheney, E. W.,Introduction to Approximation Theory, McGraw-Hill Book Company, New York, New York, 1966.

  27. 27.

    Avriel, M.,Nonlinear Programming: Analysis and Methods, Prentice-Hall, Englewood Cliffs, New Jersey, 1976.

  28. 28.

    Luenberger, D. G.,Introduction to Linear and Nonlinear Programming, Addison-Wesley Publishing Company, Reading, Massachusetts, 1973.

  29. 29.

    Zangwill, W. I.,Nonlinear Programming: A Unified Approach, Prentice-Hall, Englewood Cliffs, New Jersey, 1969.

  30. 30.

    Dantzig, G. B., andCottle, R. W.,Positive (Semi) Definite Programming, Nonlinear Programming, Edited by J. Abadie, North-Holland Publishing Company, Amsterdam, Holland, 1967.

  31. 31.

    Lemke, C. E.,Bimatrix Equilibrium Points and Mathematical Programming, Management Science, Vol. 11, pp. 681–689, 1965.

  32. 32.

    Miller, C. E.,The Simplex Method for Local Separable Programming, Recent Advances in Mathematical Programming, Edited by R. L. Graves and P. Wolfe, McGraw-Hill Book Company, New York, New York, 1963.

  33. 33.

    Zoutendijk, G.,Nonlinear Programming: A Numerical Survey, SIAM Journal on Control, Vol. 4, pp. 194–210, 1966.

  34. 34.

    Avriel, M., Dembo, R., andPassy, U.,Solution of Generalized Geometric Programs, International Journal for Numerical Methods in Engineering, Vol. 9, pp. 149–168, 1975.

  35. 35.

    Dembo, R. S.,Current State of the Art of Algorithms and Computer Software for Geometric Programming, Journal of Optimization Theory and Applications, Vol. 26, pp. 149–183, 1978.

  36. 36.

    Rijckaert, M. J., andMartens, X. M.,Comparison of Generalized Geometric Programming Algorithms, Journal of Optimization Theory and Applications, Vol. 26, pp. 205–242, 1978.

  37. 37.

    Sarma, P. V. L. N., Martens, X. M., Reklaitis, G. V., andRijckaert, M. J.,A Comparison of Computational Strategies for Geometric Programs, Journal of Optimization Theory and Applications, Vol. 26, pp. 185–203, 1978.

  38. 38.

    Goldstein, A. A.,Constructive Real Analysis, Harper and Row, New York, New York, 1967.

  39. 39.

    Rockafellar, R. T.,Convex Analysis, Princeton University Press, Princeton, New Jersey, 1970.

  40. 40.

    Zoutendijk, G.,Methods of Feasible Directions, Elsevier Publishing Company, Amsterdam, Holland, 1960.

  41. 41.

    Dantzig, G. B.,Linear Programming and Extensions, Princeton University Press, Princeton, New Jersey, 1963.

  42. 42.

    Fiacco, A. V., andMcCormick, G. P.,Nonlinear Programming: Sequential Unconstrained Minimization Techniques, John Wiley and Sons, New York, New York, 1968.

  43. 43.

    Robinson, S. M.,An Application of Error Bounds for Convex Programming in a Linear Space, SIAM Journal on Control, Vol. 13, pp. 271–273, 1975.

  44. 44.

    Hoffman, A. J.,On Approximate Solutions of Systems of Linear Inequalities, Journal of Research of the National Bureau of Standards, Vol. 49, pp. 263–265, 1952.

  45. 45.

    Robinson, S. M.,Stability Theory for Systems of Inequalities, Part 2: Differentiable Nonlinear Systems, SIAM Journal on Numerical Analysis, Vol. 13, pp. 497–513, 1976.

  46. 46.

    Stern, M. H., andTopkis, D. M.,Rates of Stability in Nonlinear Programming, Operations Research, Vol. 24, pp. 462–476, 1976.

Download references

Author information

Additional information

The author is grateful to W. Oettli for bringing to his attention the linearly convergent cutting-plane algorithm of Ref. 15 and to the referee for a comment that stimulated an extension of the convergence rate results from an earlier version where δ k depended on certain parameters of the problem.

Communicated by D. G. Luenberger

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Topkis, D.M. A cutting-plane algorithm with linear and geometric rates of convergence. J Optim Theory Appl 36, 1–22 (1982). https://doi.org/10.1007/BF00934337

Download citation

Key Words

  • Cutting-plane algorithms
  • outer approximation methods
  • rates of convergence
  • convex programming