Abstract
This state of the art review starts with a discussion of the classical (Courant) penalty function and of the various theoretical results which can be proved. The function is used sequentially and numerical results are disappointing; the reasons for this are explained, and cannot be alleviated by extrapolation. These difficulties are apparently overcome by using the multiplier (augmented Lagrangian) penalty function, and the theoretical background and practical possibilities are described. However the current approach for forcing convergence has its disadvantages and there is scope for more research.
Most interest currently centres on exact penalty functions, and in particular the l 1 exact penalty function. This is a nonsmooth function so cannot be minimized adequately by current techniques for smooth functions. However it is very useful as a criterion function in association with other techniques such as sequential QP. A thorough description of the l 1 penalty function is given, which aims to clarify the first and second order conditions associated with the minimizing point. There are some disadvantages of using non-smooth penalty functions including the existence of curved grooves which can be difficult to follow, and the possibility of the Maratos effect occurring. Suggestions for alleviating these difficulties are discussed.
These effects have recently caused more emphasis in the search for suitable smooth exact penalty functions, and a survey of research in this area, and the theoretical and practical possibilities, is given.
Finally some of the many other ideas for penalty functions are discussed briefly.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bandler J. W. and Charalambous C. (1972), “Practical least p-th optimization of networks”, IEEE Trans. Microwave Theo. Tech. (1972 Symposium Issue), 20, 834– 840.
Bertsekas D. P. (1982), “Augmented Lagrangian and differentiable exact penalty methods”, in “Nonlinear Optimization 1981” ed. M. J. D. Powell, Academic Press, London.
Boggs P. T. and Tolle J. W. (1980), “Augmented Lagrangians which are quadratic in the multiplier”, J. Opt. Theo. Applns., 31, 17–26.
Boggs P. T. and Tolle J. W. (1981), “An implementation of a quasi-Newton method for constrained optimization”, University of N. Carolina at Chapel Hill, O. R. and Systems Analysis Report 81–3.
Carroll C. W. (1961), “The created response surface technique for optimizing nonlinear restrained systems”, Operations Res., 9, 169–184.
Chamberlain R. M., Lemarechal C., Pedersen H. C. and Powell M. J. D. (1982), “The watchdog technique for forcing convergence in algorithms for constrained optimization” in “Mathematical Programming Study 16, Algorithms for Constrained Minimization of Smooth Nonlinear Functions” eds. A. G. Buckley and J.-L. Goffin, North Holland, Amsterdam.
Charalambous C. (1977), “Nonlinear least p-th optimization and nonlinear programming”, Math. Prog., 12, 195–225.
Coleman T. F. and Conn A. R. (1980), “Nonlinear programming via an exact penalty function: Asymptotic analysis”, Univ. of Waterloo, Dept. of Comp. Sci. Report CS- 80–30.
Coope I. D. and Fletcher R. (1980), “Some numerical experience with a globally convergent algorithm for nonlinearly constrained optimization”, J. Opt. Theo. Applns., 32, 1–16.
Courant R. (1943), “Variational methods for the solution of problems of equilibrium and vibration”, Bull. Amer. Math. Soc., 49, 1–23.
Di Pillo G. and Grippo L. (1979), “A new class of augmented Lagrangians in nonlinear programming” SIAM J. Control Opt., 17, 618–628.
Di Pillo G., Grippo L. and Lampariello F. (1981), “A class of algorithms for the solution of optimization problems with inequalities”, CNR Inst, di Anal, dei Sistemi ed Inf. Report R18.
Fiacco A. V. and McCormick G. P. (1968), “Nonlinear Programming”, John Wiley, New York.
Fletcher R. (1970), “A class of methods for nonlinear programming with termination and convergence properties”, in “Integer and Nonlinear Programming” ed. J. Aba-die, North Holland, Amsterdam.
Fletcher R. (1972), “A class of methods for nonlinear programming, III Rates of convergence” in “Numerical Methods for Nonlinear Optimization”, ed. F. A. Lootsma, Academic Press, London.
Fletcher R. (1973), “An exact penalty function for nonlinear programming with inequalities”, Math. Progr. 5, 129–150.
Fletcher R. (1975), “An ideal penalty function for constrained optimization”, J. Inst. Maths. Applns., 7, 76–91.
Fletcher R. (1981a), “Practical Methods of Optimization, Vol. 2, Constrained Optimization”, John Wiley, Chichester.
Fletcher R. (1981b), “Numerical experiments with an exact penalty function method”, in “Nonlinear Programming 4”, eds. O. L. Mangasarian, R. R. Meyer and S. M. Robinson, Academic Press, New York.
Fletcher R. (1982), “Second order corrections for nondifferentiable optimization”, in “Numerical Analysis Proceedings, Dundee 1981”, ed. G. A. Watson, Lecture Notes in Mathematics 912, Springer Verlag, Berlin.
Fletcher R. and Lill S. A. (1972), “A class of methods for nonlinear programming, II Computational experience”, in “Nonlinear Programming”, eds. J. B. Rosen, O. L. Mangasarian and K. Ritter, Academic Press, New York.
Frisch K. R. (1955), “The logarithmic potential method of convex programming”, Memo., Univ. Inst, of Economics, Oslo, May 1955.
Gill P. E. and Murray W. (1976), “Nonlinear least squares and nonlinearly constrained optimization”, in “Numerical Analysis, Dundee 1975”, ed. G. A. Watson, Lecture Notes in Mathematics 506, Springer Verlag, Berlin.
Han S. P. (1977), “A globally convergent method for nonlinear programming”, J. Opt. Theo. Applns., 22, 297–309.
Han S. P. and Mangasarian O. L. (1981). L. (1981), “A dual differentiable exact penalty function”, Univ. of Wisconsin, Dept of Comp. Sci. Report 434, and in Math. Progr., 25, 293–306, (1983).
Hestenes M. R. (1969), “Multiplier and gradient methods”, J. Opt. Theo. Applns., 4, 303–320.
Lootsma F. A. (1972), “A survey of methods for solving constrained minimization problems via unconstrained minimization” in “Numerical Methods for Nonlinear Optimization”, ed. F. A. Lootsma, Academic Press, London.
Mangasarian O. L. (1973), “Unconstrained Lagrangians in nonlinear programming”, Univ. of Wisconsin, Dept. of Comp. Sci. Report 174.
Mayne D. Q. (1980), “On the use of exact penalty functions to determine step length in optimization algorithms” in “Numerical Analysis, Dundee 1979”, ed. G. A. Watson, Lecture Notes in Mathematics 773, Springer-Verlag, Berlin.
Morrison D. D. (1968), “Optimization by least squares”, SIAM J. Num. Anal., 5, 83–88.
Murray W. and Wright M. H. (1978), “Projected Lagrangian methods based on the trajectories of penalty and barrier functions”, Stanford Univ. Dept. of O. R. Report SOL 78–23.
Osborne M. R. and Ryan D. M. (1972), “A hybrid algorithm for nonlinear programming” in “Numerical Methods for Nonlinear Optimization”, ed. F. A. Lootsma, Academic Press, London.
Powell M. J. D. (1969), “A method for nonlinear constraints in minimization problems”, in “Optimization”, ed. R. Fletcher, Academic Press, London.
Powell M. J. D. (1978), “A fast algorithm for nonlinearly constained optimization calculations”, in “Numerical Analysis, Dundee 1977”, ed. G. A. Watson, Lecture Notes in Mathematics 630, Springer Verlag, Berlin.
Powell M. J. D. (ed.) (1982), “Nonlinear Optimization 1981”, Academic Press, London.
Rockafellar R. T. (1974), “Augmented Lagrange multiplier functions and duality in non-convex programming”, SIAM J. Control, 12, 268–285.
Schittkowski K. (1980), “Nonlinear Programming Codes”, Lecture Notes in Economics and Mathematical Systems 183, Springer-Verlag, Berlin.
Wilson R. B. (1963), “A simplicial algorithm for concave programming”, PhD dissertation, Harvard Univ. Grad. School of Business Admin.
Wolfe P. (1965), “The composite simplex algorithm”, SIAM Rev., 7, 42–54.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1983 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Fletcher, R. (1983). Penalty Functions. In: Bachem, A., Korte, B., Grötschel, M. (eds) Mathematical Programming The State of the Art. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-68874-4_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-68874-4_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-68876-8
Online ISBN: 978-3-642-68874-4
eBook Packages: Springer Book Archive