Skip to main content
SpringerLink
Log in

Constrained Optimization: From Lagrangian Mechanics to Optimal Control and PDE Constraints

  • Chapter
  • First Online:
Optimization with PDE Constraints

Part of the book series: Lecture Notes in Computational Science and Engineering ((LNCSE,volume 101))

  • 1778 Accesses

Abstract

The history of constrained optimization spans nearly three centuries. The principal warhorse, Lagrange multipliers, was discovered by Lagrange in the Statics section of his famous book on Mechanics from 1788, by applying the idea of virtual velocities to problems in statics with constraints. The idea of virtual velocities, in turn, goes back to a letter of Johann Bernoulli from 1715 to Varignon, in which he announced a very simple rule for solving hundreds of Varignon’s problems in the blink of an eye. Varignon then explains this rule in his book published in 1725. Half a century later, Bernoulli’s rule was chosen by Lagrange as the general principle for the foundation of his mechanics, with the multipliers as the main tool for treating mechanical constraints. In the second edition of his mechanics, published in 1811, Lagrange stressed the importance of his multipliers also for constrained optimization. In particular, they provide spectacular simplifications of entire chapters of Euler’s treatise on Variational Calculus from 1744. Lagrange multipliers is however a much farther reaching concept; we show how one can discover the important primal and dual equations in optimal control and the famous maximum principle of Pontryagin using only Lagrange multipliers. Pontryagin and his group, however, did not discover the maximum principle this way, since they were coming from a completely different area of mathematics. We finally give the complete formulation of PDE constrained optimization based on duality introduced by Lions, and conclude with an outlook on more recent applications.

Mathematics Subject Classification (2010). Primary 01-02; Secondary 49-03, 65K10

Our intention is not to write a full historical paper, but to highlight the parts of the historical development we find interesting as mathematicians. For full details on the history of constrained optimization with complete references, see [45] and [46].

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    On the frontispiece is written “Dont le projet fut donné en M.DC.LXXXVII”.

  2. 2.

    “Votre projet d’une nouvelle mechanique fourmille d’un grand nombre d’exemples, dont quelques uns à en juger par les figures paroissent assez compliqués; mais je vous deffie de m’en proposer un à votre choix, que je ne resolve sur le champ et comme en jouant par ma dite regle.”

  3. 3.

    “…cependant permettez moy que je vous reproche ici une nonchalance qui vous est arrivé assez souvent en ce que vous portez quelques fois votre jugement un peu à la legere, sans examiner, si ce que vous croyez etre une objection en est veritablement une; …c’est donc pour une autre fois que je vous donne cet avertissement à fin que vous soyez à l’avenir sur vos gardes, quand il s’agit de juger…”

  4. 4.

    Varignon gave in his book the wrong date 1717, which was also copied by Lagrange.

  5. 5.

    dp, dq, dr are not independent at the equilibrium point.

  6. 6.

    Up to now, we have preserved all letters exactly as they appear in Lagrange, but we have changed this potential, denoted \(\Pi \) by Lagrange, to U, as it is usual now.

  7. 7.

    See also Carathéodory [16] for a general study of equivalent formulations.

  8. 8.

    This was already discovered by Carathéodory [16], see also Sect. 3.7.

  9. 9.

    For more details on the historical context for this development, see Plail [46] and also [45].

  10. 10.

    Personal communication of Plail with Boltyanski, and explanation by Gamkrelidze in his paper about the discovery of the maximum principle:

  11. 11.

    In fact, since the endpoint is fixed as well, no variations are allowed at the endpoint either, but then Pontryagin could not have obtained the solution (3.43) of the then overdetermined system of ordinary differential equations (3.42), and thus he decided to first only fix the starting point [27, p. 442]. This flaw was only later fixed by Boltyanski, see the end of this subsection.

  12. 12.

    To solve the time optimal control problem correctly using Lagrange multipliers, we need to introduce the time variable as a state variable, y 0(t): = t, which implies \(\dot{y}_{0} = 1\), y 0(0) = 0. The correct Lagrangian then becomes \(\mathcal{L}(\boldsymbol{y},\boldsymbol{\lambda },\boldsymbol{u}) = y_{0}(T) +\int _{ 0}^{T}\boldsymbol{\lambda }^{T}(\dot{\boldsymbol{y}} -\boldsymbol{ g}(\boldsymbol{y},\boldsymbol{u}))\mathit{dt}\), where all vectors are now one element longer. Computing the variational derivative with respect to \(\boldsymbol{y}\), we obtain now in addition to the earlier equations \(\dot{\lambda }_{0} = 0\) and \(z_{0}(T) +\lambda _{0}(T)z_{0}(T) = 0\) for arbitrary variation z 0, which implies \(\lambda _{0}(T) = -1\) and hence \(\lambda _{0}(t) = -1\) to complete the time optimality system with y 0(t): = t.

  13. 13.

    See also Footnote 12.

  14. 14.

    On the following pages we will solve the general problem of variational calculus in an (n+1) dimensional space with p ordinary differential equations as constraints, using the method of geodesic equal distances.

  15. 15.

    Das Hauptresultat besteht darin, dass unsere Gefällkurven mit den Cauchyschen Charakteristiken zusammenfallen und Lösungen der kanonischen Differentialgleichungen (3.62) sind, die in der Mechanik eine so bedeutende Rolle spielen.

  16. 16.

    According to J.-L. Lions: “Le travail de Yu. V. Egorov contient une étude détaillée de ce problème, mais nous n’avons pas pu comprendre tous les points des démonstrations de cet auteur, les résultats étant très probablement tous corrects.”

  17. 17.

    Institut de Recherche en Informatique et Automatique, the precursor of the modern INRIA.

  18. 18.

    “La formulation (1.31) peut être considérée comme un analogue du principe du maximum de Pontryagin, pour lequel nous référons […] à Pontryagin-Boltyanski-Gamkrelidze-Mischenko” [37].

  19. 19.

    According to J. Blum, it was R. Glowinski, one of the former students of Lions, who once showed Lions on the board that the adjoint state can simply be interpreted as a Lagrange multiplier. This was confirmed by R. Glowinski (personal communication).

References

  1. Archimedes, On the Equilibrium of Planes. publ. Basel (Latin-Greek, 1544), Paris (Latin-Greek, 1615, p.145), Heath (Engl., 1897, p.189), Ver Eecke (French, 1921, I, p.237–299), 250 B.C., in Opera of Archimedes

    Google Scholar 

  2. R. Bellman, I. Glicksberg, O. Gross, On the ’bang-bang’ control problem. Technical Report, DTIC Document, 1955

    Google Scholar 

  3. M. Benzi, G.H. Golub, J. Liesen, Numerical solution of saddle point problems. Acta Numerica, 14(1), pp. 1–137 (2005)

    Google Scholar 

  4. J. Bernoulli, Opera Omnia, 4 vols. (Bousquet & Socios., Lausannae/Genevae, 1742)

    Google Scholar 

  5. G.A. Bliss, Lectures on the Calculus of Variations, vol. 850 (University of Chicago Press, Chicago, 1946)

    MATH  Google Scholar 

  6. H.G. Bock, Randwertproblemmethoden zur Parameteridentifizierung in Systemen nichtlinearer Differentialgleichungen. Ph.D. thesis, Rheinische Friedrich-Wilhelms-Universität, Bonn, 1987 (No. 183)

    Google Scholar 

  7. V.G. Boltyanski, The maximum principle in the theory of optimal processes (Russian). Doklady AN SSSR 119(6), 1070–1073 (1958)

    Google Scholar 

  8. V.G. Boltyanski, The Maximum Principle—How it Came to Be? (Inst. für Mathematik, Technische Univ. München, München, 1994)

    Google Scholar 

  9. V.G. Boltyanski, R.V. Gamkrelidze, L.S. Pontryagin, On the theory of optimal processes (Russian). Doklady AN SSSR 110, 7–10 (1956)

    Google Scholar 

  10. V.G. Boltyanski, R.V. Gamkrelidze, L.S. Pontryagin, The theory of optimal processes. I. The maximum principle. Izv. Akad. Nauk SSSR. Ser. Mat. 24, 3–42 (1960)

    Google Scholar 

  11. V. Boltyanski, H. Martini, V. Soltan, Geometric Methods and Optimization Problems, vol. 4 (Springer, New York, 1999)

    MATH  Google Scholar 

  12. A. Borzì, V. Schulz, Computational Optimization of Systems Governed by Partial Differential Equations. Computational Science & Engineering (SIAM, Philadelphia, 2012)

    MATH  Google Scholar 

  13. D.W. Bushaw, Differential equations with a discontinuous forcing term. Ph.D. thesis, Department of Mathematics, Princeton University, 1952

    Google Scholar 

  14. D.W. Bushaw, Experimental towing tank. Technical Report, Stevens Institute of Technology, Reprint 169, Hoboken, 1953

    Google Scholar 

  15. C. Carathéodory, Über die diskontinuierlichen Lösungen in der Variationsrechnung. Ph.D. thesis, Universität Göttingen, 1904 (Gesammelte Mathematische Schriften, Band I), pp. 1–71

    Google Scholar 

  16. C. Carathéodory, Die Methode der geodätischen Äquidistanten und das Problem von Lagrange. Acta Mathematica 47(3), 199–236 (1926)

    Article  MathSciNet  MATH  Google Scholar 

  17. T. Carraro, M. Geiger, R. Rannacher, Indirect multiple shooting for nonlinear parabolic optimal control problems with control constraints. SIAM J. Sci. Comput. 36(2), A452–A481 (2014)

    Article  MathSciNet  Google Scholar 

  18. Y.V. Egorov, Some problems in the theory of optimal control. Dokl. Akad. Nauk. SSSR 145, 720–723 (1962)

    MathSciNet  MATH  Google Scholar 

  19. Y.V. Egorov, Sufficient conditions for optimal control in Banach spaces. Mat. Sbornik 64, 79–101 (1964)

    Google Scholar 

  20. L. Euler, Methodus inveniendi lineas curvas maximi minimive proprietate gaudentes sive solutio problematis isoperimetrici latissimo sensu accepti (Bousquet & Socios., Lausannae/Genevae, 1744) [Enestr. 65, Opera Omnia, Ser.I, vol. 24]

    Google Scholar 

  21. H.O. Fattorini, Time-optimal control of solutions of operational differential equations. J. SIAM Control Ser. A 2(1), 54–59 (1964)

    MathSciNet  MATH  Google Scholar 

  22. A.A. Feldbaum, The simplest relay system of automatic control (Russian). Avtomatika i Telemehanika 10(4), 249–266 (1949)

    Google Scholar 

  23. A.A. Feldbaum, Optimal processes in systems of automatic control (Russian). Avtomatika i Telemehanika 14(6), 712–728 (1953)

    Google Scholar 

  24. A.A. Feldbaum, On synthesis of optimal systems of automatic control (Russian), in Transactions of the 2nd National Conference on the Theory of Automatic Control, Izdat. AN SSSR, vol. 2 (1955), pp. 325–360

    Google Scholar 

  25. A.A. Feldbaum, On synthesis of optimal systems with the aid of phase space (Russian). Avtomatika i Telemehanika 16(2), 129–149 (1955)

    MathSciNet  Google Scholar 

  26. A. Friedman, Optimal control for parabolic equations. J. Math. Anal. Appl. 18, 479–491 (1967)

    Article  MATH  Google Scholar 

  27. R.V. Gamkrelidze, Discovery of the maximum principle. J. Dyn. Control Syst. 5(4), 437–451 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  28. M.J. Gander, G. Wanner, From Euler, Ritz and Galerkin to modern computing. SIAM Rev. 54, 627–666 (2013)

    MathSciNet  Google Scholar 

  29. M. Heinkenschloss, A time-domain decomposition iterative method for the solution of distributed linear quadratic optimal control problems. J. Comput. Appl. Math. 173, 169–198 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  30. H.K. Hesse, G. Kanschat, Mesh adaptive multiple shooting for partial differential equations. Part I: linear quadratic optimal control problems. J. Numer. Math. 17(3), 195–217 (2009)

    MathSciNet  MATH  Google Scholar 

  31. M.R. Hestenes, A general problem in the calculus of variations with applications to paths of least time. Technical Report, RAND Memorandum RM-100, 1950. ASTIA Document Number AD 112382

    Google Scholar 

  32. H.B. Keller, Numerical Methods for Two-Point Boundary Value Problems (Waltham, Blaisdell, 1968)

    MATH  Google Scholar 

  33. J.L. Lagrange, Méchanique analitique (Chez la Veuve Desaint, A Paris, 1788)

    Google Scholar 

  34. J.L. Lagrange, Mécanique analytique. (Mme Ve Courcier, Paris, 1811/1815) [Second enlarged edition in two volumes; third edition 1853 publ. by J. Bertrand; fourth edition in Oeuvres de Lagrange, vol. 11,12, 1888]

    Google Scholar 

  35. A. Lerner, Improving of dynamic properties of automatic compensators with the aid of nonlinear connections I (Russian). Avtomatika i Telemehanika 13(2), 134–144 (1952)

    Google Scholar 

  36. A. Lerner, Constructing of time-optimal systems of automatic control with constrained values of coordinates of controled object (Russian), in Transactions of the 2nd National Conference on the Theory of Automatic Control, Izdat. AN SSSR, vol. 2 (1955), pp. 305–324

    Google Scholar 

  37. J.L. Lions, Contrôle optimal de systèmes gouvernés par des équations aux dérivées partielles (Dunod, Paris, 1968)

    MATH  Google Scholar 

  38. J.L. Lions, Exact controllability, stabilization and perturbations for distributed systems. SIAM Rev. 30, 1–68 (1988)

    Article  MathSciNet  MATH  Google Scholar 

  39. S. Mac Lane, P.L. Duren, R.A. Askey, U.C. Merzbach, Mathematics at the University of Chicago: A Brief History (American Mathematical Society, Providence, 1989)

    Google Scholar 

  40. E.J. McShane, On multipliers for Lagrange problems. Am. J. Math. 61(4), 809–819 (1939)

    Article  MathSciNet  Google Scholar 

  41. D.D. Morrison, J.D. Riley, J.F. Zancanaro, Multiple shooting method for two-point boundary value problems. Commun. ACM 5(12), 613–614 (1962)

    Article  MathSciNet  MATH  Google Scholar 

  42. M.R. Osborne, On shooting methods for boundary value problems. J. Math. Anal. Appl. 27(2), 417–433 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  43. H.J. Pesch, Carathéodory’s royal road of the calculus of variations: Missed exits to the maximum principle of optimal control theory. Numer. Algebra Control Optim. 3(1), 161–173 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  44. H.J. Pesch, R. Bulirsch, The maximum principle, Bellman’s equation, and Carathéodory’s work. J. Optim. Theory Appl. 80(2), 199–225 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  45. H.J. Pesch, M. Plail, The maximum principle of optimal control: a history of ingenious ideas and missed opportunities. Control Cybern. 38(4A), 973–995 (2009)

    MathSciNet  MATH  Google Scholar 

  46. M. Plail, Die Entwicklung der optimalen Steuerungen: von den Anfängen bis zur eigenständigen Disziplin in der Mathematik (Vandenhoeck und Ruprecht, Göttingen, 1998)

    MATH  Google Scholar 

  47. L.S. Pontryagin, Optimal regulation processes. Uspekhi Matematicheskikh Nauk 14(1), 3–20 (1959)

    MathSciNet  Google Scholar 

  48. L.S. Pontryagin, V.G. Boltyanski, R.V. Gamkrelidze, E.F. Mishchenko, The Mathematical Theory of Optimal Processes (Interscience Publishers/Wiley, New York, 1962)

    MATH  Google Scholar 

  49. R. Serban, S. Li, L.R. Petzold, Adaptive algorithms for optimal control of time-dependent partial differential-algebraic equation systems. Int. J. Numer. Methods Eng. 57(10), 1457–1469 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  50. F. Tröltzsch, Optimal Control of Partial Differential Equations: Theory, Methods and Applications. Graduate studies in mathematics, vol. 112 (American Mathematical Society, Providence, 2010)

    Google Scholar 

  51. P. Varignon, Nouvelle mechanique ou statique, 2 vols. (Chez Claude Jombert, A Paris, 1725)

    Google Scholar 

  52. L. von Wolfersdorf, Optimal control for processes governed by mildly nonlinear differential equations of parabolic type I. ZAMM 56, 531–538 (1976)

    Article  MATH  Google Scholar 

  53. L. von Wolfersdorf, Optimal control for processes governed by mildly nonlinear differential equations of parabolic type II. ZAMM 57, 11–17 (1977)

    Article  MATH  Google Scholar 

  54. E. Zuazua, Some problems and results on the controllability of partial differential equations, in Proceedings of the Second European Conference of Mathematics, Budapest, July 1996. Progress in mathematics (Birkhäuser Verlag, Basel, 1998), pp. 276–311

    Google Scholar 

  55. E. Zuazua, Controllability of partial differential equations and its semi-discrete approximations. Discrete Continuous Dyn. Syst. 8, 469–513 (2002)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors are grateful to M. Mattmüller for providing us with a copy of Bernoulli’s letter (Univ. Bibl. Basel, Handschriften-Signatur L I a 669, Nr. 50). We further thank Ph. Henry, C. Lubich and E. Hairer for helpful discussions which greatly improved the manuscript. We are also grateful to Armen Sergeev from the Steklov Institute in Moscow for his invaluable help to get the original sources of A.A. Feldbaum, and Peter Kloeden for obtaining the RAND report of Hestenes for us. We thank the Bibliothèque de Genève for granting permission to reproduce photographs from the original sources under catalogue numbers Kc62 (Varignon), Kc110 [33], Kc111 [34], Ka495 [4], Ka368 (Euler’s Methodus E65), Ka459 (Archimedes) and also for Figs. 26, 27, and 28 from [48]. We also thank Tatiana Smirnova-Nagnibeda, Rinat Kashaev, Zdeněk Strakoš and Ivana Gander for their valuable help in translating several texts that originally appeared in Russian. The authors acknowledge support by the European ScienceFoundation, the Swiss National Science Foundation and the Centro Stefano Franscini.

Author information

Authors and Affiliations

  1. Faculté des Sciences, Section Mathématiques, Université de Genève, CH-1211, Genève 4, Suisse, Geneva, Switzerland

    Martin J. Gander, Felix Kwok & Gerhard Wanner

Authors
  1. Martin J. Gander
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Felix Kwok
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerhard Wanner
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Martin J. Gander .

Editor information

Editors and Affiliations

  1. Institut für Mathematik, Universität Augsburg, Augsburg, Germany

    Ronald Hoppe

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Gander, M.J., Kwok, F., Wanner, G. (2014). Constrained Optimization: From Lagrangian Mechanics to Optimal Control and PDE Constraints. In: Hoppe, R. (eds) Optimization with PDE Constraints. Lecture Notes in Computational Science and Engineering, vol 101. Springer, Cham. https://doi.org/10.1007/978-3-319-08025-3_5

Download citation

Publish with us

Policies and ethics

Search

Navigation