Skip to main content
SpringerLink
Log in
Book cover

Optimization and Control Techniques and Applications pp 91–114Cite as

Use of Approximations of Hamilton-Jacobi-Bellman Inequality for Solving Periodic Optimization Problems

  • Conference paper
  • First Online:

  • 1155 Accesses

Part of the book series: Springer Proceedings in Mathematics & Statistics ((PROMS,volume 86))

Abstract

We show that necessary and sufficient conditions of optimality in periodic optimization problems can be stated in terms of a solution of the corresponding HJB inequality, the latter being equivalent to a max–min type variational problem considered on the space of continuously differentiable functions. We approximate the latter with a maximin problem on a finite dimensional subspace of the space of continuously differentiable functions and show that a solution of this problem (existing under natural controllability conditions) can be used for construction of near optimal controls. We illustrate the construction with a numerical example.

The work was supported by the Australian Research Council Discovery-Project Grants DP120100532 and DP130104432

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Anderson, E.J., Nash, P.: Linear Programming in Infinite-Dimensional Spaces. Wiley, Chichester (1987)

    MATH  Google Scholar 

  2. Ash, R.B.: Measure, Integration and Functional Analysis. Academic Press, New York (1972)

    MATH  Google Scholar 

  3. Bardi, M., Capuzzo-Dolcetta, I.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhäuser, Boston (1997)

    Book  MATH  Google Scholar 

  4. Bertsekas, D.P., Shreve, S.E.: Stochastic Optimal Control : The Discrete Time Case. Academic Press, New York (1978)

    MATH  Google Scholar 

  5. Bhatt, A.G., Borkar, V.S.: Occupation measures for controlled Markov processes: characterization and optimality. Ann. Probab. 24, 1531–1562 (1996)

    Article  MATH  MathSciNet  Google Scholar 

  6. Billingsley, P.: Convvergence of Probability Measures. John Wiley & Sons, New York (1968)

    Google Scholar 

  7. Colonius, F.: Optimal Periodic Control. Lecture Notes in Mathematics 1313. Springer, Berlin (1988)

    Google Scholar 

  8. Finlay, L., Gaitsgory, V., Lebedev, I.: Linear programming solutions of periodic optimization problems: approximation of the optimal control. J. Ind Manage. Optim. 3(2), 399–413 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  9. Finlay, L., Gaitsgory, V., Lebedev, I.: Duality in linear programming problems related to long run average problems of optimal control. SIAM J. Control Optim. 47, 1667–1700 (2008)

    Article  MATH  MathSciNet  Google Scholar 

  10. Fleming, W.H., Vermes, D.: Convex duality approach to the optimal control of diffusion. SIAM J. Control Optim. 27, 1136–1155 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  11. Gaitsgory, V.: ‘Control of Systems with Slow and Fast Motions: Averaging and Asymptotic Analysis. Nauka, Moscow (1991). (in Rusian)

    Google Scholar 

  12. Gaitsgory, V.: Suboptimization of singularly perturbed control systems. SIAM J. Control Optim. 30, 1228–1249 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  13. Gaitsgory, V.: On representation of the limit occupational measures set of a control systems with applications to singularly perturbed control systems. SIAM J. Control Optim. 43(1), 325–340 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  14. Gaitsgory, V., Quincampoix, M.: Linear programming approach to deterministic infinite horizon optimal control problems with discounting. SIAM J. Control Optim. 48(4), 2480–2512 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  15. Gaitsgory, V., Rossomakhine, S.: Linear programming approach to deterministic long run average problems of optimal control. SIAM J. Control Optim. 44(6), 2006–2037 (2005/2006)

    Google Scholar 

  16. Gaitsgory, V., Rossomakhine, S., Thatcher N.: Approximate solutions of the HJB inequality related to the infinite horizon optimall control problem with discounting. Dyn. Continuous Impulsive Systems Series B: Applications and Algorithms, 19 (2012), pp. 65–92.

    Google Scholar 

  17. Grammel, G.: On nonlinear control systems with multiple time scales. J. Dyn. Control Syst. 10, 11–28 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  18. Hernandez-Hernandez, D., Hernandez-Lerma, O., Taksar, M.: The linear programming approach to deterministic optimal control problems. Appl. Math. 24(1), 17–33 (1996)

    MATH  MathSciNet  Google Scholar 

  19. Kurtz, T.G., Stockbridge, R.H.: Existence of Markov controls and characterization of optimal Markov controls. SIAM J. Control Optim. 36(2), 609–653 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  20. Llavona, J.G.: Approximation of Continuously Differentiable Functions. Mathematics Studies, 130. North Holland, Amsterdam (1986)

    Google Scholar 

  21. Parthasarathy, K.R.: Probability Measures on Metric Spaces. Academic Press, New York (1967)

    MATH  Google Scholar 

  22. Rubio, J.E.: Control and Optimization. The Linear Treatment of Nonlinear Problems. Manchester University Press, Manchester (1985)

    Google Scholar 

  23. Stockbridge, R.H.: Time-average control of a martingale problem: a linear programming formulation. Ann. Probab. 18, 206–217 (1990)

    Google Scholar 

  24. Vinter, R.: Convex duality and nonlinear optimal control. SIAM J. Control Optim. 31, 518–538 (1993)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Flinders Mathematical Sciences Laboratory, School of Computer Science, Engineering and Mathematics, Flinders University, GPO Box 2100, Adelaide, SA, 5001, Australia

    Vladimir Gaitsgory

  2. Centre for Industrial and Applicable Mathematics, University of South Australia, Mawson Lakes, SA, 5095, Australia

    Ludmila Manic

Authors
  1. Vladimir Gaitsgory
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ludmila Manic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vladimir Gaitsgory .

Editor information

Editors and Affiliations

  1. Department of Mathematics and Statistics, Curtin University, Perth, West Australia, Australia

    Honglei Xu

  2. Department of Mathematics and Statistics, Curtin Univeristy, Perth, Australia

    Kok Lay Teo

  3. Department of Mathematics, China University of Petroleum, Beijing, China

    Yi Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Gaitsgory , V., Manic, L. (2014). Use of Approximations of Hamilton-Jacobi-Bellman Inequality for Solving Periodic Optimization Problems. In: Xu, H., Teo, K., Zhang, Y. (eds) Optimization and Control Techniques and Applications. Springer Proceedings in Mathematics & Statistics, vol 86. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-43404-8_5

Download citation

Publish with us

Policies and ethics

Search

Navigation