Skip to main content
SpringerLink
Log in

Local solutions of the Hamilton-Jacobi-Bellman equation for some stochastic problems

  • Stochastic Systems
  • Published:
Automation and Remote Control Aims and scope Submit manuscript

Abstract

Consideration is given to the control problem of motion of a linear oscillator, which is subject to the external Gaussian and Poisson random actions, with the aim to minimize the mean energy with the aid of the external bounded control force. The hybrid solution method is suggested for the solution of the stated problem. This method relies on the search in a portion of the phase space for the exact analytical solution of the appropriate Hamilton-Jacobi-Bellman (HJB) equation and the numerical solution of this equation in the remaining (bounded) portion of the space. It is proved that the found analytical solutions represent the asymptotics of solutions of the Hamilton-Jacobi-Bellman equation. With the aid of the decomposition method, the obtained results are applied to the problem for the suppression with the aid of the actuator of vibrations of an elastic rod (plate) that is found to be under the action of Gaussian random actions. Results of the numerical modeling are given.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bellman, R., Dynamic Programming, Princeton: Princeton Univ. Press, 1957.

    Google Scholar 

  2. Dreyfus, S.E., Dynamic Programming and Calculus of Variations, New York: Academic, 1965.

    MATH  Google Scholar 

  3. Chernous’ko, F.L. and Kolmanovskii, V.B., Optimal’noe upravlenie pri sluchainykh vozmuchsheniyak (Optimal Control at Random Perturbations), Moscow: Nauka, 1978.

    Google Scholar 

  4. Fleming, W.H and Rishel, R.W., Deterministic and Stochastic Optimal Control, New York: Springer-Verlag, 1975. Translated under the title Optimal’noe upravlenie determinirovannymi i stokhasticheskimi sistemami, Moscow: Mir, 1978.

    MATH  Google Scholar 

  5. Stengel, R.F., Stochastic Optimal Control, Theory and Application, New York: Wiley, 1986.

    MATH  Google Scholar 

  6. Boyd, S.P. and Barratt, C.H., Linear Controller Design: Limits of Performance, New Jersey: Prentice Hall, 1991.

    MATH  Google Scholar 

  7. Crandall, M.G., Ishii, H., and Lions, P.L., User’s Guide to Viscosity Solutions of Second Order Partial Differential Equations, Bull. Am. Math. Soc., 1992, no. 27, pp. 1–67.

  8. Lions, P.L., Generalized Solution of Hamilton-Jacobi Equations, Boston: Pitman, 1982.

    Google Scholar 

  9. Subbotin, A.I., Minimaksnye neravenstva i uravneniya Gamil’tona-Yakobi (Minimax Inequalities and Hamilton-Jacobi Equations), Moscow: Nauka, 1991.

    MATH  Google Scholar 

  10. Bardi, M. and Capuzzo-Dolcetta, I., Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, Boston: Birkhauser, 1997.

    MATH  Google Scholar 

  11. Bratus, A.S., The Method of Approximate Solution of the Bellman Equation for Problems of Optimal Control of the System Subjected to Random Perturbations, Prikl. Mat. Mekh., 1975, vol. 39, no. 39, pp. 235–245.

    MathSciNet  Google Scholar 

  12. Bratus, A.S. and Kolmanovskii, V.B., Approximate Optimal Control of Motion under the Action of the Poisson and Gaussian Random Processes, Diff. Uravn., 1997, vol. 8, no. 3, pp. 1558–1569.

    MathSciNet  Google Scholar 

  13. Bensoussan, A., Perturbation Methods in Optimal Control, New York: Wiley, 1988.

    MATH  Google Scholar 

  14. Bratus, A.S. and Chernous’ko, F.L., Numerical Solution of Problems for Optimal Correction at Random Perturbations, Zh. Vychisl. Mat. Mat. Fiz., 1974, vol. 14, no. 1, pp. 68–78.

    MATH  Google Scholar 

  15. Bancora-Imbert, M.C., Chow, P.L., and Menaldi, J.L., On the Numerical Approximation of an Optimal Correction Problem, SIAM J. Sci. Stat. Comput., 1988, no. 9, pp. 970–991.

  16. Crespo, L.G. and Sun, J.Q., Stochastic Optimal Control of Nonlinear Systems via Short-Time Gaussian Approximation and Cell Mapping, Nonlinear Dynamics, 2002, vol. 28, pp. 323–342.

    Article  MATH  MathSciNet  Google Scholar 

  17. Afanas’ev, V.N., Kolmanovskii, V.B., and Nosov, V.R., Matematicheskaya teoriya konstruirovaniya sistem upravleniya (Mathematical Theory of Design of Control Systems), Moscow: Vysshaya Shkola, 1989.

    MATH  Google Scholar 

  18. Bensoussan, A., Stochastic Control of Partially Observable Systems, Cambridge: Cambridge Univ. Press, 1992.

    MATH  Google Scholar 

  19. Bensoussan, A. and Menaldi, J.L., Hybrid Control and Dynamic Programming, Dynam. Continuous Discrete and Impulsive Syst., 1997, no. 3, pp. 395–442.

  20. Menaldi, J.L., Stochastic Hybrid Optimal Control Models, Aportaciones Mat., 2001, no. 16, pp. 205–250.

  21. Melikyan, A.A., Generalized Characteristics of First Order PDEs: Applications in Optimal Control and Differential Games, Boston: Burkhauser, 1998.

    MATH  Google Scholar 

  22. Bratus, A., Dimentberg, M., and Iourtchenko, D., Optimal Bounded Response Control for a Second-Order System under a White-Noise Excitation, J. Vibration and Control, 2000, no. 6, pp. 741–755.

  23. Bratus, A., Dimentberg, M., Iourtchenko, D., and Noori, M., Hybrid Solution Method for Dynamic Programming Equations for MDOF Stochastic Systems, Dynam. Control 2000, no. 10, pp. 107–116.

  24. Iourtchenko, D.V., Stochastic Optimal Bounded Control for a System with the Boltz Cost Function, J. Vibration and Control, 2000, no. 6, pp. 1195–1204.

  25. Bratus, A.S. and Volosov, K.A., Exact Solution of the Hamilton-Jacobi-Bellman Equation for Optimal Correction Problems with the Bounded Total Control Resource, Prikl. Mat. Mekh., 2004, vol. 68, no. 5, pp. 819–832.

    MATH  MathSciNet  Google Scholar 

  26. Dimentberg, M. and Bratus, A., Bounded Parametric Control of Random Vibrations, Proc. Royal Soc. Ser. A, 2000, no. 456, pp. 2351–2363.

  27. Iourtchenko, D.V., Parametric Control of a Stochastic Second-Order System, Izv. Akad. Nauk, Theory and Control Systems, 2004, vol. 43, no. 1, pp. 79–83.

    Google Scholar 

  28. Iourtchenko, D.V., Comparison of the Bounded and the Unbounded Control with Feedback for the Stochastic Linear-Quadratic Problem, Avtom. Telemekh., 2006, no. 7, pp. 88–94.

  29. Iourtchenko, D.V., Optimal Control of the Pendulum with Viscous Friction That is under the Action of the Gaussian and the Poisson Perturbation, VIII Rossiiskii kongress po teoreticheskoi i prikladnoi mekhanike (VIII Ros. Congr. Teor. Priklad. Mechan.), Perm, Russia, 2001.

    Google Scholar 

  30. Polyanin, A.D., Lineinye uravneniya matematicheskoi fiziki (spravochnik) (Linear Equations of Mathematical Physics (Handbook)), Moscow: Fizmatlit, 2001.

    Google Scholar 

  31. Shiryaev, A.N., Veroyatnost’ (Probability), Moscow: Nauka, 1980.

    Google Scholar 

  32. Bratis, A.S. and Ivanova, A.P., Local Solutions of the Hamilton-Jacobi-Bellman Equations and Their Application to the Problem of Optimal Control of Vibrations of Elastic Distributed Systems, Izv. Akad. Nauk, Teor. Sist. Upravlen., 2004, no. 2, pp. 52–61.

  33. Ivanova, A.P., Control in the Form of Synthesis for the Inhomogeneous Equation of Heat Conduction, Izv. Akad. Nauk, Teor. Sist. Upravlen., 2003, no. 5, pp. 26–34.

Download references

Author information

Authors and Affiliations

  1. Moscow State University of Railway Transport, Moscow, Russia

    A. S. Bratus, A. P. Ivanova & D. V. Iourtchenko

  2. Wayne State University, Detroit, USA

    J. L. Menaldi

Authors
  1. A. S. Bratus
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. P. Ivanova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. D. V. Iourtchenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. J. L. Menaldi
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Original Russian Text © A.S. Bratus, A.P. Ivanova, D.V. Iourtchenko, J.L. Menaldi, 2007, published in Avtomatika i Telemekhanika, 2007, No. 6, pp. 99–115.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bratus, A.S., Ivanova, A.P., Iourtchenko, D.V. et al. Local solutions of the Hamilton-Jacobi-Bellman equation for some stochastic problems. Autom Remote Control 68, 1023–1038 (2007). https://doi.org/10.1134/S0005117907060094

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0005117907060094

PACS numbers

Search

Navigation