Applied Mathematics & Optimization

, Volume 75, Issue 3, pp 499–523 | Cite as

A Numerical Approximation Framework for the Stochastic Linear Quadratic Regulator on Hilbert Spaces

Article

Abstract

We present an approximation framework for computing the solution of the stochastic linear quadratic control problem on Hilbert spaces. We focus on the finite horizon case and the related differential Riccati equations (DREs). Our approximation framework is concerned with the so-called “singular estimate control systems” (Lasiecka in Optimal control problems and Riccati equations for systems with unbounded controls and partially analytic generators: applications to boundary and point control problems, 2004) which model certain coupled systems of parabolic/hyperbolic mixed partial differential equations with boundary or point control. We prove that the solutions of the approximate finite-dimensional DREs converge to the solution of the infinite-dimensional DRE. In addition, we prove that the optimal state and control of the approximate finite-dimensional problem converge to the optimal state and control of the corresponding infinite-dimensional problem.

Keywords

Stochastic linear quadratic regulator problems Feedback control Approximation schemes Riccati equations 

Notes

Acknowledgments

The paper was partially supported by the project Solution of large-scale Lyapunov Differential Equations (P 27926) founded by the Austrian Science Foundation.

References

  1. 1.
    Arias, E., Hernández, V., Ibanes, J., Peinado, J.: A family of BDF algorithms for solving differential matrix Riccati equations using adaptive techniques. Procedia Comput. Sci. 1(1), 2569–2577 (2010)Google Scholar
  2. 2.
    Avalos, G., Lasiecka, I.: Differential Riccati equation for the active control of a problem in structural acoustics. J. Optim. Theory Appl. 91(3), 695–728 (1996)MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Banks, H.T., Kunisch, K.: The linear regulator problem for parabolic systems. SIAM J. Cont. Optim. 22, 684–698 (1984)MathSciNetCrossRefMATHGoogle Scholar
  4. 4.
    Benner, P., Ezzatti, P., Mena, H., Quintana-Ortí, E.S., Remón, A.: Solving matrix equations on multi-core and many-core architectures. Algorithms 6(4), 857–870 (2013)MathSciNetCrossRefGoogle Scholar
  5. 5.
    Benner, P., Mena, H.: Numerical solution of the infinite-dimensional LQR-problem and the associated differential Riccati equations. Technical Report MPIMD/12-13, MPI Magdeburg Preprint, 2012Google Scholar
  6. 6.
    Benner, P., Mena, H.: Rosenbrock methods for solving differential Riccati equations. IEEE Trans Automat Contr 58(11), 2950–2957 (2013)CrossRefGoogle Scholar
  7. 7.
    Bensoussan, A., Da Prato, G., Delfour, M.C.: Representation and Control of Infinite Dimensional Systems. Systems & Control: Foundations & Applications, vol. I. Birkäuser, Boston (1992)MATHGoogle Scholar
  8. 8.
    Bensoussan, A., Da Prato, G., Delfour, M.C., Mitter, S.K.: Representation and Control of Infinite Dimensional Systems. Systems & Control: Foundations & Applications, vol. II. Birkäuser, Boston (1992)MATHGoogle Scholar
  9. 9.
    Bismut, J.-M.: Linear quadratic optimal stochastic control with random coefficients. SIAM J. Control. Optim. 14, 419–444 (1976)MathSciNetCrossRefMATHGoogle Scholar
  10. 10.
    Bismut, J.-M.: Contrôle des systḿes linéaires quadratiques: applications de l’intégrale stochastique. In: Séminaire de Probabilités, XII. Lecture Notes in Math., vol. 649. Springer, Berlin, 1978Google Scholar
  11. 11.
    Bucci, F., Lasiecka, I.: Singular estimates and Riccati theory for thermoelastic plate models with boundary thermal control. Dyn. Cont. Disc. Impuls. Syst. 11, 545–568 (2004)MathSciNetMATHGoogle Scholar
  12. 12.
    Da Prato, G.: Direct solution of a riccati equation arising in stochastic control theory. Appl. Math. Optim. 11, 191–208 (1984)MathSciNetCrossRefMATHGoogle Scholar
  13. 13.
    Da Prato, G., Zabczyk, J.: Stochastic equations in infinite dimensions. Encyclopedia of Mathematics and its Applications, vol. 44. Cambridge University Press, Cambridge (1992)Google Scholar
  14. 14.
    Dragan, V., Morozan, T., Stoica, A.-M.: Mathematical Methods in Robust Control of Linear Stochastic Systems, 2nd edn. Springer, New York (2013)CrossRefMATHGoogle Scholar
  15. 15.
    Flandoli, F.: Direct solution of a Riccati equation arising in a stochastic control problem with control and observation on the boundary. Appl. Math. Optim. 14, 107–129 (1986)MathSciNetCrossRefMATHGoogle Scholar
  16. 16.
    Gibson, J.S.: The Riccati integral equation for optimal control problems in Hilbert spaces. SIAM J. Cont. Optim. 17(4), 537–565 (1979)CrossRefMATHGoogle Scholar
  17. 17.
    Guatteri, G., Tessitore, G.: On the backward stochastic Riccati equation in infinite dimensions. SIAM J. Control Optim. 44(1), 159–194 (2005)MathSciNetCrossRefMATHGoogle Scholar
  18. 18.
    Guatteri, G., Tessitore, G.: Backward stochastic Riccati equations in infinite horizon l-q optimal control with infinite dimensional state space and random coefficients. Appl. Math. Optim. 57, 207–235 (2008)MathSciNetCrossRefMATHGoogle Scholar
  19. 19.
    Hafizoglu, C.: Linear quadratic regulatory boundary/point control of stochastic partial differential equations systems with unbounded coefficients. Ph.D. Thesis, University of Virginia, US, 2006Google Scholar
  20. 20.
    Hafizoglu, C., Lasiecka, I., Levajković, T., Mena, H., Tuffaha, A.: The stochastic linear quadratic problem with singular estimates. Preprint 2015Google Scholar
  21. 21.
    Ichikawa, A.: Dynamic programming approach to stochastic evolution equations. SIAM J. Control. Optim. 17(1), 152–174 (1979)MathSciNetCrossRefMATHGoogle Scholar
  22. 22.
    Kohlmann, M., Tang, S.: New developments in backward stochastic Riccati equations and their applications. In: Mathematical Finance, Konstanz, 2000. Trends Math. Birkhuser, Basel (2001)Google Scholar
  23. 23.
    Kohlmann, M., Tang, S.: Global adapted solution of one-dimensional backward stochastic Riccati equations with application to the mean-variance hedging. Stoch. Process. Appl. 97, 1255–1288 (2002)MathSciNetCrossRefMATHGoogle Scholar
  24. 24.
    Kohlmann, M., Tang, S.: Multidimensional backward stochastic Riccati equations and applications. SIAM J. Control. Optim. 41, 1696–1721 (2003)MathSciNetCrossRefMATHGoogle Scholar
  25. 25.
    Kohlmann, M., Zhou, X.Y.: Relationship between backward stochastic differential equations and stochastic controls: a linear-quadratic approach. SIAM J. Control. Optim. 38, 1392–1407 (2000)MathSciNetCrossRefMATHGoogle Scholar
  26. 26.
    Kroller, M., Kunisch, K.: Convergence rates for the feedback operators arising in the linear quadratic regulator problem governed by parabolic equations. SIAM J. Numer. Anal. 28(5), 1350–1385 (1991)MathSciNetCrossRefMATHGoogle Scholar
  27. 27.
    Kushner, H.J.: Optimal stochastic control. IRE Trans. Auto. Control AC–7, 120–122 (1962)CrossRefGoogle Scholar
  28. 28.
    Lang, N., Mena, H., Saak, J.: On the benefits of the LDL factorization for large-scale differential matrix equation solvers. Linear Algebra Appl. 480, 44–71 (2015)MathSciNetCrossRefMATHGoogle Scholar
  29. 29.
    Lasiecka, I.: Optimal Control Problems and Riccati Equations for Systems with Unbounded Controls and Partially Analytic Generators: Applications to Boundary and Point Control Problems. Lecture Notes in Mathematics, vol. 1855. Springer, Berlin (2004)MATHGoogle Scholar
  30. 30.
    Lasiecka, I., Triggiani, R.: Control Theory for Partial Differential Equations: Continuous and Approximation Theories I. Abstract Parabolic Systems. Cambridge University Press, Cambridge, UK (2000)CrossRefMATHGoogle Scholar
  31. 31.
    Lasiecka, I., Triggiani, R.: Control Theory for Partial Differential Equations: Continuous and Approximation Theories II. Abstract Hyperbolic-Like Systems Over a Finite Time Horizon. Cambridge University Press, Cambridge, UK (2000)CrossRefMATHGoogle Scholar
  32. 32.
    Lasiecka, I., Triggiani, R., Optimal control and differential Riccati equations under singular estimates for \(e^{At}B \) in the absence of analyticity. Adv. Dyn. Control, Special Volume dedicated to A. V. Balakrishnan, Chapman and Hall/CRC Press, 271–309, 2004Google Scholar
  33. 33.
    Lasiecka, I., Tuffaha, A.: Riccati theory and singular estimates for control of a generalized fluid structure interaction model. Syst. Cont. Lett. 58(7), 499–509 (2009)CrossRefMATHGoogle Scholar
  34. 34.
    Levajković, T., Pilipović, S., Seleši, D., Žigić, M.: Stochastic evolution equations with multiplicative noise. Electron. J. Probab. 20(19), 1–23 (2015)MathSciNetMATHGoogle Scholar
  35. 35.
    Levajković, T., Mena, H.: On deterministic and stochastic linear quadratic control problem. Current Trends in Analysis and Its Applications, Trends in Mathematics, Research Perspectives, Springer International Publishing Switzerland 315–322, (2015)Google Scholar
  36. 36.
    Levajković, T., Mena, H., Tuffaha, A., The Stochastic Linear Quadratic Control Problem: A Chaos Expansion Approach. Evolution Equations and Control Theory, accepted for publication (2016)Google Scholar
  37. 37.
    Saak, J.: Efiziente numerische lösung eines optimalsteuerungsproblems für die abkühlung von stahlprofilen. Diplomarbeit, Fachbereich 3/Mathematik und Informatik, Universität Bremen, D-28334 Bremen, Germany, 2003Google Scholar
  38. 38.
    Tröltzsch, F., Unger, A.: Fast solution of optimal control problems in the selective cooling of steel. Z. Angew. Math. Mech. 81, 447–456 (2001)MathSciNetMATHGoogle Scholar
  39. 39.
    Wonham, W.M.: On the separation theorem of stochastic control. SIAM J. Control. 6, 312–326 (1968)MathSciNetCrossRefMATHGoogle Scholar
  40. 40.
    Wonham, W.M.: On a matrix Riccati equation of stochastic control. SIAM J. Control. 6, 681–697 (1968)MathSciNetCrossRefMATHGoogle Scholar
  41. 41.
    Yong, J., Zhou, X.Y.: Stochastic Controls - Hamiltonian Systems and HJB Equations. Applications of Mathematics, Stochastic Modelling and Applied Probability, vol. 43. Springer, New York (1999)MATHGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2016

Authors and Affiliations

  • Tijana Levajković
    • 1
    • 2
  • Hermann Mena
    • 1
  • Amjad Tuffaha
    • 3
  1. 1.Department of MathematicsUniversity of InnsbruckInnsbruckAustria
  2. 2.Faculty of Traffic and Transport EngineeringUniversity of BelgradeBelgradeSerbia
  3. 3.Department of MathematicsAmerican University of SharjahSharjahUAE

Personalised recommendations