Skip to main content
SpringerLink
Log in
Book cover

Stochastic Control Theory pp 31–78Cite as

Optimal Control for Diffusion Processes

  • Chapter
  • First Online:

  • 5231 Accesses

Part of the book series: Probability Theory and Stochastic Modelling ((PTSM,volume 72))

Abstract

This chapter deals with completely observable stochastic control problems for diffusion processes, described by SDEs. The decision maker chooses an optimal decision among all possible ones to achieve the goal. Namely, for a control process, its response evolves according to a (controlled) SDE and the payoff on a finite time interval is given. The controller wants to minimize (or maximize) the payoff by choosing an appropriate control process from among all possible ones. Here we consider three types of control processes:

  1. 1.

    \((\mathcal{F}_{t})\)-progressively measurable processes.

  2. 2.

    Brownian-adapted processes.

  3. 3.

    Feedback controls.

In order to analyze the problems, we mainly use the dynamic programming principle (DPP) for the value function.The reminder of this chapter is organized as follows. Section 2.1 presents the formulation of control problems and basic properties of value functions, as preliminaries for later sections. Section 2.2 focuses on DPP. Although DPP is known as a two stage optimization method, we will formulate DPP by using a semigroup and characterize the value function via the semigroup. In Sect. 2.3, we deal with verification theorems, which give recipes for finding optimal Markovian policies. Section 2.4 considers a class of Merton-type optimal investment models, as an application of previous results.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   69.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. R.A. Adams, Sobolev Spaces, 2nd edn. (Academic, Amsterdam/Boston, 2003)

    MATH  Google Scholar 

  2. A. Bain, D. Crisan, Fundamentals of Stochastic Filtering (Springer, New York/ London, 2009)

    Book  MATH  Google Scholar 

  3. R. Bellman, On the theory of dynamic programming. Proc. Nat. Sci. U.S.A. 38, 716–719 (1952)

    Article  MATH  Google Scholar 

  4. R. Bellman, Dynamic Programming (Princeton University Press, Princeton, 1957)

    MATH  Google Scholar 

  5. V.E. Benes, Composition and invariance methods for solvinig some stochastic control problems. Adv. Appl. Prob. 7, 299–329 (1975)

    Article  MATH  MathSciNet  Google Scholar 

  6. V.E. Venes, L.A. Shepp, H.S. Witsenhaussen, Some solvable stochastic control problems. Stochastics 4, 39–83 (1980)

    Article  MathSciNet  Google Scholar 

  7. A. Bensoussan, Stochastic Control of Partially Observable Systems (Cambridge University Press, Cambridge/New York, 1992)

    Book  MATH  Google Scholar 

  8. A. Bensoussan, M. Nisio, Nonlinear semigroup arising in the control of diffusions with partial observation. Stoch. Stoch. Rep. 30, 1–45 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  9. T.R. Bieleckii, S.R. Pliska, Risk sensitive dynamic asset management. Appl. Math. Optim. 39, 337–366 (1999)

    Article  MathSciNet  Google Scholar 

  10. F. Black, M. Scholes, The pricing of options and corporate liabilities. J. Polit. Econ. 81, 637–659 (1973)

    Article  Google Scholar 

  11. R. Buckdahn, J. Ma, Pathwise stochastic control problems and stochastic HJB equations. SIAM J. Control Optim. 45, 2224–2256 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  12. S. Brendle, Portfolio selection under incomplete information. Stoch. Proc. Appl. 116, 701–723 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  13. M.H. Chsng, T. Pang, J. Yong, Optimal stopping problem for stochastic differential equations with random cofficients. SIAM J. Control Optim. 48, 941–971 (2009)

    Article  MathSciNet  Google Scholar 

  14. M.G. Crandall, H. Ishii, The maximum principle for semicontinuous functions. Differ. Integral Equ. 3, 1001–1014 (1990)

    MATH  MathSciNet  Google Scholar 

  15. M.G. Crandall, H. Ishii, P.L. Lions, A user’s guide to viscosity solutions. Bull. AMS NS 27, 1–67 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  16. G. Da Prato, J. Zabczyk, Stochastic Equations in Infinite Dimenmensions (Cambridge University Press, Cambridge, 1992)

    Book  Google Scholar 

  17. N. El Karoui, S. Peng, M.C. Quenez, Backward stochastic differential equations in finance. Math. Finance 7, 1–71 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  18. N. El Karoui, M.C. Quenez, Dynamic programming and pricing of contingent claims in an incomplete market. SIAM. J. Control Optim. 33, 29–66 (1950)

    Article  Google Scholar 

  19. L.C. Evans, Classical solutions of Hamilton-Jacobi-Bellman equation for uniformly elliptic operators. Trans. AMS 275, 245–255 (1983)

    Article  Google Scholar 

  20. L.C. Evans, Partial Differential Equations. GSM19 (AMS, Providence, 1998)

    Google Scholar 

  21. W.H. Fleming, D. Hernandez-Hernandez, On the value of stochastic differential games. Commun. Stoch. Anal. 5, 341–351 (2011)

    MathSciNet  Google Scholar 

  22. W.H. Fleming, H. Kaise, S.J. Sheu, Max-plus stochastic control and risk sensitivity. Appl. Math. Optim 62, 81–144 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  23. W.H. Fleming, W.H. McEneaney, Risk sensitive control on infinite time horizon. SIAM J. Control Optim. 33, 1881–1915 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  24. W.H. Fleming, R.W. Rishel, Deterministic and Stochastic Optimal Control (Springer, Berlin/New York, 1975)

    Book  MATH  Google Scholar 

  25. W.H. Fleming, S.J. Sheu, Optimal long term growth rate of expected utility of wealth. Ann. Appl. Prob. 9, 871–903 (1999)

    Article  MATH  MathSciNet  Google Scholar 

  26. W.H. Fleming, S.J. Sheu, Risk sensitive control and an optimal investment model. Math. Finance 10, 197–213 (2000)

    Article  MATH  MathSciNet  Google Scholar 

  27. W.H. Fleming, S.J. Sheu, Risk sensitive control and an optimal investment model II. Ann. Appl. Prob. 12, 730–767 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  28. W.H. Fleming, H.M. Soner, Controlled MarkovProcesses and Viscosity Solutions, 2nd edn. (Springer, New York 2006)

    Google Scholar 

  29. W.H. Fleming, P.E. Souganidis, On the existence of value function of two-plsyer, zero-sum stochastic differential games. Indiana Math. J. 38, 293–314 (1989)

    Article  MATH  MathSciNet  Google Scholar 

  30. M. Fujisaki, G. Kallianpur, H. Kunita, Stochastic differential equations for the nonlinear filtering problem. Osaka J. Math. 9, 19–40 (1972)

    MATH  MathSciNet  Google Scholar 

  31. F. Gozzi, A. Świech, Hamilton-Jacobi-bellman equations for the optimal control of the Duncan-Mortensen-Zakai equation. J. Funct. Analy. 172, 466–510 (2000)

    Article  MATH  Google Scholar 

  32. J.M. Harrison, S.R. Pliska, Martingales and stochastic integrals in the theory of continuous ytading. Stoch. Proc. Appl. 11, 215–260 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  33. J.M. Harrison, S.R. Pliska, Stochastic calculus model of continuous trading; complete markets. Stoch. Proc. Appl. 15, 313–316 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  34. H. Hata, H. Nagai, S.I. Sheu, Asymptotics of the probability minimizing a down-side risk. Ann. Appl. Prob. 20, 52–89 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  35. H. Hata, J. Sekine, Explicit solution to a certain nonELQG risk-sensitive stochastic control problem. Appl. Math. Optim. 62, 341–380 (2010)

    Article  MATH  MathSciNet  Google Scholar 

  36. N. Ikeda, S. Watanabe, Stochastic Differential Equations and Diffusion Processes (North Holland, Amsterdam/New York, 1981)

    MATH  Google Scholar 

  37. H. Ishii, Viscosity solutions for a class of Hamilton-Jacobi equations in Hilbert spaces. J. Funct. Anal. 105, 301–341 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  38. K. Itô, Differential equations determining Markov processes. Zenkoku Shijo Sugaku Danwakai 244, 1352–1400 (1942). (In Japanese)

    Google Scholar 

  39. K. Itô, On Stochastic Differential Equations. Memoirs of the American Mathematical Society, vol. 4 (AMS, New York City, 1951)

    Google Scholar 

  40. P. Jaillet, D. Lamberton, B. Lapeyer, Variational inequalities and the pricing of American options. Acta Appl. Math. 21, 263–289 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  41. I. Karatzas, S.E. Sheve, Brownian Motion and Stochastic Calculus, 2nd edn. (Springer, New York, 1991)

    MATH  Google Scholar 

  42. I. Karatzas, S.E. Sheve, Methods of Mathematical Finance (Springer, New York, 1998)

    Book  MATH  Google Scholar 

  43. S. Koike, A Biginner’s Guide to the Theory of Viscisity Solutions. MSJ Memoirs, vol. 13 (JMS, Tokyo, 2004)

    Google Scholar 

  44. K. Kuroda, H. Nagai, Risk sensitive portfolio optimization and infinite time horizon. Stoch. Stoch. Rep. 73, 309–331 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  45. N.V. Krylov, Controlled Diffusion Processes, 2nd edn. (Springer, Berlin, 2009)

    MATH  Google Scholar 

  46. H.J. Kushner, Dynamical equations for optimal nonlinear filtering. J. Differ. Equ. 3, 179–190 (1967)

    Article  MATH  MathSciNet  Google Scholar 

  47. S. Lang, Real Analysis, 2nd edn. (Addison-Wesley, New York, 1983)

    MATH  Google Scholar 

  48. P.L. Lions, Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations I. J. Commun. PDE. 8, 1101–1134 (1983)

    Article  MATH  Google Scholar 

  49. P.L. Lions, Viscosity solutions of fully nonlinear second-order equations and optimal stochastic control in infinite dimensions. Part I. The case of bounded stochastic evolution. Acta Math. 161, 243–278 (1988)

    MATH  Google Scholar 

  50. P.L. Lions, Viscosity solutions of fully nonlinear second-order equations and optimal stochastic control in infinite dimensions. Part II. Optimal control of Zakai equation, in Stochastic Partial Differential Equations and Applications II, ed. by G. Da Prato, L. Tubaro. Lecture Notes in Mathematics, vol. 1390 (Springer, Berlin/Heidelberg, 1989), pp. 147–170. Part III. Uniqueness of viscosity solutions for general second order equations. J. Funct. Anal. 86, 1–18 (1989)

    Google Scholar 

  51. P.L. Lions, M. Nisio, A uniqueness result for the semigroup associated with HJB equations. Proc. Jpn. Acad. 58, 273–276 (1983)

    Article  MathSciNet  Google Scholar 

  52. R.S. Liptser, A.N. Shiryayev, Statistics of Random Processes I, II, 2nd edn. (Springer, New York/Berlin, 2001)

    Book  Google Scholar 

  53. C. Martini, American option prices as unique viscosity solutions to degenerate HJB equations, Rapport de rech, INRIA, 2000

    Google Scholar 

  54. R.C. Merton, Optimal consumption and portfolio rules in continuous time. J. Econ. Theory 3, 373–413 (1971)

    Article  MATH  MathSciNet  Google Scholar 

  55. R.C. Merton, Theory of rational option pricing. Bell J. Econ. Manag. Sci. 4, 141–183 (1973)

    Article  MathSciNet  Google Scholar 

  56. M. Metivier, Stochastic Partial Differential Equations in Infinite Dimensional Spaces (Scuola Superiore, Pisa, 1988)

    MATH  Google Scholar 

  57. P.A. Meyer, Probability and Potentials (Blaisdell, Waltham, 1966)

    MATH  Google Scholar 

  58. H. Morimoto, Stochastic Control and Mathematical Modeling, Applications in Economics (Cambridge University Press, Cambridge/New York, 2010)

    Book  MATH  Google Scholar 

  59. H. Nagai, Optimal strategies for risk sensitive portfolio optimization problems for general factor models. SIAM J. Control Optim. 41, 1779–1800 (2003)

    Article  MATH  MathSciNet  Google Scholar 

  60. M. Nisio, On a nonlinear semigroup attached to stochastic optimal control. Pull. RIMS 12, 513–537 (1976)

    Article  MATH  MathSciNet  Google Scholar 

  61. M. Nisio, On nonlinear semigroups for Markov processes associated with optimal stopping. Appl. Math. Optim. 4, 143–169 (1978)

    Article  MATH  MathSciNet  Google Scholar 

  62. M. Nisio, Lecture on Stochastic Control Theory. ISI Lecture Notes, vol. 9 (McMillan India, Delhi 1981)

    Google Scholar 

  63. M. Nisio, Stochastic differential games and viscosity solutions of Isaacs equations. Nagoya Math. J. 110, 163–184 (1988)

    MATH  MathSciNet  Google Scholar 

  64. E. Pardoux, Stochastic partial differential equations and filtering of diffusion processes. Stochastics 3, 127–167 (1979)

    Article  MATH  MathSciNet  Google Scholar 

  65. E. Pardoux, Stochastic partial differential equations, a review. Bull. Sc. Math. 117, 29–47 (1993)

    MATH  MathSciNet  Google Scholar 

  66. E. Pardoux, S. Peng, Adapted solution of backward stoshastic equation. Syst. Control Lett. 14, 55–61 (1990)

    Article  MATH  MathSciNet  Google Scholar 

  67. S. Peng, Stochastic Hamilton-Jacobi-Bellman equations. SIAM J. Control Optim. 30, 284–304 (1992)

    Article  MATH  MathSciNet  Google Scholar 

  68. H. Pham, Smooth solutions to optimal investment models with stochastic volatilities and portfolio constraints. Appl. Math. Optim. 46, 55–78 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  69. B.L. Rozovskii, Stochastic Evolution Systems (Kluwer Academic, Dordrecht/Boston, 1990)

    Book  MATH  Google Scholar 

  70. A.N. Shiryayev, Optimal Stopping Rules, 2nd edn. (Springer, Berlin/Heidelberg 2008)

    Google Scholar 

  71. L. Stettner, Penalty method for finite horizon stopping problems. SIAM J. Control Optim. 49, 1078–1099 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  72. D.V. Strook, S.R.S. Varadhan, Multidimensional Diffusion Processes (Springer, Berlin/New York, 1979)

    Google Scholar 

  73. J.C. Willems, Least squares stationary optimal control and the algebraic Ricatti equation. IEEE. Trans. Auto. Control 16, 621–635 (1971)

    Article  MathSciNet  Google Scholar 

  74. W.M. Wonham, on a matrix Ricatti equation of stochastic control. SIAM J. Control Optim. 6, 681–697 (1968)

    Google Scholar 

  75. K. Yosida, Functional Analysis, 6th edn. (Springer, Berlin/New York, 1980)

    Book  MATH  Google Scholar 

  76. J. Yong, X.Y. Zhou, Stochastic Controls, Hamiltonian Systems and HJB Equations (Springer, New York, 1999)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Kobe University, Kobe, Japan

    Makiko Nisio (emeritus)

  2. Osaka Electro–Communication University, Osaka, Japan

    Makiko Nisio (emeritus)

Authors
  1. Makiko Nisio
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer Japan

About this chapter

Cite this chapter

Nisio, M. (2015). Optimal Control for Diffusion Processes. In: Stochastic Control Theory. Probability Theory and Stochastic Modelling, vol 72. Springer, Tokyo. https://doi.org/10.1007/978-4-431-55123-2_2

Download citation

Publish with us

Policies and ethics

Search

Navigation