Skip to main content
SpringerLink
Log in

Optimal Control of Jump-Markov Processes and Viscosity Solutions

  • Conference paper
Stochastic Differential Systems, Stochastic Control Theory and Applications

Part of the book series: The IMA Volumes in Mathematics and Its Applications ((IMA,volume 10))

Abstract

We investigate the Bellman equation that arises in the optimal control of Markov processes. This is a fully nonlinear integro-differential equation. The notion of viscosity solutions is introduced and then existence and uniqueness results are obtained. Also, the connection between the optimal control problem and the Bellman equation is developed.

This research was supported in part by the Institute for Mathematics and its Applications with funds provided by the National Science Foundation and the Office of Naval Research.

This research was completed while the author was visiting the Institute for Mathematics and its Applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. M.C. Crandall, L.C. Evans and P.-L. Lions, Some properties of viscosity solutions of Hamilton-Jacobi equations, Trans. of AMS, 282 (1984), pp. 487–502.

    Article  MathSciNet  MATH  Google Scholar 

  2. M.C. Crandall and P.-L. Lions, Viscosity solutions of Hamilton-Jacobi equations, Trans. of AMS, 277 (1983), pp. 1–42.

    Article  MathSciNet  MATH  Google Scholar 

  3. M. Davis, Piecewise deterministic Markov processes: A general class of non-diffusion stochastic models, J. Royal Stat. Society (B), 46 (1984), pp. 343–388.

    Google Scholar 

  4. W.H. Fleming and H.M. Soner, work in progress.

    Google Scholar 

  5. Y.-C. Liao and S. Lenhart, Integro-differential equations associated with optimal stopping of a piecewise deterministic process, Stochastics, 15 (1985), pp. 183–207.

    Article  MathSciNet  MATH  Google Scholar 

  6. P.-L. Lions and P.E. Souganidis, Viscosity solutions of second-order equations, stochastic control and stochastic differential games, IMA Volumes in Mathematics and its Applications (1987).

    Google Scholar 

  7. H. Pragarauskas, On the control of jump processes, Lecture Notes in Control and Information Systems, 43, pp. 338–344.

    Google Scholar 

  8. R. Rishel, Dynamics programming and minimum principles for systems, SIAM J. Cont. and Opt., 13 (1975), pp. 338–371.

    Article  MathSciNet  MATH  Google Scholar 

  9. A.V. Skorokhod, Studies in the Theory of Random Processes, Dover, New York.

    Google Scholar 

  10. H.M. Soner, Optimal control with state-space constraint, II, SIAM Control and Opt., 24 /6 (1986), pp. 1110–1122.

    Article  MathSciNet  MATH  Google Scholar 

  11. D. Stroock, Diffusion processes associated with Lévy generators, Z. Warsch, 32 (1975), pp. 209–244.

    Article  MathSciNet  MATH  Google Scholar 

  12. D. Vermes, Optimal control of piecewise deterministic processes, Stochastics, 14 (1985), pp. 165–208.

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Deparment of Mathematics, Carnegie-Mellon University, Schenley Park, Pittsburgh, PA, 15213, USA

    Halil Mete Soner

Authors
  1. Halil Mete Soner
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Division of Applied Mathematics, Brown University, 02912, Providence, Rhode Island, USA

    Wendell Fleming

  2. Ceremade, Universite Paris-Dauphine, Place de Lattre de Tassigny, 75775, Paris Cedex 16, France

    Pierre-Louis Lions

Rights and permissions

Reprints and permissions

Copyright information

© 1988 Springer-Verlag New York Inc.

About this paper

Cite this paper

Soner, H.M. (1988). Optimal Control of Jump-Markov Processes and Viscosity Solutions. In: Fleming, W., Lions, PL. (eds) Stochastic Differential Systems, Stochastic Control Theory and Applications. The IMA Volumes in Mathematics and Its Applications, vol 10. Springer, New York, NY. https://doi.org/10.1007/978-1-4613-8762-6_29

Download citation

  • DOI: https://doi.org/10.1007/978-1-4613-8762-6_29

  • Publisher Name: Springer, New York, NY

  • Print ISBN: 978-1-4613-8764-0

  • Online ISBN: 978-1-4613-8762-6

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation