Skip to main content
SpringerLink
Log in

Control of piecewise-deterministic processes via discrete-time dynamic programming

  • Conference paper
  • First Online:
Stochastic Differential Systems

Part of the book series: Lecture Notes in Control and Information Sciences ((LNCIS,volume 78))

Abstract

Controlled piecewise-deterministic Markov processes have deterministic trajectories punctuated by random jumps, at which the sample path is right-continuous. By considering the sequence of states visited by the process at its jump times, it is shown that a discounted infinite horizon control problem can be reformulated as a discrete-time Markov decision problem (the ‘positive’ case). Under certain continuity assumptions it is shown that an optimal stationary policy exists in relaxed controls.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. D. Bertsekas and S.E. Shreve, Stochastic Optimal Control: the Discrete-time Case, Academic Press, New York, 1978.

    Google Scholar 

  2. M.H.A. Davis, Piecewise-deterministic Markov processes:a general class of non diffusion stochastic models, J. Royal Statist. Soc(B)46(1984)

    Google Scholar 

  3. N. El Karoui, Les aspectes probabilistes du contrôle stochastique, Lecture Notes in Mathematics, Springer-Verlag, Berlin.

    Google Scholar 

  4. U. Gugerli, Optimal stopping of piecewise deterministic Markov processes, sub mitted to Stochastics.

    Google Scholar 

  5. S.M. Lenhart and Y.C. Liao, Integro-differential equations associated with optimal stopping of a piecewise-deterministic process, Stochastics, to appear 1985.

    Google Scholar 

  6. M. Soner, PhD Thesis, Brown University, 1984

    Google Scholar 

  7. F.A. van der Duyn Schouten, Markov Decision Drift Processes, CWI Tract, Amsterdam 1983.

    Google Scholar 

  8. D. Vermes, Optimal control of piecewise-deterministic Markov processes, Stochastics, 14(1985)165–208.

    Google Scholar 

  9. J. Warga, Optimal Control of differential and functional equations, Academic Press New York 1972

    Google Scholar 

  10. A.A. Yushkevich, Continuous-time Markov decision processes with interventions, Stochastics 9(1983)235–274.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Imperial College, SW7 2BT, London, England

    M. H. A. Davis

Authors
  1. M. H. A. Davis
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Norbert Christopeit Kurt Helmes Michael Kohlmann

Rights and permissions

Reprints and permissions

Copyright information

© 1986 Springer-Verlag

About this paper

Cite this paper

Davis, M.H.A. (1986). Control of piecewise-deterministic processes via discrete-time dynamic programming. In: Christopeit, N., Helmes, K., Kohlmann, M. (eds) Stochastic Differential Systems. Lecture Notes in Control and Information Sciences, vol 78. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0041157

Download citation

  • DOI: https://doi.org/10.1007/BFb0041157

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-16228-5

  • Online ISBN: 978-3-540-39767-0

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation