Abstract
In this chapter we deal with optimization problems where the state process is a Piecewise Deterministic Markov Process. These processes evolve through random jumps at random time points while the behavior between jumps is governed by an ordinary differential equation. They form a general and important class of non-diffusions. It is known that every strongMarkov process with continuous paths of bounded variation is necessarily deterministic.We assume that both the jump behavior as well as the drift behavior between jumps can be controlled. Hence this leads to a control problem in continuous-time which can be tackled for example via the Hamilton-Jacobi-Bellman equation. However, since the evolution between jumps is deterministic these problems can also be reduced to a discrete-time Markov Decision Process where however the action space is now a function space. We can treat these problems with the methods we have established in the previous chapters. More precisely we will restrict the presentation to problems with infinite horizon, thus we will use the results of Chapter 7. We show that under some continuity and compactness conditions the value function of the Piecewise Deterministic Markov Decision Process is a fixed point of the Bellman equation (Theorem 8.2.6) and the computational methods of Chapter 7 apply. In Section 8.3 the important special class of continuous-time Markov Decision Chains is investigated, in particular for problems with finite time horizon.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Bäuerle, N., Rieder, U. (2011). Piecewise Deterministic Markov Decision Processes. In: Markov Decision Processes with Applications to Finance. Universitext. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-18324-9_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-18324-9_8
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-18323-2
Online ISBN: 978-3-642-18324-9
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)