Abstract
In this paper we study the problem of the optimal stopping of a Markov chain with a countable state space. In each state i the controller receives a reward r(i) if he stops the process or he must pay the cost c(i) otherwise. We show that, under the condition that there exists an optimal stopping rule, the policy iteration method, introduced by Howard, produces a sequence of stopping rules for which the expected return converges to the value function. For random walks on the integers with a special reward and cost structure, we show that the policy iteration method gives the solution of a discrete two point boundary value problem with a free boundary. We give a simple algorithm for the computation of the optimal stopping rule.
Chapter PDF
Similar content being viewed by others
Literature
DYNKIN, E.B., JUSCHKEWITSCH, A.A.; Sätze und Aufgaben über Markoffsche Prozesse. Springer-Verslag (1969).
HORDIJK, A., POTHARST, R., RUNNENBURG, J. Th.; Optimaal stoppen van Markov ketens. MC-syllabus19 (1973).
HORDIJK, A.; Dynamic programming and Markov potential theory. MC tract (1974).
HOWARD, R.A.; Dynamic programming and Markov processes. Technology Press, Cambridge Massachusetts (1960).
VAN HEE, K.M., HORDIJK, A.; A sequential sampling problem solved by optimal stopping. MC-rapport SW 25/73 (1973).
VAN HEE, K.M.; Note on memoryless stopping rules. COSOR-notitie R-73-12, T.H. Eindhoven (1974).
ROSS, S.; Applied probability models with optimization applications. Holden-Day (1970).
Editor information
Rights and permissions
Copyright information
© 1976 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
van Hee, K.M. (1976). The policy iteration method for the optimal stopping of a markov chain with an application. In: Cea, J. (eds) Optimization Techniques Modeling and Optimization in the Service of Man Part 2. Optimization Techniques 1975. Lecture Notes in Computer Science, vol 41. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-07623-9_277
Download citation
DOI: https://doi.org/10.1007/3-540-07623-9_277
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-07623-0
Online ISBN: 978-3-540-38150-1
eBook Packages: Springer Book Archive