Summary
It is well-known that for the problem of stopping a Markov chain with finite state space there exists an optimal a.s. finite stopping time which is the entrance time into the set where the value function coincides with the utility function. In this paper, this result is extended to the case where more than one continue action is available. The result of the paper also yields a sufficient condition for the existence of a stationary optimal policy in a leavable gambling house with a compact action space.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Blackwell, D.: Discrete dynamic programming. Ann. Math. Statist. 33, 719–726 (1962).
Chow, Y.S., Robbins, H., Siegmund, D.: Great Expectations: The theory of optimal stopping. Boston: Houghton Mifflin Company (1971).
Deppe, H.: On the existence of average optimal policies in semi-regenerative decision models. Univ. Bónn, Inst. Angew. Math., SFB 72 preprint No. 476 (1981).
Dubins, L.E., Savage, L.J.: How to gamble if you must. New York: Mc Graw’Hill 1965.
Hordijk, A.; Dekker, R.: Denumerable Markov chains: sensitive optimality criteria. Operations Research Proc. 1982, Springer-Verlag Berlin Heidelberg (1983).
Mann, E.: Optimality equations and bias-optimality in bounded Markov decision processes. Univ. Bonn, Inst. Angew. Math., SFB 72 preprint No. 574 (1983).
Schweitzer, P.J.: Perturbation theory and finite Markov chains. J. Appl. Prob. 5, 401 413 (1968).
Sudderth, W.D.: On measurable gambling problems. Ann. Math. Statisi. 42, 260–269 (1971).
Wijngaard, J.: Sensitive optimality in stationary Markov decision problems on a general state space. Math. Centre Tracts 93, 85–93, Amsterdam (1977).
Zijm, W.H.M.: The optimality equations in multichain denumerable state Markov decision processes with the average cost criterion: the bounded cost case. A & E Report 2182, Univ. of Amsterdam (1982).
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Additional information
Dedicated to Professor Dr. P.H. Müller on the occasion of his sixtieth birthday.
Rights and permissions
Copyright information
© 1985 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Schäl, M. (1985). Optimal Stopping and Leavable Gambling Models with the Average Return Criterion. In: Neumann, K., Pallaschke, D. (eds) Contributions to Operations Research. Lecture Notes in Economics and Mathematical Systems, vol 240. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-46534-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-642-46534-5_10
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-15205-7
Online ISBN: 978-3-642-46534-5
eBook Packages: Springer Book Archive