Abstract
This paper presents a particular case of a Bayesian dynamic controlled model, namely an infinite-horizon partially observed one, where the core process is a controlled finite state space, discrete-time, semi-Markov process.
We assume that the times of the control reset and the noise corrupted observations of the core process occur at times of core process transitions. The control employed each time of the core process transitions is allowed to be functionally dependent on the sample path of the core process only through the history of the corrupted observations.
Based on the construction of a sufficient statistic, the reduction of the considered controlled models with incomplete- information to controlled models with completely state-information was established and conditions for the average cost optimality criterion were stated in Drăguţ (1983).
In order to outline algorithms to calculate the optimal control policy and optimal payoff functions, new conditions must be imposed on the state structure of the partially observed semi-Markov process.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Drăguţ M. (1981): Non-discounted semi-Markov decision processes with incomplete state-information. In: Proc. of the Sixth Brasov Conference, 1979, Publishing House of the Academy of RSR, 403–411.
Drăguţ M. (1983): On semi-Markov decision processes with incomplete state-information. Doctoral Thesis, Centre of Mathematical Statistics, Bucharest, Romania.
Hinderer K. (1970): Foundations of non-stationary dynamic programming with discrete time parameter. In: Lecture Notes in Oper. Research and Mathem. Systems, vol. 33, Springer- Verlag.
Iosifescu M.: On invariant densities for piecewise monotonic transformation. To be published.
Kurano M. (1976): On the existence of an optimal stationary I-policy in non-discounted Markovian decision process with incomplete state-information. Bull. Math. Statist. 17, No. 3–4, 75–81.
Rhenius D. (1974): Incomplete information in Markov decision models. Ann. Statist. 2, No. 2, 1327–1334.
Ross S.M. (1970): Average cost semi-Markov decision processes. J. Appl. Probab. 7, No. 3, 649–656.
Sondik E. (1978): The optimal control of partially observable Markov processes over the infinite horizon: discounted costs. Oper. Res. 26, No. 2, 282–304.
Tijms H.C.: On dynamic programming with arbitrary state space, compact action space and the average return as criterion. To be published.
White C.C. (1976): Procedures for the solution of a finite horizon partially observed semi-Markov optimization problems. Oper. Res. 24, No. 2, 348–358.
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1988 Academia, Publishing House of the Czechoslovak Academy of Sciences, Prague
About this chapter
Cite this chapter
Drăguţ, M. (1988). Sufficient Optimality Conditions for Semi-Markov Decision Processes with Incomplete State-Information: Undiscounted Case. In: Višek, J.Á. (eds) Transactions of the Tenth Prague Conference. Czechoslovak Academy of Sciences, vol 10A-B. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-3859-5_26
Download citation
DOI: https://doi.org/10.1007/978-94-009-3859-5_26
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-010-8216-7
Online ISBN: 978-94-009-3859-5
eBook Packages: Springer Book Archive