Abstract
In this paper we propose recursive methods in probability control. We maximize the threshold probability that a “total” reward is greater than or equal to a given lower level. We take three types of reward function — general function, forward recursive function and backward recursive function. By three types of imbedding method, we derive a backward recursive equation for maximum probability function. The first is to view any history (alternating sequence of state and decision) as a new state. The second is to incorporate past values (cumulative rewards) in state dynamics. The third is a state expansion by attachment of future values (threshold levels). It is shown that the threshold probability problem has an optimal policy in a common large class.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bellman, R.E.: Dynamic Programming. Princeton University Press, Princeton, NJ 1957
Bellman, R.E.: Some Vistas of Modern Mathematics. University of Kentucky Press, Lexington, KY 1968
Blackwell, D.: Discounted dynamic programming. Ann. Math. Stat. 36, 226–235 (1965)
Denardo, E.V.: Contraction mappings in the theory underlying dynamic programming. SIAM Review 9, 165–177 (1968)
Iwamoto, S.: Conditional decision processes with recursive reward function. J. Math. Anal. Appl. 230, 193–210 (1999)
Iwamoto, S.: Recursive method in stochastic optimization under compound criteria. Adv. Math. Econ. 3, 63–82 (2001)
Iwamoto, S., Tsurusaki, K., Fujita, T.: Conditional decision-making in a fuzzy environment. J. Operations Res. Soc. Japan 42, 198–218 (1999)
Iwamoto, S., Ueno, T., Fujita, T.: Controlled Markov chains with utility functions. In: Markov Processes and Controlled Markov Chains (Z. Hou et al. eds.), pp. 135–148 Kluwer, Dordrecht 2002
Le Van, C., Dana, D.-A.: Dynamic Programming in Economics. Kluwer, Boston, MA 1993
Ozaki, H., Streufert, P.A.: Dynamic programming for non-additive stochastic objects. J. Math. Eco. 25, 391–442 (1996)
Sargent, T.J.: Dynamic Macroeconomic Theory. Harvard University Press, Cambridge, MA 1987
Stokey, N.L., Lucas, R.E.: Recursive Methods in Economic Dynamics. Harvard University Press, Cambridge, MA 1989
Strauch, R.: Negative dynamic programming. Ann. Math. Stat. 37, 871–890 (1966)
Streufert, P.A.: Recursive Utility and Dynamic Programming. In: Handbook of Utility Theory Vol. 1 (S. Barbera et al. eds.), Chap. III. Kluwer, Boston, MA 1987
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag
About this chapter
Cite this chapter
Iwamoto, S. (2004). Recursive methods in probability control. In: Kusuoka, S., Maruyama, T. (eds) Advances in Mathematical Economics. Advances in Mathematical Economics, vol 6. Springer, Tokyo. https://doi.org/10.1007/978-4-431-68450-3_3
Download citation
DOI: https://doi.org/10.1007/978-4-431-68450-3_3
Publisher Name: Springer, Tokyo
Print ISBN: 978-4-431-68452-7
Online ISBN: 978-4-431-68450-3
eBook Packages: Springer Book Archive