Building Optimal Operation Policies for Dam Management Using Factored Markov Decision Processes
- 1.1k Downloads
Abstract
In this paper, we present the conceptual model of a real-world application of factored Markov Decision Processes to dam management. The idea is to demonstrate that it is possible to efficiently automate the construction of operation policies by modelling compactly the problem as a sequential decision problem that can be easily solved using stochastic dynamic programming. We will explain the problem domain and provide an analysis of the resulting value and policy functions. We will also present a useful discussion about the issues that will appear when the conceptual model to be extended into a real-world application.
Notes
Acknowledgments
Authors wish to thank the Control, Electronics and Communication department and the Enabling Technologies division of the Electrical Research Institute-Mexico for the financial support to perform this research.
References
- 1.Andersen, S.K., Olesen, K.G., Jensen, F.V., Jensen, F.: HUGIN: a shell for building bayesian belief universes for expert systems. In: Proceeding Eleventh Joint Conference on Artificial Intelligence, IJCAI, pp. 1080–1085, Detroit, 20–25 August 1989Google Scholar
- 2.Bellman, R.E.: Dynamic Programming. Princeton U. Press, Princeton (1957)zbMATHGoogle Scholar
- 3.Boutilier, C., Dean, T., Hanks, S.: Decision-theoretic planning: structural assumptions and computational leverage. J. AI Res. 11, 1–94 (1999)MathSciNetzbMATHGoogle Scholar
- 4.Darwiche, A., Goldszmidt, M.: Action networks: a framework for reasoning about actions and change under understanding. In: Proceedings of the Tenth Conference on Uncertainty in AI, UAI-1994, pp. 136–144, Seattle (1994)Google Scholar
- 5.Dean, T., Kanazawa, K.: A model for reasoning about persistence and causation. Comput. Intell. 5, 142–150 (1989)CrossRefGoogle Scholar
- 6.Lamond, B.F., Boukhtouta, A.: Water reservoir applications of Markov decision processes. In: Feinberg, E.A., Shwartz, A. (eds.) Handbook of Markov Decision Processes, Methods and Applications. Kluwer, US (2002)Google Scholar
- 7.Haddawy, P., Suwandy, M.: Decision-theoretic refinement planning using inheritance abstraction. In: Hammond, K. (ed.) Proceedings of the Second International Conference on Artificial Intelligence Planning Systems. AAAI Press, Menlo Park (1994)Google Scholar
- 8.Howard, R.A., Matheson, J.E.: Influence diagrams. In: Howard, R.A., Matheson, J.E. (eds.) Principles and Applications of Decision Analysis, Menlo Park (1984)Google Scholar
- 9.Loucks, D.P., van Beek, E.: Water resources systems planning and management: an introduction to methods, models and applications. In: Dynamic Programming. United Nations Educational, Scientific and Cultural Organization (UNESCO) (2005)Google Scholar
- 10.Veloso, M., Carbonel, J., Perez, A., Borrajo, D., Fink, E., Blythe, J.: Integrating planning and learning: the prodigy architecture. J. Exp. Theor. AI 1, 81–120 (1995)CrossRefGoogle Scholar
- 11.Majercik, S.M., Littman, M.L.: MAXPLAN: a new approach to probabilistic planning. In: Simmons, R., Smith, S. (eds.) Proceedings of the Fourth International Conference on Artificial Intelligence Planning Systems, pp. 86–93. AAAI Press, Menlo Park (1998)Google Scholar
- 12.Pearl, J.: Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, San Francisco (1988)Google Scholar
- 13.Puterman, M.L.: Markov Decision Processes. Wiley, New York (1994)CrossRefzbMATHGoogle Scholar
- 14.Quinlan, J.R.: Induction of decision trees. Mach. Learn. 1(1), 81–106 (1986)Google Scholar