Abstract
This work provides a systematic review of the criteria most commonly used to classify sequential decision problems and discusses their impact on the performance of reinforcement learning and evolutionary computation. The paper also proposes a further division of one class of decision problems into two subcategories, which delimits a set of decision tasks particularly difficult for optimization techniques in general and evolutionary methods in particular. A simple computational experiment is presented to illustrate the subject.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barreto, A.M.S., Anderson, C.W.: Restricted gradient-descent algorithm for value-function approximation in reinforcement learning. Artificial Intelligence 172(4-5), 454–482 (2008)
Boyan, J.A., Moore, A.W.: Generalization in reinforcement learning: Safely approximating the value function. In: Advances in Neural Information Processing Systems, pp. 369–376. MIT Press, Cambridge (1995)
Bull, L., Kovacs, T. (eds.): Foundations of Learning Classifier Systems (Spring 2005)
Drugowitsch, J.: Design and Analysis of Learning Classifier Systems—A Probabilistic Approach. Springer, Heidelberg (2008)
Goldberg, D.: Genetic algorithms in search, optimization, and machine learning. Addison-Wesley, Reading (1989)
Gomez, F., Schmidhuber, J., Miikkulainen, R.: Accelerated neural evolution through cooperatively coevolved synapses. Journal of Machine Learning Research 9, 937–965 (2008)
Gomez, F.J.: Robust non-linear control through neuroevolution. Ph.D. thesis, The University of Texas at Austin (2003), Technical Report AI-TR-03-303
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, Heidelberg (2002)
Lagoudakis, M.G., Parr, R.: Least-squares policy iteration. Journal of Machine Learning Research 4, 1107–1149 (2003)
Moriarty, D.E.: Symbiotic Evolution of Neural Networks in Sequential Decision Tasks. Ph.D. thesis, The University of Texas at Austin (1997), Technical Report UT-AI97-257
Moriarty, D.E., Miikkulainen, R.: Efficient reinforcement learning through symbiotic evolution. Machine Learning 22(1–3), 11–32 (1996)
Moriarty, D.E., Schultz, A.C., Grefenstette, J.J.: Evolutionary algorithms for reinforcement learning. Journal of Artificial Intelligence Research 11, 241–276 (1999)
Puterman, M.L.: Markov Decision Processes—Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc., Chichester (1994)
Randløv, J., Alstrøm, P.: Learning to drive a bicycle using reinforcement learning and shaping. In: Proceedings of the Fifteenth International Conference on Machine Learning, pp. 463–471. Morgan Kaufmann Publishers Inc., San Francisco (1998)
Stanley, K.O.: Efficient evolution of neural networks through complexification. Ph.D. thesis, The University of Texas at Austin (2004), Technical Report AI-TR-04-314
Sutton, R.S.: Learning to predict by the methods of temporal differences. Machine Learning 3, 9–44 (1988)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, Cambridge (1998)
Tsitsiklis, J.N., Roy, B.V.: Feature-based methods for large scale dynamic programming. Machine Learning 22, 59–94 (1996)
Tsitsiklis, J.N., Roy, B.V.: An analysis of temporal-difference learning with function approximation. IEEE Transactions on Automatic Control 42, 674–690 (1997)
White, D.J.: Real applications of Markov decision processes. Interfaces 15, 73–83 (1985)
Whitley, D.: The GENITOR algorithm and selective pressure:why rank-based allocation of reproductive trials is best. In: Schaffer, J. (ed.) Proceedings of the Third International Conference on Genetic Algorithms and their Applications, pp. 116–121. Morgan Kaufmann, San Francisco (1989)
PWhitley, D., Dominic, S., Das, R., Anderson, C.W.: Genetic reinforcement learning for neurocontrol problems. Machine Learning 13(2-3), 259–284 (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Barreto, A.M.S., Augusto, D.A., Barbosa, H.J.C. (2010). On the Characteristics of Sequential Decision Problems and Their Impact on Evolutionary Computation and Reinforcement Learning. In: Collet, P., Monmarché, N., Legrand, P., Schoenauer, M., Lutton, E. (eds) Artifical Evolution. EA 2009. Lecture Notes in Computer Science, vol 5975. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-14156-0_17
Download citation
DOI: https://doi.org/10.1007/978-3-642-14156-0_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-14155-3
Online ISBN: 978-3-642-14156-0
eBook Packages: Computer ScienceComputer Science (R0)