Abstract
This chapter introduces a novel Estimation of Distribution Algorithm for solving Reinforcement Learning Problems, i.e., EDA-RL. As the probabilistic model of the EDA-RL, the Conditional Random Fields proposed by Lafferty et al. are employed. The Conditional Random Fields can estimate conditional probability distributions by using Markov Network. Moreover, the structural search of probabilistic model by using X 2-test, and data correction method are examined. One of the primary features of the EDA-RL is the direct estimation of reinforcement learning agents’ policies by using the Conditional Random Fields. Another feature is that a kind of undirected graphical probabilistic model is used in the proposed method. The experimental results on Probabilistic Transition Problems and Maze Problems show the effectiveness of the EDA-RL.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Butz, M.V., Pelikan, M.: Studying XCS/BOA learning in boolean functions: structure encoding and random boolean functions. In: Proc. of the 2006 Genetic and Evol. Comput. Conf., pp. 1449–1456 (2006)
Butz, M.V., Pelikan, M., Llorá, X., Goldberg, D.E.: Automated global structure extraction for effective local building block processing in XCS. Evolutionary Computation 14(3), 345–380 (2006)
Handa, H.: EDA-RL: estimation of distribution algorithms for reinforcement learning problems. In: GECCO 2009: Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, pp. 405–412 (2009)
Handa, H., Isozaki, M.: Evolutionary fuzzy systems for generating better Ms.PacMan players. In: 2008 IEEE International Conference on Fuzzy Systems, pp. 2182–2185 (2008)
Lafferty, J.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of 18th International Conference on Machine Learning, pp. 282–289. Morgan Kaufmann (2001)
Miyazaki, K., Yamamura, M., Kobayashi, S.: On the rationality of profit sharing in reinforcement learning. In: Proc. 3rd International Conference on Fuzzy Logic, Neural Nets and Soft Computing (IIZUKA 1994), pp. 285–288 (1994)
Ono, I., Nijo, T., Ono, N.: A genetic algorithm for automatically designing modular reinforcement learning agents. In: Proc. of the Genetic and Evol. Comput. Conf., pp. 203–210 (2000)
Sutton, C., Mccallum, A.: An introduction to conditional random fields for relational learning. In: Getoor, L., Taskar, B. (eds.) Introduction to Statistical Relational Learning, ch. 4, pp. 93–128. MIT Press, Cambridge (2007)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press (1998)
Watkins, C., Dayan, P.: Technical note:
-learning. Machine Learning 08, 279–292 (1992)Yamazaki, A., Shibuya, T., Hamagami, T.: Complex-valued reinforcement learning with hierarchical architecture. In: Proc. of IEEE International Conference on Systems, Man, and Cybernetics, pp. 1925–1931 (2010)
-learning. Machine Learning 08, 279–292 (1992)Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer Berlin Heidelberg
About this chapter
Cite this chapter
Handa, H. (2012). EDA-RL: EDA with Conditional Random Fields for Solving Reinforcement Learning Problems. In: Shakya, S., Santana, R. (eds) Markov Networks in Evolutionary Computation. Adaptation, Learning, and Optimization, vol 14. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-28900-2_14
Download citation
DOI: https://doi.org/10.1007/978-3-642-28900-2_14
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-28899-9
Online ISBN: 978-3-642-28900-2
eBook Packages: EngineeringEngineering (R0)