Advertisement

QFCS: A Fuzzy LCS in Continuous Multi-step Environments with Continuous Vector Actions

  • José Ramírez-Ruiz
  • Manuel Valenzuela-Rendón
  • Hugo Terashima-Marín
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5199)

Abstract

This paper introduces the QFCS, a new approach to fuzzy learning classifier systems. QFCS can solve the multistep reinforcement learning problem in continuous environments and with a set of continuous vector actions. Rules in the QFCS are small fuzzy systems. QFCS uses a Q-learning algorithm to learn the mapping between inputs and outputs. This paper presents results that show that QFCS can evolve rules to represent only those parts of the input and action space where the expected values are important for making decisions. Results for the QFCS are compared with those obtained by Q-learning with a high discretization to show that the new approach converges in a way similar to how Q-learning does for one-dimension problems with an optimal solution, and for two dimensions QFCS learns suboptimal solutions while it is difficult for Q-learning to converge due to that high discretization.

Keywords

Learning Classifier Systems Fuzzy Classifier Systems Fuzzy Logic Genetic Algorithm Induction Theory 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Holland, J.H.: Escaping brittleness: The possibilities of general-purpose learning algorithms applied to parallel rule-based systems. Machine Learning: An Artificial Intelligence Aproach 2 (1986)Google Scholar
  2. 2.
    Wilson, S.W.: Classifier fitness based on accuracy. Evolutionary Computation 3(2), 1–44 (1994)CrossRefGoogle Scholar
  3. 3.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, Ribera del Loira, 28. 28042 Madrid (Spain) (2004)Google Scholar
  4. 4.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press, London (1998)Google Scholar
  5. 5.
    Wilson, S.W.: Get real! XCS with continuous valued inputs. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 1999. LNCS (LNAI), vol. 1813, p. 209. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  6. 6.
    Stone, C., Bull, L.: For real! XCS with continuous valued inputs. Evolutionay Computation 11(3), 299–336 (2003)CrossRefGoogle Scholar
  7. 7.
    Dam, H.H., Abbass, H.A., Lokan, C.: Be real! XCS with continuous valued inputs. In: Genetic and Evolutionary Computation Conference, pp. 85–87 (2005)Google Scholar
  8. 8.
    Wilson, S.W.: Function approximation with a classifier system. In: Proceedings of Genetic and Evolutionary Computation Conference (GECCO 2001), vol. 4, pp. 974–981 (July 2001)Google Scholar
  9. 9.
    Lanzi, P.L., Loiacono, D., Wilson, S.W., Goldberg, D.: XCS with computable prediction in continuous multistep environments. ILLiGAL Report 2005018, Genetic Algorithms Laboratory, University of Illinois at Urbana-Champaign (May 2005)Google Scholar
  10. 10.
    Wilson, S.W.: Three architectures for continuous action. In: Kovacs, T., Llorà, X., Takadama, K., Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 2003. LNCS (LNAI), vol. 4399, pp. 239–257. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  11. 11.
    Tran, T.H., Cédric Sanza, Y.D., Nguyen, D.T.: Xcsf with computed continuous action. In: Proceedings of the 9th annual conference on Genetic and evolutionary computation, pp. 1861–1869 (2007)Google Scholar
  12. 12.
    Wang, L.X.: A Course in Fuzzy Systems and Control. Prentice Hall, Upper Saddle River (1996)Google Scholar
  13. 13.
    Valenzuela-Rendón, M.: The fuzzy classifier system: A classifier system for continuosly varying variables. In: Proceedings of the Fourth International Conference in Genetic Algorithms, pp. 346–353 (1991)Google Scholar
  14. 14.
    Valenzuela-Rendón, M.: Reinforcement learning in the fuzzy classifier system. Expert Systems with Applications 14, 237–247 (1998)CrossRefGoogle Scholar
  15. 15.
    Parodi, A., Bonelli, P.: A new approach to fuzzy classifier system. In: Proceedings of the Fifth International Conference on Genetic Algorithms, pp. 223–230 (1993)Google Scholar
  16. 16.
    Bonarini, A.: Evolutionary learning of fuzzy rules: Competition and cooperation. Fuzzy Modeling: Paradigms and Practice, 265–284 (1996)Google Scholar
  17. 17.
    Bonarini, A., Bonacina, C., Matteucci, M.: Fuzzy and crisp representations of real-valued input for learning classifier systems. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 1999. LNCS (LNAI), vol. 1813, pp. 228–235. Springer, Heidelberg (2000)Google Scholar
  18. 18.
    Matteucci, M.: Learning fuzzy classifier systems: Architecture and explorations (May (2000), http://citeseer.ist.psu.edu/matteucci00learning.html

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • José Ramírez-Ruiz
    • 1
  • Manuel Valenzuela-Rendón
    • 1
  • Hugo Terashima-Marín
    • 1
  1. 1.Center for Intelligent Systems, Tecnológico de MonterreyMonterrey, N.L.Mexico

Personalised recommendations