A k-NN Based Perception Scheme for Reinforcement Learning

  • José Antonio Martín H.
  • Javier de Lope
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4739)


A perception scheme for Reinforcement Learning (RL) is developed as a function approximator. The main motivation for the development of this scheme is the need for generalization when the problem to be solved has continuous state variables. We propose a solution to the generalization problem in RL algorithms using a k-nearest-neighbor pattern classification (k-NN). By means of the k-NN technique we investigate the effect of collective decision making as a mechanism of perception and action-selection and a sort of back-propagation of its proportional influence in the action-selection process as the factor that moderate the learning of each decision making unit. A very well known problem is presented as a case study to illustrate the results of this k-NN based perception scheme.


Reinforcement Learning k-Nearest-Neighbors Collective Decision Making 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Reinforcement Learning. In: Sutton, R.S (ed.) SECS, vol. 173, Kluwer Academic Publishers (1992). Reprinted from of Machine Learning, vol. 8(3-4) (1992) Google Scholar
  2. 2.
    Sutton, R.S.: Reinforcement learning and artificial intelligence (2006)Google Scholar
  3. 3.
    Sutton, R.S.: Reinforcement learning architectures. In: Proc. Int. Symp. on Neural Information Processing, Kyushu Inst.of Technology, Japan (1992)Google Scholar
  4. 4.
    Sutton, R.S., Barto, A.G.: Reinforcement Learning. An Introduction. MIT Press, Cambridge (1998)Google Scholar
  5. 5.
    Cover, T.M., Hart, P.E.: Nearest neighbor pattern classification. IEEE Transactions on Information Theory, IT 13(1), 21–27 (1967)zbMATHCrossRefGoogle Scholar
  6. 6.
    Dudani, S.A.: The distance-weighted k-nearest-neighbor rule. IEEE Transactions on Systems, Man and Cybernetics SMC-6(4), 325–327 (1976)Google Scholar
  7. 7.
    Watkins, C.J., Dayan, P.: Technical note Q-learning. Machine Learning 8, 279 (1992)zbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • José Antonio Martín H.
    • 1
  • Javier de Lope
    • 2
  1. 1.Dep. Sistemas Informáticos y Computación, Universidad Complutense de Madrid 
  2. 2.Dept. of Applied Intelligent Systems, Universidad Politécnica de Madrid 

Personalised recommendations