Reducing Memory Requirements of Scope Approximator in Reinforcement Learning

  • Artur Michalski
Conference paper
Part of the Advances in Soft Computing book series (AINSC, volume 17)


Scope classification is an instance-based technique, which can be used as a function approximator in reinforcement learning system. However, without any storage management mechanism, its memory requirements can be huge. This paper presents modified version of scope approximator using density threshold to control memory usage. Computational experiments investigating the performance of the system and results achieved are reported.


Action Space Query Point Function Approximator Density Threshold Memory Utilization 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Atkeson CG, (1991) Memory-based learning control. In: Proceedings of the 1991 American Control Conference, volume 3, Boston, pp 2131–2136Google Scholar
  2. 2.
    Barto AG, Sutton RS, & Anderson CW (1983) Neuronlike elements that can solve difficult learning control problems. IEEE SMC, 13: 835–846Google Scholar
  3. 3.
    Lachiche N and Marquis P (1998) Scope Classification: An Instance-Based Learning Algorithm with a Rule-Based Characterisation. In: Proceedings of the 10th European Conference on Machine Learning, Chemnitz, Germany. Springer-Verlag, LNAI 1398Google Scholar
  4. 4.
    Michalski A (2000) An Application of Scope Classification in Reinforcement Learning. In: Proceedings of the 9th International Workshop on Intelligent Information Systems, Bystra, Poland 2000. IPI PAN, Warsaw, pp 89–93Google Scholar
  5. 5.
    McCallum RA (1996) Hidden state and reinforcement learning with instance-based state identification. IEEE Tans. on System, Man, and Cybernetics (special issue on Robot Learning)Google Scholar
  6. 6.
    Moore AW and Atkeson CG (1992) An investigation of memory-based function approximators for learning control. Technical Report, MIT Artificial Intelligence LaboratoryGoogle Scholar
  7. 7.
    Santamaria JC, Sutton RS, Ram A (1996) Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces. COINS Technical Report 96–088. University of Massachusetts, AmherstGoogle Scholar
  8. 8.
    Watkins CJCH and Dayan P (1992) Technical Note: Q-learning. Machine Learning 8: 279–292MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Artur Michalski
    • 1
  1. 1.Institute of Computing SciencePoznan University of TechnologyPoznanPoland

Personalised recommendations