Reducing Memory Requirements of Scope Approximator in Reinforcement Learning
Scope classification is an instance-based technique, which can be used as a function approximator in reinforcement learning system. However, without any storage management mechanism, its memory requirements can be huge. This paper presents modified version of scope approximator using density threshold to control memory usage. Computational experiments investigating the performance of the system and results achieved are reported.
KeywordsAction Space Query Point Function Approximator Density Threshold Memory Utilization
Unable to display preview. Download preview PDF.
- 1.Atkeson CG, (1991) Memory-based learning control. In: Proceedings of the 1991 American Control Conference, volume 3, Boston, pp 2131–2136Google Scholar
- 2.Barto AG, Sutton RS, & Anderson CW (1983) Neuronlike elements that can solve difficult learning control problems. IEEE SMC, 13: 835–846Google Scholar
- 3.Lachiche N and Marquis P (1998) Scope Classification: An Instance-Based Learning Algorithm with a Rule-Based Characterisation. In: Proceedings of the 10th European Conference on Machine Learning, Chemnitz, Germany. Springer-Verlag, LNAI 1398Google Scholar
- 4.Michalski A (2000) An Application of Scope Classification in Reinforcement Learning. In: Proceedings of the 9th International Workshop on Intelligent Information Systems, Bystra, Poland 2000. IPI PAN, Warsaw, pp 89–93Google Scholar
- 5.McCallum RA (1996) Hidden state and reinforcement learning with instance-based state identification. IEEE Tans. on System, Man, and Cybernetics (special issue on Robot Learning)Google Scholar
- 6.Moore AW and Atkeson CG (1992) An investigation of memory-based function approximators for learning control. Technical Report, MIT Artificial Intelligence LaboratoryGoogle Scholar
- 7.Santamaria JC, Sutton RS, Ram A (1996) Experiments with Reinforcement Learning in Problems with Continuous State and Action Spaces. COINS Technical Report 96–088. University of Massachusetts, AmherstGoogle Scholar