Abstract
XCS with computed prediction, namely XCSF, has been recently extended in several ways. In particular, a novel prediction update algorithm based on recursive least squares and the extension to polynomial prediction led to significant improvements of XCSF. However, these extensions have been studied so far only on single step problems and it is currently not clear if these findings might be extended also to multistep problems. In this paper we investigate this issue by analyzing the performance of XCSF with recursive least squares and with quadratic prediction on continuous multistep problems. Our results show that both these extensions improve the convergence speed of XCSF toward an optimal performance. As showed by the analysis reported in this paper, these improvements are due to the capabilities of recursive least squares and of polynomial prediction to provide a more accurate approximation of the problem value function after the first few learning problems.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Boyan, J.A., Moore, A.W.: Generalization in reinforcement learning: Safely approximating the value function. In: Tesauro, G., Touretzky, D.S., Leen, T.K. (eds.) Advances in Neural Information Processing Systems 7, pp. 369–376. The MIT Press, Cambridge (1995)
Butz, M.V., Pelikan, M.: Analyzing the evolutionary pressures in xcs. In: Spector, L., Goodman, E.D., Wu, A., Langdon, W.B., Voigt, H.-M., Gen, M., Sen, S., Dorigo, M., Pezeshk, S., Garzon, M.H., Burke, E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001), July 7-11, pp. 935–942. Morgan Kaufmann, San Francisco (2001)
Butz, M.V., Wilson, S.W.: An algorithmic description of xcs. Journal of Soft Computing 6(3-4), 144–153 (2002)
Goodwin, G.C., Sin, K.S.: Adaptive Filtering: Prediction and Control, Prentice-Hall information and system sciences series (March 1984)
Haykin, S.: Adaptive Filter Theory, 4th edn. Prentice-Hall, Englewood Cliffs (2001)
Lanzi, P.L., Loiacono, D.: Xcsf with neural prediction. In: IEEE Congress on Evolutionary Computation, CEC 2006, pp. 2270–2276 (2006)
Lanzi, P.L., Loiacono, D., Wilson, S.W., Goldberg, D.E.: Extending XCSF beyond linear approximation. In: Genetic and Evolutionary Computation – GECCO-2005, Washington DC, USA, pp. 1859–1866. ACM Press, New York (2005)
Lanzi, P.L., Loiacono, D., Wilson, S.W., Goldberg, D.E.: XCS with computed prediction for the learning of boolean functions. In: Proceedings of the IEEE Congress on Evolutionary Computation – CEC 2005, Edinburgh, UK, pp. 588–595. IEEE, Los Alamitos (September 2005)
Lanzi, P.L., Loiacono, D., Wilson, S.W., Goldberg, D.E.: XCS with computed prediction in continuous multistep environments. In: Proceedings of the IEEE Congress on Evolutionary Computation – CEC 2005, Edinburgh, UK, pp. 2032–2039. IEEE, Los Alamitos (September 2005)
Lanzi, P.L., Loiacono, D., Wilson, S.W., Goldberg, D.E.: Prediction update algorithms for XCSF: RLS, kalman filter, and gain adaptation. In: GECCO 2006: Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation, pp. 1505–1512. ACM Press, New York (2006)
Lanzi, P.L., Loiacono, D., Wilson, S.W., Goldberg, D.E.: Generalization in the XCSF classifier system: Analysis, improvement, and extension. Evolutionary Computation 15(2), 133–168 (2007)
Loiacono, D., Marelli, A., Lanzi, P.L.: Support vector regression for classifier prediction. In: GECCO 2007: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, pp. 1806–1813. ACM Press, New York (2007)
Watkins, C.J.C.H.: Learning from delayed reward. PhD thesis (1989)
Watkins, C.J.C.H., Dayan, P.: Technical note: Q-Learning. Machine Learning 8, 279–292 (1992)
Widrow, B., Hoff, M.E.: Neurocomputing: Foundation of Research. In: Adaptive Switching Circuits, pp. 126–134. The MIT Press, Cambridge (1988)
Wilson, S.W.: Classifier Fitness Based on Accuracy. Evolutionary Computation 3(2), 149–175 (1995), http://prediction-dynamics.com/
Wilson, S.W.: Mining Oblique Data with XCS. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (workshop organisers): Proceedings of the International Workshop on Learning Classifier Systems (IWLCS-2000), in the Joint Workshops of SAB 2000 and PPSN 2000, pp. 158–174 (2000)
Wilson, S.W.: Classifiers that approximate functions. Journal of Natural Computing 1(2-3), 211–234 (2002)
Wilson, S.W.: Classifier systems for continuous payoff environments. In: Deb, K., Poli, R., Banzhaf, W., Beyer, H.-G., Burke, E., Darwen, P., Dasgupta, D., Floreano, D., Foster, J., Harman, M., Holland, O., Lanzi, P.L., Spector, L., Tettamanzi, A., Thierens, D., Tyrrell, A. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 824–835. Springer, Heidelberg (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Loiacono, D., Lanzi, P.L. (2010). Recursive Least Squares and Quadratic Prediction in Continuous Multistep Problems. In: Bacardit, J., Browne, W., Drugowitsch, J., Bernadó-Mansilla, E., Butz, M.V. (eds) Learning Classifier Systems. IWLCS IWLCS 2009 2008. Lecture Notes in Computer Science(), vol 6471. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-17508-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-642-17508-4_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-17507-7
Online ISBN: 978-3-642-17508-4
eBook Packages: Computer ScienceComputer Science (R0)