Higher Order Functions for Kernel Regression

  • Alexandros Agapitos
  • James McDermott
  • Michael O’Neill
  • Ahmed Kattan
  • Anthony Brabazon
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8599)


Kernel regression is a well-established nonparametric method, in which the target value of a query point is estimated using a weighted average of the surrounding training examples. The weights are typically obtained by applying a distance-based kernel function, which presupposes the existence of a distance measure. This paper investigates the use of Genetic Programming for the evolution of task-specific distance measures as an alternative to Euclidean distance. Results on seven real-world datasets show that the generalisation performance of the proposed system is superior to that of Euclidean-based kernel regression and standard GP.


Mean Square Error Genetic Programming Query Point Kernel Regression Genetic Programming System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    Agapitos, A., Lucas, S.M.: Evolving efficient recursive sorting algorithms. In: Proceedings of the 2006 IEEE Congress on Evolutionary Computation, July 6-21, pp. 9227–9234. IEEE Press, Vancouver (2006)Google Scholar
  3. 3.
    Agapitos, A., O’Neill, M., Brabazon, A.: Adaptive distance metrics for nearest neighbour classification based on genetic programming. In: Krawiec, K., Moraglio, A., Hu, T., Etaner-Uyar, A.Ş., Hu, B. (eds.) EuroGP 2013. LNCS, vol. 7831, pp. 1–12. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  4. 4.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006)Google Scholar
  5. 5.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010),
  6. 6.
    Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighbourhood components analysis. In: Advances in Neural Information Processing Systems 17, pp. 513–520. MIT Press (2004)Google Scholar
  7. 7.
    Goutte, C., Larsen, J.: Adaptive metric kernel regression. Journal of VLSI Signal Processing (26), 155–167 (2000)Google Scholar
  8. 8.
    Huang, R., Sun, S.: Kernel regression with sparse metric learning. Journal of Intelligent and Fuzzy Systems 24(4), 775–787 (2013)MathSciNetGoogle Scholar
  9. 9.
    McDermott, J., Byrne, J., Swafford, J.M., O’Neill, M., Brabazon, A.: Higher-order functions in aesthetic EC encodings. In: 2010 IEEE World Congress on Computational Intelligence, July 18-23, pp. 2816–2823. IEEE Computation Intelligence Society, IEEE Press, Barcelona, Spain (2010)Google Scholar
  10. 10.
    Poli, R., Langdon, W.B., McPhee, N.F.: A Field Guide to Genetic Programming. Lulu Enterprises, UK Ltd. (2008)Google Scholar
  11. 11.
    Takeda, H., Farsiu, S., Milanfar, P.: Robust kernel regression for restoration and reconstruction of images from sparse, noisy data. In: Proceeding of the International Conference on Image Processing (ICIP), pp. 1257–1260 (2006)Google Scholar
  12. 12.
    Trevor, H., Robert, T., Jerome, F.: The Elements of Statistical Learning, 2nd edn. Springer (2009)Google Scholar
  13. 13.
    Weinberger, K.Q., Tesauro, G.: Metric learning for kernel regression. In: Eleventh International Conference on Artificial Intelligence and Statistics, pp. 608–615 (2007)Google Scholar
  14. 14.
    Yu, T.: Hierachical processing for evolving recursive and modular programs using higher order functions and lambda abstractions. Genetic Programming and Evolvable Machines 2(4), 345–380 (2001)CrossRefzbMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Alexandros Agapitos
    • 1
  • James McDermott
    • 2
  • Michael O’Neill
    • 1
  • Ahmed Kattan
    • 3
  • Anthony Brabazon
    • 2
  1. 1.School of Computer Science and InformaticsUniversity College DublinIreland
  2. 2.School of BusinessUniversity College DublinIreland
  3. 3.Dept. of Computer ScienceUm Al Qura UniversityKingdom of Saudi Arabia

Personalised recommendations