Higher Order Functions for Kernel Regression

  • Alexandros Agapitos
  • James McDermott
  • Michael O’Neill
  • Ahmed Kattan
  • Anthony Brabazon
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8599)


Kernel regression is a well-established nonparametric method, in which the target value of a query point is estimated using a weighted average of the surrounding training examples. The weights are typically obtained by applying a distance-based kernel function, which presupposes the existence of a distance measure. This paper investigates the use of Genetic Programming for the evolution of task-specific distance measures as an alternative to Euclidean distance. Results on seven real-world datasets show that the generalisation performance of the proposed system is superior to that of Euclidean-based kernel regression and standard GP.




Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
  2. 2.
    Agapitos, A., Lucas, S.M.: Evolving efficient recursive sorting algorithms. In: Proceedings of the 2006 IEEE Congress on Evolutionary Computation, July 6-21, pp. 9227–9234. IEEE Press, Vancouver (2006)Google Scholar
  3. 3.
    Agapitos, A., O’Neill, M., Brabazon, A.: Adaptive distance metrics for nearest neighbour classification based on genetic programming. In: Krawiec, K., Moraglio, A., Hu, T., Etaner-Uyar, A.Ş., Hu, B. (eds.) EuroGP 2013. LNCS, vol. 7831, pp. 1–12. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  4. 4.
    Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006)Google Scholar
  5. 5.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010), http://archive.ics.uci.edu/ml
  6. 6.
    Goldberger, J., Roweis, S., Hinton, G., Salakhutdinov, R.: Neighbourhood components analysis. In: Advances in Neural Information Processing Systems 17, pp. 513–520. MIT Press (2004)Google Scholar
  7. 7.
    Goutte, C., Larsen, J.: Adaptive metric kernel regression. Journal of VLSI Signal Processing (26), 155–167 (2000)Google Scholar
  8. 8.
    Huang, R., Sun, S.: Kernel regression with sparse metric learning. Journal of Intelligent and Fuzzy Systems 24(4), 775–787 (2013)MathSciNetGoogle Scholar
  9. 9.
    McDermott, J., Byrne, J., Swafford, J.M., O’Neill, M., Brabazon, A.: Higher-order functions in aesthetic EC encodings. In: 2010 IEEE World Congress on Computational Intelligence, July 18-23, pp. 2816–2823. IEEE Computation Intelligence Society, IEEE Press, Barcelona, Spain (2010)Google Scholar
  10. 10.
    Poli, R., Langdon, W.B., McPhee, N.F.: A Field Guide to Genetic Programming. Lulu Enterprises, UK Ltd. (2008)Google Scholar
  11. 11.
    Takeda, H., Farsiu, S., Milanfar, P.: Robust kernel regression for restoration and reconstruction of images from sparse, noisy data. In: Proceeding of the International Conference on Image Processing (ICIP), pp. 1257–1260 (2006)Google Scholar
  12. 12.
    Trevor, H., Robert, T., Jerome, F.: The Elements of Statistical Learning, 2nd edn. Springer (2009)Google Scholar
  13. 13.
    Weinberger, K.Q., Tesauro, G.: Metric learning for kernel regression. In: Eleventh International Conference on Artificial Intelligence and Statistics, pp. 608–615 (2007)Google Scholar
  14. 14.
    Yu, T.: Hierachical processing for evolving recursive and modular programs using higher order functions and lambda abstractions. Genetic Programming and Evolvable Machines 2(4), 345–380 (2001)CrossRefMATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Alexandros Agapitos
    • 1
  • James McDermott
    • 2
  • Michael O’Neill
    • 1
  • Ahmed Kattan
    • 3
  • Anthony Brabazon
    • 2
  1. 1.School of Computer Science and InformaticsUniversity College DublinIreland
  2. 2.School of BusinessUniversity College DublinIreland
  3. 3.Dept. of Computer ScienceUm Al Qura UniversityKingdom of Saudi Arabia

Personalised recommendations