Natural Computing

, Volume 8, Issue 1, pp 133–148 | Cite as

An evolutionary approach for achieving scalability with general regression neural networks

  • Kenan Casey
  • Aaron Garrett
  • Joseph Gay
  • Lacey Montgomery
  • Gerry Dozier


In this paper, we present an approach to overcome the scalability issues associated with instance-based learners. Our system uses evolutionary computational techniques to determine the minimal set of training instances needed to achieve good classification accuracy with an instance-based learner. In this way, instance-based learners need not store all the training data available but instead store only those instances that are required for the desired accuracy. Additionally, we explore the utility of evolving the optimal feature set used by the learner for a given problem. In this way, we attempt to deal with the so-called “curse of dimensionality” associated with computational learning systems. To these ends, we introduce the Evolutionary General Regression Neural Network. This design uses an estimation of distribution algorithm to generate both the optimal training set as well as the optimal feature set for a general regression neural network. We compare its performance against a standard general regression neural network and an optimized support vector machine across four benchmark classification problems.


General regression neural network Evolutionary computation Support vector machine Estimation of distribution algorithm 


  1. Bellman RE (1961) Adaptive control processes: a guided tour. Princeton University PressGoogle Scholar
  2. Böhm C, Berchtold S, Keim DA (2001) Searching in high-dimensional spaces: index structures for improving the performance of multimedia databases. ACM Comput Surveys 33(3):322–373CrossRefGoogle Scholar
  3. Burges CJC (1998) A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2(2):955–974CrossRefGoogle Scholar
  4. Carlisle A, Dozier G (2001) An off-the-shelf PSO. In: Workshop on particle swarm optimization. Indianapolis, IN, pp 1–6.Google Scholar
  5. El-Naqa I, Yang Y, Galatsanos NP, Nishikawa RM, Wernick MN (2004) A similarity learning approach to content-based image retrieval: application to digital mammography. IEEE Trans Med Imaging 23(10):1233–1244CrossRefGoogle Scholar
  6. Engelbrecht AP (2002) Computational intelligence: an introduction. John Wiley and Sons, Ltd.Google Scholar
  7. Fu J, Lee S, Wong S, Yeh A, Wang JY, Wu H (2005) Image Segmentation Feature Selection and Pattern Classification for Mammographic Microcalcifications. Computerized Medical Imaging Graphics 29(6):419–429CrossRefGoogle Scholar
  8. Haykin S (1999) Neural Networks: A Comprehensive Foundation, 2nd edn. Prentice Hall.Google Scholar
  9. Hearst MA (1998) Support vector machines. IEEE Intelligent Syst 18–21Google Scholar
  10. Joachims T (1999) Making large-scale SVM learning practical. In: Schölkopf B, Burges C, Smola A (eds) Advances in Kernel Methods – Support Vector Learning. MIT PressGoogle Scholar
  11. Köppen M (2000) The curse of dimensionalityGoogle Scholar
  12. Kushmerick N (1999) Learning to remove internet advertisements. In: Proceedings of 3rd international conference on autonomous agentsGoogle Scholar
  13. Larranaga P, Lozano J (2002) Estimation of distribution algorithms: a new tool for evolutionary computation. Kluwer Academic PublishersGoogle Scholar
  14. Li R, Emmerich MT, Eggermont J, Bovenkamp EG (2006) Mixed-integer optimization of coronary vessel image analysis using evolution strategies. In: GECCO ’06: proceedings of the 8th annual conference on Genetic and evolutionary computationGoogle Scholar
  15. Mangasarian OL, Wolberg WH (1990) Cancer diagnosis via linear programming. SIAM News 23(5):1–18Google Scholar
  16. Mehrotra K, Mohan CK, Ranka S (1997) Elements of artificial neural networks. MIT PressGoogle Scholar
  17. Mitchell TM (1997) Machine learning. McGraw-Hill, 1st ednGoogle Scholar
  18. Novak E, Ritter K (1997) The curse of dimension and a universal method for numerical integration. In: Nürnberger G, Schmidt JW, Walz G (eds) Multivariate approximation and splines, ISNM. Birkhäuser, Basel, pp 177–188Google Scholar
  19. Okun O, Priisalu H (2006) Fast nonnegative matrix factorization and its application for protein fold recognition. EURASIP J Appl Signal Processing 2006, Article ID 71817, 8 pages.  10.1155/ASP/2006/71817.
  20. Picton P (2000) Neural Networks. Palgrave.Google Scholar
  21. Schölkopf B (2000) Statistical Learning and Kernel Methods. Technical Report MSR-TR-2000-23, Microsoft Research.Google Scholar
  22. Specht D (1991) A general regression neural network. IEEE Trans Neural Networks 2(6):568–576CrossRefGoogle Scholar
  23. Vapnik V (1982) Estimation of dependences based on empirical data; translated by Samuel Kotz. Springer Verlag.Google Scholar
  24. Vapnik V (1995) The nature of statistical learning theory. Springer.Google Scholar
  25. Zitzler E, Laumanns M, Bleuler S (2004) A tutorial on evolutionary multiobjective optimization. In: Gandibleux X, Sevaux M, Sorensen K, T’kindt V (eds) Metaheuristics for multiobjective optimisation. Springer, Berlin, pp 3–37Google Scholar

Copyright information

© Springer Science+Business Media B.V. 2007

Authors and Affiliations

  • Kenan Casey
    • 1
  • Aaron Garrett
    • 1
  • Joseph Gay
    • 1
  • Lacey Montgomery
    • 1
  • Gerry Dozier
    • 1
  1. 1.Department of Computer Science and Software EngineeringAuburn UniversityAuburnUSA

Personalised recommendations