Expression Inference — Genetic Symbolic Classification Integrated with Non-linear Coefficient Optimisation

  • Andrew Hunter
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2385)


Expression Inference is a parsimonious, comprehensible alternative to semi-parametric and non-parametric classification techniques such as neural networks, which generates compact symbolic mathematical expressions for classification or regression. This paper introduces a general framework for inferring symbolic classifiers, using the Genetic Programming paradigm with non-linear optimisation of embedded coefficients. An error propagation algorithm is introduced to support the optimisation. A multiobjective variant of Genetic Programming provides a range of models trading off parsimony and classification performance, the latter measured by ROC curve analysis. The technique is shown to develop extremely concise and effective models on a sample real-world problem domain.


Symbolic Regression Classification Genetic Programming ROC Curves Multiobjective Optimisation 


Symbolic Computations for Expert Systems and Machine Learning 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    A. Ben-Tal. Characterization of pareto and lexicographic optimal solutions. In Fandel and Gal, editors, Multiple Criteria Decision Making Theory and Application, pages 1–11. Springer-Verlag, 1979.Google Scholar
  2. 2.
    Christopher M. Bishop. Neural Networks for Pattern Recognition. Clarendon Press, Oxford, 1995.Google Scholar
  3. 3.
    Stefan Bleuler, Martin Brack, Lothar Thiele, and Eckart Zitzler. Multiobjective genetic programming: Reducing bloat by using spea2. In L. Spector, E. Goodman, A. Wu, W.B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. Garzon, and E. Burke, editors, Proceedings of the Congress on Evolutionar Computation, CEC-2001, pages 536–543, IEEE Press, 2001. Piscataway, NJ.Google Scholar
  4. 4.
    Edwin D. De Jong, Richard A. Watson, and Jordan B. Pollack. Reducing bloat and promoting diversity using multi-objective methods. In L. Spector, E. Goodman, A. Wu, W.B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. Garzon, and E. Burke, editors, Proceedings of the Genetic and Evolutionary Computation Conference, GECCO-2001, pages 11–18, San Francisco, CA, 2001. Morgan Kaufmann.Google Scholar
  5. 5.
    Christos Emmanouilidis, Andrew Hunter, and John MacIntyre. A multiobjective evolutionary setting for feature selection and a commonality-based crossover operator. In Proc. of the 2000 Congress on Evolutionary Computation, pages 309–316, Piscataway, NJ, 2000. IEEE Service Center.Google Scholar
  6. 6.
    S.E. Fahlman and C. Lebiere. The cascade-correlation learning architecture. In D.S. Touretzky, editor, Advances in Neural Information Processing Systems, volume 2, pages 524–532, San Mateo, CA, 1990. Morgan Kaufmann.Google Scholar
  7. 7.
    C. Fonseca and P. Fleming. An overview of evolutionary algorithms in multiobjective optimization. Evolutionary Computation, 3(1):1–16, 1995.CrossRefGoogle Scholar
  8. 8.
    Jeffrey Horn and Nicholas Nafpliotis. Multiobjective optimization using the niched pareto genetic algorithm. Technical Report IllIGAL 93005, University of Illinois, Urbana, IL, 1993.Google Scholar
  9. 9.
    Andrew Hunter. Using multiobjective genetic programming to infer logistic polynomial regression models. In Proceedings of the European Conference on Artificial Intelligence, EC AI 2002. IOS Press, 2002.Google Scholar
  10. 10.
    A. Hunter, L. Kennedy, J. Henry, and R.I. Ferguson. Application of neural networks and sensitivity analysis to improved prediction of trauma survival. Computer Methods and Algorithms in Biomedicine, 62:11–19, 2000.CrossRefGoogle Scholar
  11. 11.
    R.A. Jacobs, M.I. Jordan, S.J. Nowlan, and G.E. Hinton. Adaptive mixtures of local experts. Neural Computation, 3:79–87, 1991.CrossRefGoogle Scholar
  12. 12.
    J.R. Koza. Genetic Programming. MIT Press, Cambridge, MA., 1992.zbMATHGoogle Scholar
  13. 13.
    William H. Press, Brian P. Flannery, Saul A. Teukolsky, and William T. Vetterling. Numerical Recipes: The Art of Scientific Computing. Cambridge University Press, Cambridge, UK, 1986.Google Scholar
  14. 14.
    Katya Rodríguez-Vázquez, Carlos M. Fonseca, and Peter J. Fleming. Multiobjective Genetic Programming: A Nonlinear System Identification Application. In John R. Koza, editor, Late Breaking Papers at the Genetic Programming 1997 Conference, pages 207–212, Stanford University, California, 1997. Stanford Bookstore.Google Scholar
  15. 15.
    A.S. Weigend, D.E. Rumelhart, and B.A. Huberman. Generalization by weight-elimination with application to forecasting. In R.P. Lippmann, J.E. Moody, and D.S. Touretzky, editors, Advances in Neural Information Processing Systems, volume 3, pages 875–882, San Mateo, CA, 1990. Morgan Kaufmann.Google Scholar
  16. 16.
    M. Zweig and G. Cambell. ROC plots: A fundamental evaluation tool in clinical medicine. Clinical Chemistry, 39(4):551–577, 1993.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2002

Authors and Affiliations

  • Andrew Hunter
    • 1
  1. 1.Department of Computer ScienceUniversity of Durham Science LabsDurhamUK

Personalised recommendations