Econometric Genetic Programming in Binary Classification: Evolving Logistic Regressions Through Genetic Programming

  • André Luiz Farias NovaesEmail author
  • Ricardo Tanscheit
  • Douglas Mota Dias
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10423)


Logistic Regression and Genetic Programming (GP) have already been compared to each other in classification tasks. In this paper, Econometric Genetic Programming (EGP), first introduced as a regression methodology, is extended to binary classification tasks and evolves logistic regressions through GP, aiming to generate high accuracy classifications with potential interpretability of parameters, while uses statistical significance as a feature-selection tool and GP for model selection. EGP-Classification (or EGP-C), the name of this proposed EGP’s extension, was tested against a large group of algorithms in three cross-sectional datasets, showing competitive results in most of them. EGP-C successfully competed against highly non-linear algorithms, like Support Vector Machines and Multilayer Perceptron with Back Propagation, and still allows interpretability of parameters and models generated.


Genetic programming Binary classification Logistic regression Model selection 


  1. 1.
    Nourani, V., Pradhan, B., Ghaffari, H., Sharifi, H.: Landslide susceptibility mapping at Zonouz plain, Iran using genetic programming and comparison with frequency ratio, logistic regression, and artificial neural network models. Commun. J. Int. Soc. Prev. Mitig. Nat. Hazards 71(1), 523–547 (2014)Google Scholar
  2. 2.
    Ritchie, M.D., Motsinger, A.A., Bush, W.S., Coffey, C.S., Moore, J.H.: Genetic programming neural networks: a powerful bioinformatics tool for human genetics. Commun. Appl. Soft Computing J. 7(1), 471–479 (2007)CrossRefGoogle Scholar
  3. 3.
    Ong, C.-S., Huang, J.-J., Tzeng, G.-H.: Building credit scoring models using genetic programming. Commun. Expert Syst. Appl.: Int. J. 29(1), 41–47 (2005)CrossRefGoogle Scholar
  4. 4.
    Momm, H.G., Easson, G., Kuszmaul, J.: Integration of logistic regression and genetic programming to model coastal Louisiana land loss using remote sensing. In: Proceedings of the American Society for Photogrammetry and Remote Sensing 2007 Annual Conference, ASPRS 2007, Tampa, FL, USA (2007)Google Scholar
  5. 5.
    Novaes, A.L.F., Tanscheit, R., Dias, D.M.: Programação Genética Econométrica Aplicada a Problemas de Regressão em Conjuntos de Dados Seccionais. In: Proceedings of XIII Encontro Nacional de Inteligência Artificial, ENIAC 2016, Recife, PE, Brazil (2016)Google Scholar
  6. 6.
    Wooldridge, J.: Introductory Econometrics: A Modern Approach, 4th edn. Cengage Learning, Boston (2009)Google Scholar
  7. 7.
    Novaes, A.L.F.: Programação Genética Econométrica: uma Nova Abordagem para Problemas de Regressão e Classificação em Conjuntos de Dados Seccionais. Master’s thesis. Pontifícia Universidade Católica do Rio de Janeiro (PUC-Rio), Rio de Janeiro, Brazil (2015)Google Scholar
  8. 8.
    Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning, 1st edn. Springer New York Inc., New York (2001)CrossRefGoogle Scholar
  9. 9.
    Davidson, R., MacKinnon, J.: Estimation and Inference in Econometrics, 1st edn. Oxford University Press, Oxford (1993)zbMATHGoogle Scholar
  10. 10.
    Pratt, J.W.: Concavity of the log likelihood. J. Am. Stat. Assoc. 76(1), 103–106 (1981)MathSciNetCrossRefGoogle Scholar
  11. 11.
    Murray, W.: Newton-Type Methods. 1st edn. Stanford (2010)Google Scholar
  12. 12.
    Greene, W.H.: Econometric Analysis, 7th edn. Prentice Hall, Upper Saddle River (2011)Google Scholar
  13. 13.
    Poli, R., Langdon, W.B., McPhee, N.F.: A Field Guide to Genetic Programming, 1st edn. Lulu Enterprises, Raleigh (2008)Google Scholar
  14. 14.
    Luke, S., Panait, L.: Lexicographic parsimony pressure. In: Proceedings of the 2002 Conference on Genetic and Evolutionary Computation, GECCO 2002, pp. 829–836. ACM, San Francisco (2002)Google Scholar
  15. 15.
    Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection (Complex Adaptive Systems), 1st edn. The MIT Press, Cambridge (1992)zbMATHGoogle Scholar
  16. 16.
    Silva, S., Almeida, J.: Gplab – a genetic programming toolbox for matlab. In: Proceedings of the Nordic MATLAB Conference, pp. 273–278 (2003)Google Scholar
  17. 17.
    Searson, D.P., Leahy, D.E., Willis, M.J.: GPTIPS: an open source genetic programming toolbox for multigene symbolic regression. In: Proceedings of The International Multiconference of Engineers and Computer Scientists 2010, IMECS 2010, Hong Kong, pp. 77–80 (2010)Google Scholar
  18. 18.
    De Melo, V.V.: Kaizen programming. In: Proceedings of the 2014 Conference on Genetic and Evolutionary Computation, GECCO 2014, pp. 895–902. ACM, New York (2014)Google Scholar
  19. 19.
    De Melo, V.V., Banzhaf, W.: Improving logistic regression classification of credit approval with features constructed by Kaizen programming. In: Proceedings of the 2016 Conference on Genetic and Evolutionary Computation, GECCO 2016, pp. 61–62. ACM, New York (2016)Google Scholar
  20. 20.
    UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine, CA. Accessed 24 Feb 2015
  21. 21.
    Datasets used for classification: comparison of results. Nicolaus Copernicus University, Department of Informatics, Computational Intelligence Laboratory, Toruń, Poland. Accessed 24 Feb 2015

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • André Luiz Farias Novaes
    • 1
    Email author
  • Ricardo Tanscheit
    • 2
  • Douglas Mota Dias
    • 3
  1. 1.Informatics DepartmentUniversity of LisbonLisbonPortugal
  2. 2.Electrical Engineering DepartmentPUC-RioRio de JaneiroBrazil
  3. 3.Electronics and Telecommunications Engineering DepartmentUERJRio de JaneiroBrazil

Personalised recommendations