Genetic Programming of Heterogeneous Ensembles for Classification

  • Hugo Jair Escalante
  • Niusvel Acosta-Mendoza
  • Alicia Morales-Reyes
  • Andrés Gago-Alonso
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8258)


The ensemble classification paradigm is an effective way to improve the performance and stability of individual predictors. Many ways to build ensembles have been proposed so far, most notably bagging and boosting based techniques. Evolutionary algorithms (EAs) also have been widely used to generate ensembles. In the context of heterogeneous ensembles EAs have been successfully used to adjust weights of base classifiers or to select ensemble members. Usually, a weighted sum is used for combining classifiers outputs in both classical and evolutionary approaches. This study proposes a novel genetic program that learns a fusion function for combining heterogeneous-classifiers outputs. It evolves a population of fusion functions in order to maximize the classification accuracy. Highly non-linear functions are obtained with the proposed method, subsuming the existing weighted-sum formulations. Experimental results show the effectiveness of the proposed approach, which can be used not only with heterogeneous classifiers but also with homogeneous-classifiers and under bagging/boosting based formulations.


Heterogeneous ensembles Genetic programming 


  1. 1.
    Dietterich, T.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  2. 2.
    Kuncheva, L., Whitaker, C.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51(2), 181–207 (2003)Google Scholar
  3. 3.
    Freund, Y., Schapire, R.E.: A decision-theoretic generalization of on-line learning and an application to boosting. J. Comput. Syst. Sci. 55, 119–139 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Breiman, L.: Random forest. Mach. Learn. 24(2), 123–140 (2001)MathSciNetGoogle Scholar
  5. 5.
    Bian, S., Wang, W.: On diversity and accuracy of homogeneous and heterogeneous ensembles. International Journal of Hybrid Intelligent Systems 4, 103–128 (2007)zbMATHGoogle Scholar
  6. 6.
    de Oliveira, D., Canuto, A., De Souto, M.C.P.: Use of multi-objective genetic algorithms to investigate the diversity/accuracy dilemma in heterogeneous ensembles. In: Proc. of IJCNN, pp. 2339–2346 (2010)Google Scholar
  7. 7.
    Park, C., Cho, S.: Evolutionary computation for optimal ensemble classifier in lymphoma cancer classification. In: Zhong, N., Raś, Z.W., Tsumoto, S., Suzuki, E. (eds.) ISMIS 2003. LNCS (LNAI), vol. 2871, pp. 521–530. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  8. 8.
    Macaš, M., Gabrys, B., Ruta, D., Lhotská, L.: Particle swarm optimisation of multiple classifier systems. In: Sandoval, F., Prieto, A.G., Cabestany, J., Graña, M. (eds.) IWANN 2007. LNCS, vol. 4507, pp. 333–340. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  9. 9.
    Yang, L., Qin, Z.: Combining classifiers with particle swarms. In: Wang, L., Chen, K., S. Ong, Y. (eds.) ICNC 2005. LNCS, vol. 3611, pp. 756–763. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  10. 10.
    Langdon, W.B., Barrett, S.J., Buxton, B.F.: Combining decision trees and neural networks for drug discovery. In: Foster, J.A., Lutton, E., Miller, J., Ryan, C., Tettamanzi, A.G.B. (eds.) EuroGP 2002. LNCS, vol. 2278, pp. 60–70. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  11. 11.
    Espejo, P., Ventura, S., Herrera, F.: A survey on the application of genetic programming to classification. IEEE T. Syst. Man. Cyb. C 40(2), 121–144 (2010)CrossRefGoogle Scholar
  12. 12.
    Escalante, H.J., Montes, M., Sucar, L.E.: Ensemble particle swarm model selection. In: Proc. of IJCNN, pp. 1–10 (2010)Google Scholar
  13. 13.
    Bhowan, U., Johnston, M., Zhang, M., Yao, X.: Evolving diverse ensembles using genetic programming for classification with unbalanced data. IEEE Transactions on Evolutionary Computation 17(3), 368–386 (2013)CrossRefGoogle Scholar
  14. 14.
    Langdon, W.B., Poli, R.: Foundations of Genetic Programming. Springer (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Hugo Jair Escalante
    • 1
  • Niusvel Acosta-Mendoza
    • 1
    • 2
  • Alicia Morales-Reyes
    • 1
  • Andrés Gago-Alonso
    • 2
  1. 1.Instituto Nacional de Astrofísica, Óptica y Electrónica (INAOE)PueblaMexico
  2. 2.Advanced Technologies Application Center (CENATAV)HavanaCuba

Personalised recommendations