Skip to main content
SpringerLink
Log in

EMnGA: Entropy Measure and Genetic Algorithms Based Method for Heterogeneous Ensembles Selection

  • Conference paper
  • First Online:
Intelligent Data Engineering and Automated Learning – IDEAL 2018 (IDEAL 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11315))

Abstract

Generating ensembles of classifiers increase the performances in classification and prediction but on the other hand it increases the storage space and the prediction time. Selection or simplification methods have been proposed to reduce space and time while maintaining or improving the performance of initial ensemble. In this paper we propose a method called EMnGA that uses a diversity-based entropy measure and a genetic algorithm-based search strategy to simplify a heterogeneous ensemble of classifiers. The proposed method is evaluated against its prediction performance and is compared to the initial ensemble as well as to the selection methods of heterogeneous ensembles in the literature using a sequential way.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Chapman & Hall, New York (1984)

    MATH  Google Scholar 

  2. Geurts, P.: Contributions to decision tree induction: bias/variance tradeoff and time series classification. Ph.D. thesis, Department of Electronical Engineering and Computer Science, University of Liège, May 2002

    Google Scholar 

  3. Margineantu, D., Dietterich, T.: Pruning adaptive boosting. In: Proceedings of the 14th International Conference on Machine Learning, pp. 211–218 (1997)

    Google Scholar 

  4. Dietterich, T.G.: Ensemble methods in machine learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45014-9_1

    Chapter  Google Scholar 

  5. Banfield, R.E., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P.: A comparison of decision tree ensemble creation techniques. IEEE Trans. Pat. Recog. Mach. Int. 29, 173–180 (2007)

    Article  Google Scholar 

  6. Bian, S., Wang, W.: On diversity and accuracy of homogeneous and heterogeneous ensembles. Intl. J. Hybrid Intell. Syst. 4, 103–128 (2007)

    Article  Google Scholar 

  7. Breiman, L.: Bagging predictors. Mach. Learn. 26(2), 123–140 (1996)

    MATH  Google Scholar 

  8. Schapire, R.E.: The strength of weak learnability. Mach. Learn. 5(2), 197–227 (1990)

    Google Scholar 

  9. Tang, E.K., Suganthan, P.N., Yao, X.: An analysis of diversity measures. Mach. Learn. 65(1), 247–271 (2006)

    Article  Google Scholar 

  10. Breiman, L.: Random forests. Mach. Learn. 45, 5–32 (2001)

    Article  Google Scholar 

  11. Banfield, R.E., Hall, L.O., Bowyer, K.W., Kegelmeyer, W.P.: Ensemble diversity measures and their application to thinning. Inf. Fusion 6(1), 49–62 (2005)

    Article  Google Scholar 

  12. Martinez-Munoz, G., Suarez, A.: Aggregation ordering in bagging. In: International Conference on Artificial Intelligence and Applications (IASTED), pp. 258–263. Acta Press (2004)

    Google Scholar 

  13. Martinez-Munoz, G., Suarez, A.: Pruning in ordered bagging ensembles. In: 23rd International Conference in Machine Learning (ICML-2006), pp. 609–616. ACM Press (2006)

    Google Scholar 

  14. Caruana, R., Niculescu-Mizil, A., Crew, G., Ksikes, A.: Ensemble selection from libraries of models. In: Proceedings of the 21st International Conference on Machine Learning (2004)

    Google Scholar 

  15. Partalas, I., Tsoumakas, G., Katakis, I., Vlahavas, I.: Ensemble pruning using reinforcement learning. In: Antoniou, G., Potamias, G., Spyropoulos, C., Plexousakis, D. (eds.) SETN 2006. LNCS (LNAI), vol. 3955, pp. 301–310. Springer, Heidelberg (2006). https://doi.org/10.1007/11752912_31

    Chapter  Google Scholar 

  16. Partalas, I., Tsoumakas, G., Vlahavas, I.: An ensemble uncertainty aware measure for directed hill climbing ensemble pruning. Mach. Learn. 81, 257–282 (2010)

    Article  MathSciNet  Google Scholar 

  17. Fan, W., Chu, F., Wang, H., Yu, P.S.: Pruning and dynamic scheduling of cost- sensitive ensembles. In: Eighteenth National Conference on Artificial Intelligence, American Association for Artificial Intelligence, pp. 146–151 (2002)

    Google Scholar 

  18. Zhang, Y., Burer, S., Street, W.N.: Ensemblepruning via semi-definite programming. J. Mach. Learn. Res. 7, 1315–1338 (2006)

    MathSciNet  MATH  Google Scholar 

  19. Hernández-Lobato, D., Martínez-Muñoz, G.: A statistical instance-based pruning in ensembles of independant classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 364–369 (2009)

    Article  Google Scholar 

  20. Dai, Q.: An efficient ensemble pruning algorithm using one-path and two-trips searching approach. Knowl.-Based Syst. 51, 85–92 (2013). ISSN 0950-7051

    Article  Google Scholar 

  21. Bhatnagar, V., Bhardwaj, M., Sharma, S., Haroon, S.: Accuracy-diversity based pruning of classifier ensembles. Prog. Artif. Intell. 2(2–3), 97–111 (2014)

    Article  Google Scholar 

  22. Guo, Y., et al.: A novel dynamic rough subspace based selective ensemble. Pattern Recognit. 48(5), 1638–1652 (2015). ISSN 0031-3203

    Article  Google Scholar 

  23. Partalas, I., Tsoumakas, G., Vlahavas, I.: A study on greedy algorithms for ensemble pruning. Technical report TR-LPIS-360-12, Department of Informatics Aristotle University of Thessaloniki, Greece (2012)

    Google Scholar 

  24. Haque, M.N., Noman, N., Berretta, R., Moscato, P.: Heterogeneous ensemble combination search using genetic algorithm for class imbalanced dataclassification. PLoS ONE 11(1), e0146116 (2016). https://doi.org/10.1371/journal.pone.0146116

    Article  Google Scholar 

  25. Pölsterl, S., et al.: Heterogeneous ensembles for predicting survival of metastatic, castrate-resistant prostate cancer patients. F1000Research 5(2676) (2016). PMC.Web 13 October 2017

    Article  Google Scholar 

  26. Partalas, I., Tsoumakas, G., Vlahavas, I.: Focused ensemble selection: a diversity- based method for greedy ensemble selection. In: Ghallab, M., Spyropoulos, C.D., Fakotakis, N., Avouris, N.M. (eds.) ECAI 2008 – Proceedings of the 18th European Conference on Artificial Intelligence. Frontiers in Artificial Intelligence and Applications, Patras, Greece, vol. 178, pp. 117–121 (2008)

    Google Scholar 

  27. Kuncheva, L.I., Whitaker, C.J.: Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Mach. Learn. 51, 181–207 (2003)

    Article  Google Scholar 

  28. Lerman, I., Ngouenet, F.: Algorithmes génétiques séquentiels et parallèles pour une représentation affine des proximités, Rapport de Recherche de l’INRIA Rennes - Projet REPCO 2570, INRIA (1995)

    Google Scholar 

  29. Asuncion, A., Newman, D.: UCI machine learning repository (2007). http://www.ics.uci.edu/»mlearn/MLRepository.html

Download references

Author information

Authors and Affiliations

  1. Department of Economics, University of Oran 2, Bir El Djir, Algeria

    Souad Taleb Zouggar

  2. Department of Computer Science, University of Oran 1, Oran, Algeria

    Abdelkader Adla

Authors
  1. Souad Taleb Zouggar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdelkader Adla
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdelkader Adla .

Editor information

Editors and Affiliations

  1. University of Manchester, Manchester, UK

    Hujun Yin

  2. Rm 209, Building B, Autonomous University of Madrid, Madrid, Spain

    David Camacho

  3. Campus of Gualtar, University of Minho, Braga, Portugal

    Paulo Novais

  4. University of Seville, Seville, Spain

    Antonio J. Tallón-Ballesteros

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zouggar, S.T., Adla, A. (2018). EMnGA: Entropy Measure and Genetic Algorithms Based Method for Heterogeneous Ensembles Selection. In: Yin, H., Camacho, D., Novais, P., Tallón-Ballesteros, A. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2018. IDEAL 2018. Lecture Notes in Computer Science(), vol 11315. Springer, Cham. https://doi.org/10.1007/978-3-030-03496-2_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03496-2_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03495-5

  • Online ISBN: 978-3-030-03496-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation