Advertisement

Ant-Inspired Algorithms for Decision Tree Induction

An Evaluation on Biomedical Signals
  • Miroslav BursaEmail author
  • Lenka Lhotska
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9267)

Abstract

In this paper we present an evaluation of ant-inspired method called ACO_DTree over biomedical data. The algorithm maintains and evolves a population of decision trees induced from data. The core of the algorithm is inspired by the Min-Max Ant System.

In order to increase the speed of the algorithm we have introduced a local optimization phase. The generalization ability has been improved using error based pruning of the solutions.

After parameter tuning, we have conducted experimental evaluation of the ACO_DTree method over the total of 32 different datasets versus 41 distinct classifiers. We conducted 10-fold crossvalidation and for each experiment obtained about 20 quantitative objective measures. The averaged and best-so-far values of the selected measures (precision, recall, f-measure, ...) have been statistically evaluated using Friedman test with Holm and Hochberg post-hoc procedures (on the levels of \(\alpha =0.05\) and \(\alpha =0.10\)). The ACO_DTree algorithm performed significantly better (\(\alpha =0.05\)) in 29 test cases for the averaged f-measure and in 14 cases for the best-so-far f-measure.

The best results have been obtained for various subsets of the UCI database and for the dataset combining cardiotocography data and data of myocardial infarction.

Keywords

Decision Tree Decision Tree Induction Pheromone Evaporation Pheromone Matrix Rule Induction Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgment

This research project has been supported by the project number NT11124-6/2010 “Cardiotocography evaluation by means of artificial intelligence” of the Ministry of Health Care.

The research is supported by the project No. 15-31398A Features of Electromechanical Dyssynchrony that Predict Effect of Cardiac Resynchronization Therapy of the Agency for Health Care Research of the Czech Republic.

And I would like to acknowledge the UCI repository [15] and the relevant donors and creators of the datasets. This breast cancer data (Parp_BRLj) was obtained from the University Medical Centre, Institute of Oncology, Ljubljana, Yugoslavia. Thanks go to M. Zwitter and M. Soklic for providing the data. The blood transfusion data (UCI_bloodt) has been provided by Prof. I-Cheng Yeh [29]. The Breast Cancer Wisconsin dataset (Parp_BRWis) was obtained from the University of Wisconsin Hospitals, Madison from Dr. William H. Wolberg [2]. CTG data originate from Faculty Hospital in Brno, Czech Republic. The CTG database is freely available online [30].

References

  1. 1.
    Barros, R., Basgalupp, M., de Carvalho, A., Freitas, A.: A survey of evolutionary algorithms for decision-tree induction. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 42(3), 291–312 (2012)CrossRefGoogle Scholar
  2. 2.
    Bennett, K., Mangasarian, O.: Robust linear programming discrimination of two linearly inseparable sets. Optim. Methods Softw. 1(1), 23–34 (1992)CrossRefGoogle Scholar
  3. 3.
    Boryczka, U., Kozak, J.: Ant colony decision trees – a new method for constructing decision trees based on ant colony optimization. In: Pan, J.-S., Chen, S.-M., Nguyen, N.T. (eds.) ICCCI 2010, Part I. LNCS, vol. 6421, pp. 373–382. Springer, Heidelberg (2010) Google Scholar
  4. 4.
    Boryczka, U., Kozak, J.: An adaptive discretization in the ACDT algorithm for continuous attributes. In: Jędrzejowicz, P., Nguyen, N.T., Hoang, K. (eds.) ICCCI 2011, Part II. LNCS, vol. 6923, pp. 475–484. Springer, Heidelberg (2011) Google Scholar
  5. 5.
    Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth International Group, Belmont (1984) zbMATHGoogle Scholar
  6. 6.
    Bursa, M., Lhotska, L.: Automated classification tree evolution through hybrid metaheuristics. In: Corchado, E., Corchado, J.M., Abraham, A. (eds.) Innovations in Hybrid Intelligent Systems. AISC, vol. 44, pp. 191–198. Springer, Heidelberg (2007) CrossRefGoogle Scholar
  7. 7.
    Bursa, M., Lhotska, L., Macas, M.: Hybridized swarm metaheuristics for evolutionary random forest generation. In: Proceedings of the 7th International Conference on Hybrid Intelligent Systems 2007, IEEE CSP, pp. 150–155 (2007)Google Scholar
  8. 8.
    Cox, L.A., Quiu, Y., Kuehner, W.: Heuristic least-cost computation of discrete classification functions with uncertain argument values. Ann. Oper. Res. 21(1), 1–30 (1989)MathSciNetCrossRefzbMATHGoogle Scholar
  9. 9.
    Deneubourg, J.L., Aron, S., Goss, S., Pasteels, J.M.: The self-organizing exploratory pattern of the argentine ant. J. Insect Behav. 3, 159–168 (1990)CrossRefGoogle Scholar
  10. 10.
    Dorigo, M., Di Caro, G.D.: New ideas in optimization. In: The Ant Colony Optimization Meta-Heuristic, pp. 11–32. McGraw-Hill (1999)Google Scholar
  11. 11.
    Dorigo, M., Maniezzo, V., Colorni, A.: Ant system: optimization by a colony of cooperating agents. IEEE Trans. Syst. Man Cybern. B 26(1), 29–41 (1996)CrossRefGoogle Scholar
  12. 12.
    Dorigo, M.: Optimization, learning and natural algorithms. Ph.D. thesis, Dipartimento di Elettronica, Politecnico di Milano, Italy (1992)Google Scholar
  13. 13.
    Dorigo, M., Gambardella, L.M.: Ant colony system: a cooperative learning approach to the traveling salesman problem. IEEE Trans. Evol. Comput. 1(1), 53–66 (1997)CrossRefGoogle Scholar
  14. 14.
    Engelbrecht, A.P.: Computational Intelligence: An Introduction, 2nd edn. Wiley, New York (2007) CrossRefGoogle Scholar
  15. 15.
    Frank, A., Asuncion, A.: UCI machine learning repository (2010). http://archive.ics.uci.edu/ml
  16. 16.
    Heath, D., Kasif, S., Salzberg, S.: Learning oblique decision trees. In: Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence IJCAI (1993)Google Scholar
  17. 17.
    Hyafil, L., Rivest, R.L.: Constructing optimal binary decision trees is np-complete. Inf. Process. Lett. 5(1), 15–17 (1976)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Koza, J.R.: Concept formation and decision tree induction using the genetic programming paradigm. In: Schwefel, H.-P., Männer, R. (eds.) Parallel Problem Solving from Nature. LNCS, vol. 496, pp. 124–128. Springer, Heidelberg (1991)CrossRefGoogle Scholar
  19. 19.
    Llor, X., Garrell, J.M.: Automatic classification and artificial life models. In: Proceedings of Learning Workshop (2000)Google Scholar
  20. 20.
    Maniezzo, V., Colorni, A.: The ant system applied to the quadratic assignment problem. IEEE Trans. Knowl. Data Eng. 11(5), 769–778 (1999)CrossRefGoogle Scholar
  21. 21.
    Martens, D., Baesens, B., Fawcett, T.: Editorial survey: swarm intelligence for data mining. Mach. Learn. 82, 1–42 (2011)MathSciNetCrossRefGoogle Scholar
  22. 22.
    Murthy, K.: On Growing Better Decision Trees from Data. Ph.D. thesis, Johns Hopkins University, Baltimore, MD (1997)Google Scholar
  23. 23.
    Otero, F., Freitas, A., Johnson, C.: cAnt-Miner: an ant colony classificationalgorithm to cope with continous attributes. In: CIDM, pp. 225–231 (2009)Google Scholar
  24. 24.
    Otero, F.E.B., Freitas, A.A., Johnson, C.G.: Inducing decision trees with an ant colony optimization algorithm. Appl. Soft Comput. 12(11), 3615–3626 (2012)CrossRefGoogle Scholar
  25. 25.
    Parpinelli, R.S., Lopes, H.S., Freitas, A.A.: Data mining with an ant colony optimization algorithm. IEEE Trans. Evol. Comput. 6(4), 321–332 (2002)CrossRefGoogle Scholar
  26. 26.
    Podgorelec, V., Kokol, P., Stiglic, B., Rozman, I.: Decision trees: an overview and their use in medicine. J. Med. Syst. 26(5), 445–463 (2002)CrossRefGoogle Scholar
  27. 27.
    Rokach, L., Maimon, O.: Top-down induction of decision trees classifiers - a survey. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 1(11), 1–12 (2002)Google Scholar
  28. 28.
    Stutzle, T., Hoos, H.: Max-min ant system. Future Gener. Comput. Syst. 16(8), 889–914 (2000)CrossRefGoogle Scholar
  29. 29.
    Yeh, I.C., Yang, K.J., Ting, T.M.: Knowledge discovery on RFM model using Bernoulli sequence. Expert Syst. Appl. 36(3), 5866–5871 (2009). http://dx.doi.org/10.1016/j.eswa.2008.07.018 CrossRefGoogle Scholar
  30. 30.
    Chudacek, V., Spilka, J., Bursa, M., Janku, P., Hruban, L., Huptych, M., Lhotska, L.: Open access intrapartum ctg database. BMC Pregnancy Childbirth 14, 16 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  1. 1.Czech Institute of Informatics, Robotics and CyberneticsCzech Technical University in PraguePragueCzech Republic
  2. 2.Department of Cybernetics, Faculty of Electrical EngineeringCzech Technical University in PraguePragueCzech Republic

Personalised recommendations