A Hybrid Evolutionary Algorithm for Bayesian Networks Learning: An Application to Classifier Combination

  • Claudio De Stefano
  • Francesco Fontanella
  • Cristina Marrocco
  • Alessandra Scotto di Freca
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6024)


Classifier combination methods have shown their effectiveness in a number of applications. Nonetheless, using simultaneously multiple classifiers may result in some cases in a reduction of the overall performance, since the responses provided by some of the experts may generate consensus on a wrong decision even if other experts provided the correct one. To reduce these undesired effects, in a previous paper, we proposed a combining method based on the use of a Bayesian Network. The structure of the Bayesian Network was learned by using an Evolutionary Algorithm which uses a specifically devised data structure to encode Direct Acyclic Graphs. In this paper we presents a further improvement along this direction, in that we have developed a new hybrid evolutionary algorithm in which the exploration of the search space has been improved by using a measure of the statistical dependencies among the experts. Moreover, new genetic operators have been defined that allow a more effective exploitation of the solutions in the evolving population. The experimental results, obtained by using two standard databases, confirmed the effectiveness of the method.


Mutual Information Bayesian Network Direct Acyclic Graph Statistical Dependency Genetic Operator 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Chickering, D.M., Geiger, D., Heckerman, D.: Learning bayesian networks is np-hard. Tech. rep. (1994)Google Scholar
  2. 2.
    Cooper, G.F., Herskovits, E.: A bayesian method for the induction of probabilistic networks from data. Machine Learning 9(4), 309–347 (1992)zbMATHGoogle Scholar
  3. 3.
    Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley Series in Telecommunications and Signal Processing. Wiley-Interscience, Hoboken (2006)zbMATHGoogle Scholar
  4. 4.
    De Stefano, C., D’Elia, C., Marcelli, A., Scotto di Freca, A.: Classifier combination by bayesian networks for handwriting recognition. International Journal of Pattern Recognition and Artificial Intelligence 23(5), 887–905 (2009)CrossRefGoogle Scholar
  5. 5.
    De Stefano, C., Fontanella, F., Marcelli, A., Scotto di Freca, A.: Learning bayesian networks by evolution for classifier combination. In: ICDAR 2009: Proceedings of the 2009 10th International Conference on Document Analysis and Recognition, pp. 966–970. IEEE Computer Society, Los Alamitos (2009)Google Scholar
  6. 6.
    Dietterich, T.G.: Ensemble Methods in Machine Learning. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 1–15. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  7. 7.
    Heckerman, D.: A tutorial on learning with bayesian networks. Tech. rep., Learning in Graphical Models (1995)Google Scholar
  8. 8.
    Ho, T.K., Hull, J.J., Srihari, S.N.: Decision combination in multiple classifier systems. IEEE Trans. Pattern Anal. Mach. Intell. 16(1), 66–75 (1994)CrossRefGoogle Scholar
  9. 9.
    Kittler, J., Hatef, M., Duin, R.P.W., Matas, J.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20(3), 226–239 (1998)CrossRefGoogle Scholar
  10. 10.
    Kohonen, T.: Self organizing map. Springer, Berlin (1995)Google Scholar
  11. 11.
    Kuncheva, L.I.: Combining Pattern Classifiers: Methods and Algorithms. Wiley-Interscience, Hoboken (2004)zbMATHCrossRefGoogle Scholar
  12. 12.
    Larranaga, P., Poza, M., Yurramendi, Y., Murga, R.H., Kuijpers, C.M.: Structure learning of bayesian networks by genetic algorithms: A performance analysis of control parameters. IEEE Transactions on Pattern Analysis and Machine Intelligence 18(9), 912–926 (1996)CrossRefGoogle Scholar
  13. 13.
    Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)Google Scholar
  14. 14.
    Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)CrossRefGoogle Scholar
  15. 15.
    Wong, M.L., Leung, K.S.: An efficient data mining method for learning bayesian networks using an evolutionary algorithm-based hybrid approach. IEEE Trans. Evolutionary Computation 8(4), 378–404 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Claudio De Stefano
    • 1
  • Francesco Fontanella
    • 1
  • Cristina Marrocco
    • 1
  • Alessandra Scotto di Freca
    • 1
  1. 1.Università di CassinoCassinoItaly

Personalised recommendations