Several Computational Studies About Variable Selection for Probabilistic Bayesian Classifiers

  • Adriana BroginiEmail author
  • Debora Slanzi
Conference paper
Part of the Studies in Classification, Data Analysis, and Knowledge Organization book series (STUDIES CLASS)


The Bayesian network can be considered as a probabilistic classifier with the ability of giving a clear insight into the structural relationships in the domain under investigation. In this paper we use some methodologies of feature subset selection in order to determine the relevant variables which are then used for constructing the Bayesian network. To test how the selected methods of feature selection affect the classification, we consider several Bayesian classifiers: Naïve Bayes, Tree Augmented Naïve Bayes and the general Bayesian network, which is used as benchmark for the comparison.


  1. Aliferis, C. F., Tsamardinos, I., & Statnikov, A. (2003). HITON: A novel Markov blanket algorithm for optimal variable selection. In Proceedings of the 2003 American Medical Informatics Association (AMIA) Annual Symposium (pp. 21–25).Google Scholar
  2. Cheng, J., & Greiner, R. (1999). Comparing Bayesian network classifiers. In Proceedings UAI-99.Google Scholar
  3. Cooper, G. F., & Herskovits, E. (1992). A Bayesian method for the induction of probabilistic networks from data. Machine Learning, 9(4), 309–348.zbMATHGoogle Scholar
  4. Frey, L., Fisher, D., Tsamardinos, I., Aliferis, C. F., & Statnikov, A. (2003). Identifying Markov blankets with decision tree induction. In Proceedings of third IEEE International Conference on Data Mining (ICDM) (pp. 59–66).Google Scholar
  5. Friedman, N., Geiger, D., & Goldszmidt, M. (1997). Bayesian network classifiers. Machine Learning, 29, 131–161.zbMATHCrossRefGoogle Scholar
  6. Goldberg, D. E. (1989). Genetic algorithms in search, optimization and machine learning. Reading, MA: Addison-Wesley.zbMATHGoogle Scholar
  7. Heckerman, D. (1999). A tutorial on learning Bayesian networks. In Learning graphical models. Cambridge, MA: MIT Press.Google Scholar
  8. Heckerman, D., Geiger, D., & Chickering, D. M. (1995). Learning Bayesian networks: The combinations of knowledge and statistical data. Machine Learning, 20, 197–243.zbMATHGoogle Scholar
  9. Kohavi, R., & George, H. J. (1997). Wrappers for feature subset selection. Artificial Intelligence, 1(2), 273–324.CrossRefGoogle Scholar
  10. Langley, P., Iba, W., & Thompson, K. (1992). An analysis of Bayesian classifiers. In Proceedings of AAAI-92 (pp. 223–228).Google Scholar
  11. Lauritzen, S. L. (1996). Graphical models. Oxford: Clarendon Press.Google Scholar
  12. Madden, M. G. (2003). The performance of Bayesian network classifiers constructed using different techniques. In Working notes of the ECML PkDD-03 workshop (pp. 59–70).Google Scholar
  13. Margaritis, D., & Thrun, S. (1999). Bayesian network induction via local neighborhoods. In Proceedings of conference on Neural Information Processing Systems (NIPS-12), MIT Press.Google Scholar
  14. Meek, C. (1997). Graphical models: Selecting causal and statistical models. Ph.D. Thesis, Carnegie Mellon University.Google Scholar
  15. Mitchell, M. (1996). An introduction to genetic algorithms. Cambridge, MA: MIT Press.Google Scholar
  16. Nadeau, C., & Bengio, Y. (2000). Inference for the generalization error. Advances in Neural Information Processing Systems, 12, 293–281.Google Scholar
  17. Neapolitan, R. E. (1990). Probabilistic reasoning in expert systems: Theory and algorithms. New York: Wiley.Google Scholar
  18. Pearl, J. (1988). Probabilistic reasoning in intelligence systems. Los Altos, CA: Morgan Kaufmann.Google Scholar
  19. Quinlan, J. R. (1993). C4.5: Programs for machine learning. Los Altos, CA: Morgan Kaufmann.Google Scholar
  20. Saeys, Y., Inza I., & Larrañaga, P. (2007). A review of feature selection techniques in bioinformatics. Bioinformatics, 23(19), 2507–2517.CrossRefGoogle Scholar
  21. Spirtes, S., Glymour, C., & Scheines, R. (1993). Causation, prediction and search. Berlin: Springer.zbMATHGoogle Scholar
  22. Spirtes, P., Glymour, C., & Scheines, R. (2000). Causation, prediction, and search. New York: MIT Press.Google Scholar
  23. Tsamardinos, I., & Aliferis, C. F. (2003). Towards principled feature selection: Relevancy, filters and wrappers. In Proceedings of the ninth international workshop on Artificial Intelligence and Statistics.Google Scholar
  24. Tsamardinos, I., Aliferis, C., & Statnikov, A. (2003). Algorithms for large scale Markov blanket discovery. In Proceeding of the sixteenth international FLAIRS conference.Google Scholar
  25. WEKA. (2004). On-line documentation. Waikato University, New Zeland. Retrieved from http//

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  1. 1.Department of StatisticsUniversity of PadovaPadovaItaly

Personalised recommendations