Abstract
Over a decade ago, Friedmanet al. introduced the Tree Augmented Naïve Bayes (TAN) classifier, with experiments indicating that it significantly outperformed Naïve Bayes (NB) in terms of classification accuracy, whereas general Bayesian network (GBN) classifiers performed no better than NB. This paper challenges those claims, using a careful experimental analysis to show that GBN classifiers significantly outperform NB on datasets analyzed, and are comparable to TAN performance. It is found that the poor performance reported by Friedman et al. are not attributable to the GBN per se, but rather to their use of simple empirical frequencies to estimate GBN parameters, whereas basic parameter smoothing (used in their TAN analyses but not their GBN analyses) improves GBN performance significantly. It is concluded that, while GBN classifiers may have some limitations, they deserve greater attention, particularly in domains where insight into classification decisions, as well as good accuracy, is required.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Baesens, B., Egmont-Petersen, M., Castelo, R. and Vanthienen, J. (2002) Learning Bayesian network classifiers for credit scoring using Markov Chain Monte Carlo search. Proc. 2002 International Congress on Pattern Recognition, IEEE Computer Society.
Asuncion, A. & Newman, D.J. (2007). UCI Machine Learning Repository. http://www.ics.uci.edu/~mlearn/MLRepository.html. University of California, Irvine.
Bouckaert, R.R. (2004): Bayesian networks in Weka. Technical Report 14/2004. Computer Science Department. University of Waikato.
Bouckaert, R.R. (2004): Estimating Replicability of Classifier Learning Experiments. Proc. 21st International Conference on Machine Learning.
Buntine, W.(1991). Theory Refinement on Bayesian Networks. Proc. 7th International Conference on Uncertainty in Artificial Intelligence.
Cerquides, J. and de Mántaras, R. (2005). TAN Classifiers Based on Decomposable Distributions. Machine Learning Vol. 59, pp 323–354.
Cheng, J. and Greiner, R. (2001). Learning Bayesian Belief Network Classifiers: Algorithms and System. Proc. 14th Canadian Conference on Artificial Intelligence.
Cheng, J., Greiner, R., Kelly, J., Bell, D. and Liu, W. (2002). Learning Belief Networks from Data: An Information Theory Based Approach. Artificial Intelligence, Vol. 137, pp 43–90.
Chickering, D.M. (2002). Optimal Structure Identification with Greedy Search. Journal of Machine Learning Research, Vol. 3, pp 507–554.
Chickering, D.M. and Meek, C. (2006). On the Incompatibility of Faithfulness and Monotone DAG Faithfulness. Artificial Intelligence, Vol. 170, pp 653–666.
Cooper, G.F. (1990). The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks. Artificial Intelligence, Vol. 42, pp 393–405.
Cooper, G.F. and Herskovits, E. (1992). A Bayesian Method for the Induction of Probabilistic Networks from Data. Machine Learning, Vol. 9, pp 309–347. Kluwer Academic Publishers.
Domingos, P. & Pazzani, M. (1996). Beyond Independence: Conditions for the Optimality of the Simple Bayesian Classifier. Proc. 13th International Conference on Machine Learning.
Domingos, P. & Pazzani, M. (1997). On the Optimality of the Simple Bayesian Classifier under Zero-One Loss. Machine Learning, Vol. 29. Kluwer Academic Publishers.
Dougherty, J., Kohavi, R. and Sahami, M. (1995). Supervised and Unsupervised Discretization of Continuous Features. Proc. 12th International Conference on Machine Learning.
Friedman, N., Geiger, D. and Goldszmidt, M. (1997). Bayesian Network Classifiers. Machine Learning, Vol. 29, pp 131–163. Kluwer Academic Publishers, Boston.
Garg, A. and Roth, D. (2001) Understanding Probabilistic Classifiers. Proc.12th European Conference on Machine Learning.
Grossman, D. and Domingos, P. (2004). Learning Bayesian Network Classifiers by Maximizing Conditional Likelihood. Proc. 21st International Conference on Machine Learning.
Heckerman, D., Geiger, D. and Chickering, D.M. (1995). Learning Bayesian Networks: The Combination of Knowledge and Statistical Data. Machine Learning, Vol. 20, pp 197–243.
Keogh, E. and Pazzani, M.J. (2002). Learning the Structure of Augmented Bayesian Classifiers. Internationa/Journal on Artificial Intelligence Tools. Vol. 11, No. 4, pp 587–601.
Kohavi, R., Sommerfield, D. and Dougherty, J. (1997). Data Mining using MLC++. International Journal on Artificial Intelligence Tools. Vol. 6, No. 4, pp 537–566.
Ling, C.X. and Zhang, H. (2002). The Representational Power of Discrete Bayesian Networks. Journal of Machine Learning Research, Vol. 3.
Madden, M.G. (2003). ThePerformance of Bayesian Network Classifiers Constructed using Different Techniques. Proc. European Conference on Machine Learning, Workshopon Probabilistic Graphical Models for Classification.
Nadeau, C. and Bengio, Y. (2000). Inference for the generalization error. Advances in Neural Information Processing Systems 12, MIT Press.
Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco.
Silander, T. and Myllymäki, P. (2006). A Simple Approach for Finding the Globally Optimal Bayesian Network Structure. Proc. 22nd Conference on Uncertainty in Artificial Intelligence.
Zhang, H. and Ling, C.X. (2001) An improved learning algorithm for augmented Naive Bayes. Proc. Fifth Pacific-Asia Conferenceon Knowledge Discovery in Databases.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag London Limited
About this paper
Cite this paper
Madden, M.G. (2009). On the Classification Performance of TAN and General Bayesian Networks. In: Bramer, M., Petridis, M., Coenen, F. (eds) Research and Development in Intelligent Systems XXV. SGAI 2008. Springer, London. https://doi.org/10.1007/978-1-84882-171-2_1
Download citation
DOI: https://doi.org/10.1007/978-1-84882-171-2_1
Publisher Name: Springer, London
Print ISBN: 978-1-84882-170-5
Online ISBN: 978-1-84882-171-2
eBook Packages: Computer ScienceComputer Science (R0)