Abstract
This paper investigates the methods for learning predictive classifiers based on Bayesian belief networks (BN) — primarily unrestricted Bayesian networks and Bayesian multi-nets. We present our algorithms for learning these classifiers, and discuss how these methods address the overfitting problem and provide a natural method for feature subset selection. Using a set of standard classification problems, we empirically evaluate the performance of various BN-based classifiers. The results show that the proposed BN and Bayes multinet classifiers are competitive with (or superior to) the best known classifiers, based on both BN and other formalisms; and that the computational time for learning and using these classifiers is relatively small. These results argue that BN-based classifiers deserve more attention in the data mining community.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Cheng, J., Bell, D.A. and Liu, W. (1997a). An algorithm for Bayesian belief network construction from data. In Proceedings of AI & STAT’97 (pp.83–90), Florida.
Cheng, J., Bell, D.A. and Liu, W. (1997b). Learning belief networks from data: An information theory based approach. In Proceedings of ACM CIKM’97.
Cheng, J. (1998). PowerConstructor System. http://www.cs.ualberta.ca/~jcheng/bnpc.htm.
Cheng, J. (2000). PowerPredictor System. http://www.cs.ualberta.ca/~jcheng/bnpp.htm.
Cheng, J., Greiner, R. (1999). Comparing Bayesian network classifiers. In UAI-99.
Chow, C.K. and Liu, C.N. (1968). Approximating discrete probability distributions with dependence trees. IEEE Trans. on Information Theory, 14 (pp. 462–467).
Cooper, G.F. (1990) Computational complexity of probabilistic inference using Bayesian belief networks, In Artificial Intelligence, 42 (pp. 393–405).
Cooper, G.F. and Herskovits, E. (1992). A Bayesian Method for the induction of probabilistic networks from data. Machine Learning, 9 (pp. 309–347).
Duda, R. and Hart, P. (1973). Pattern classification and scene analysis. John Wiley & Sons.
Friedman, N., Geiger, D. and Goldszmidt, M. (1997). Bayesian Network Classifiers. Machine Learning, 29, (pp. 131–161).
Geiger, D. and Heckerman, D. (1996). Knowledge representation and inference in similarity networks and Bayesian multinets. In Artificial Intelligence 82 (pp. 45–74).
Greiner, R. Grove, A. and Schuurmans, D. (1997). Learning Bayesian nets that perform well. In UAI-97.
Heckerman, D. (1995). A tutorial on learning Bayesian networks. Technical Report MSRTR-95-06. Microsoft Research.
Kohavi, R., John, G., Long, R. Manley, D. and Pfleger, K. (1994). MLC++: A machine learning library in C++. In Proceedings of Sixth International Conference on Tools with Artificial Intelligence. IEEE Computer Society.
Kohavi, R., John G. (1997) Wrappers for Feature Subset Selection. In Artificial Intelligence journal, special issue on relevance, Vol. 97, No. 1-2 (pp. 273–324).
Kononenko, I. (1991). Semi-naïve Bayesian classifier. In Y. Kodratoff (Ed.), Proceedings of sixth European working session on learning (pp.206–219). Springer-Verlag.
Langley, P., Iba, W. and Thompson, K. (1992). An analysis of Bayesian classifiers. In Proceedings of AAAI-92 (pp. 223–228).
Langley, P. and Sage, S. (1994). Induction of Selective Bayesian lassifiers. In UAI-94.
Murphy, P.M. and Aha, D.W. (1995). UCI repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html.
Neapolitan, R.E. (1990), Probabilistic reasoning in expert systems: theory and algorithms, John Wiley & Sons.
Pazzani, M.J. (1995). Searching for dependencies in Bayesian classifiers. In AI & STAT’95.
Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: networks of plausible inference, Morgan Kaufmann.
Spirtes, P., Glymour, C. and Scheines, R. (1993). Causation, Prediction, and Search. http://hss.cmu.edu/html/departments/philosophy/TETRAD.BOOK/book.html.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2001 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Cheng, J., Greiner, R. (2001). Learning Bayesian Belief Network Classifiers: Algorithms and System. In: Stroulia, E., Matwin, S. (eds) Advances in Artificial Intelligence. Canadian AI 2001. Lecture Notes in Computer Science(), vol 2056. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45153-6_14
Download citation
DOI: https://doi.org/10.1007/3-540-45153-6_14
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-42144-3
Online ISBN: 978-3-540-45153-2
eBook Packages: Springer Book Archive