Abstract
Discriminative learning of Bayesian network classifiers has recently received considerable attention from the machine learning community. This interest has yielded several publications where new methods for the discriminative learning of both structure and parameters have been proposed. In this paper we present an empirical study used to illustrate how discriminative learning performs with respect to generative learning using simple Bayesian network classifiers such as naive Bayes or TAN, and we discuss when and why a discriminative learning is preferred. We also analyzed how log-likelihood and conditional log-likelihood scores guide the learning process of Bayesian network classifiers.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco (1988)
Neapolitan, R.E.: Learning Bayesian Networks. Prentice-Hall, Englewood Cliffs (2003)
Larrañaga, P., Lozano, J.A., Peña, J.M., Inza, I.: Editorial. Machine Learning 59(3), 211–212 (2005)
Duda, R., Hart, P.: Pattern Classification and Scene Analysis. John Wiley and Sons, Chichester (1973)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian networks classifiers. Machine Learning 29, 131–163 (1979)
Sahami, M.: Learning limited dependence Bayesian classifiers. In: Proceedings of the Second International Conference on Knowledge Discovery and Data Mining, pp. 335–338 (1996)
Cheng, J., Greiner, R.: Learning Bayesian belief network classifiers: Algorithms and system. In: Proceedings of the Fourteenth Biennial Conference of the Canadian Society on Computational Studies of Intelligence: Advances in Artificial Intelligence, pp. 141–152 (2001)
Greiner, R., Zhou, W., Su, X., Shen, B.: Structural extension to logistic regression: Discriminative parameter learning of belief net classifiers. Machine Learning 59(3), 297–322 (2005)
Roos, T., Wettig, H., Grünwald, P., Myllymäki, P., Tirri, H.: On discriminative Bayesian network classifiers and logistic regression. Machine Learning 59(3), 267–296 (2005)
Feelders, A., Ivanovs, J.: Discriminative scoring of Bayesian network classifiers: A comparative study. In: Proceedings of the Third European Workshop on Probabilistic Graphical Models, pp. 75–82 (2006)
Santafé, G., Lozano, J.A., Larrañaga, P.: Discriminative learning of Bayesian network classifiers via the TM algorithm. In: Proceedings of the Eighth European Conference on Symbolic and Quantitative Approaches to Reasonig with Uncertainty, pp. 148–160 (2005)
Grossman, D., Domingos, P.: Learning Bayesian network classifiers by maximizing conditional likelihood. In: Proceedings of the Twentyfirst International Conference on Machine Learning, pp. 361–368 (2004)
Santafé, G., Lozano, J.A., Larrañaga, P.: Aprendizaje discriminativo de clasificadores Bayesianos. Revista Iberoamericana de Inteligencia Artificial (in Spanish) 29(10), 39–47 (2006)
Guo, Y., Greiner, R.: Discriminative model selection for belief net structures. In: Proceedings of the Twentieth National Conference on Artificial Intelligence (2005)
Pernkopf, F., Bilmes, J.: Discriminative versus generative parameter and structure learning of Bayesian network classifiers. In: Proceedings of the Twenty-Second International Conference on Machine Learning, pp. 657–664 (2005)
Rubinstein, Y.D., Hastie, T.: Discriminative vs. informative learning. In: Proceedings of the Third International Conference on Knowledge Discovery and Data Mining, pp. 49–53 (1997)
Vapnik, V.: Statistical Learning Theory. John Wiley and Sons, Chichester (1998)
Ng, A.Y., Jordan, M.I.: On discriminative vs. generative classifiers: A comparison of logistic regression and naïve Bayes. In: Proceedings of the Sixteenth Advances in Neural Information Processing Systems, vol. 14 (2002)
Bilmes, J.: Dynamic Bayesian multinets. In: Proceedings of the Sixteenth International Conference of Uncertainty in Artificial Intelligence, pp. 38–45 (2000)
Perez, A., Larranaga, P., Inza, I.: Information theory and classification error in probabilistic classifiers. In: Todorovski, L., Lavrač, N., Jantke, K.P. (eds.) DS 2006. LNCS (LNAI), vol. 4265, pp. 347–351. Springer, Heidelberg (2006)
Edwards, D., Lauritzen, S.L.: The TM algorithm for maximising a conditional likelihood function. Biometrika 88(4), 961–972 (2001)
Cover, T.M., Thomas, J.A.: Elements of Information Theory, 2nd edn. John Wiley & Sons, Chichester (2006)
Blake, C., Merz, C.: UCI repository of machine learning databases (1998), http://www.ics.uci.edu/~mlearn
Fayyad, U., Irani, K.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, pp. 1022–1027 (1993)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Santafé, G., Lozano, J.A., Larrañaga, P. (2007). Discriminative vs. Generative Learning of Bayesian Network Classifiers. In: Mellouli, K. (eds) Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2007. Lecture Notes in Computer Science(), vol 4724. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75256-1_41
Download citation
DOI: https://doi.org/10.1007/978-3-540-75256-1_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-75255-4
Online ISBN: 978-3-540-75256-1
eBook Packages: Computer ScienceComputer Science (R0)