Abstract
We propose a novel generative model for classification called latent tree classifier (LTC). An LTC represents each class-conditional distribution of attributes using a latent tree model, and uses Bayes rule to make prediction. Latent tree models can capture complex relationship among attributes. Therefore, LTC can approximate the true distribution behind data well and thus achieve good classification accuracy. We present an algorithm for learning LTC and empirically evaluate it on 37 UCI data sets. The results show that LTC compares favorably to the state-of-the-art. We also demonstrate that LTC can reveal underlying concepts and discover interesting subgroups within each class.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Akaike, H.: A new look at the statistical model identification. IEEE T. Automat. Contr. 19(6), 716–723 (1974)
Asuncion, A., Newman, D.J.: UCI machine learning repository (2007)
Chen, T., Zhang, N.L., Wang, Y.: Efficient model evaluation in the search-based approach to latent structure discovery. In: 4th European Workshop on Probabilistic Graphical Models, pp. 57–64 (2008)
Chickering, D.M., Heckerman, D.: Efficient approximations for the marginal likelihood of Bayesian networks with hidden variables. Mach. Learn. 29, 181–212 (1997)
Chow, C.K., Liu, C.N.: Approximating discrete probability distributions with dependence trees. IEEE T. Inform. Theory 14(3), 462–467 (1968)
Dempster, A.P., Laird, N.M., Rubin, D.R.: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc. B Met. 39(1), 1–38 (1977)
Domingos, P., Pazzani, M.: On the optimality of the simple Bayesian classifier under zero-one loss. Mach. Learn. 29, 103–130 (1997)
Duda, R.O., Hart, P.E.: Pattern Classification and Scene Analysis. Wiley, Chichester (1973)
Fayyad, U.M., Irani, K.B.: Multi-interval discretization of continuous-valued attributes for classification learning. In: 13th International Joint Conference on Artificial Intelligence, pp. 1022–1027 (1993)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29(2-3), 131–163 (1997)
John, G.H., Kohavi, R., Pfleger, K.: Irrelevant features and the subset selection problem. In: 11th International Conference on Machine Learning, pp. 121–129 (1994)
Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: 14th International Joint Conference on Artificial Intelligence, pp. 1137–1145 (1995)
Langseth, H., Nielsen, T.D.: Latent classification models. Mach. Learn. 59(3), 237–265 (2005)
Langseth, H., Nielsen, T.D.: Classification using hierarchical naive Bayes models. Mach. Learn. 63(2), 135–159 (2006)
Langseth, H., Nielsen, T.D.: Latent classification models for binary data. Pattern Recogn. 42, 2724–2736 (2009)
Monti, S., Cooper, G.F.: A Bayesian network classifier that combines a finite mixture model and a naive Bayes model. In: 11th Annual Conference on Uncertainty in Artificial Intelligence, pp. 447–456 (1995)
Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Rubinstein, Y.D., Hastie, T.: Discriminative vs informative learning. In: 3rd International Conference on Knowledge Discovery and Data Mining, pp. 49–53 (1997)
Wang, Y.: Latent Tree Models for Multivariate Density Estimation: Algorithms and Applications. Ph.D. thesis, Hong Kong University of Science & Technology (2009)
Webb, G.I., Boughton, J.R., Wang, Z.: Not so naive Bayes: Aggregating one-dependence estimators. Mach. Learn. 58, 5–24 (2005)
Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann, San Francisco (2005)
Zhang, N.L.: Hierarchical latent class models for cluster analysis. Journal of Mach. Learn. Research 5(6), 697–723 (2004)
Zhang, N.L., Nielsen, T.D., Jensen, F.V.: Latent variable discovery in classification models. Artif. Intell. Med. 30(3), 283–299 (2004)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Wang, Y., Zhang, N.L., Chen, T., Poon, L.K.M. (2011). Latent Tree Classifier. In: Liu, W. (eds) Symbolic and Quantitative Approaches to Reasoning with Uncertainty. ECSQARU 2011. Lecture Notes in Computer Science(), vol 6717. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-22152-1_35
Download citation
DOI: https://doi.org/10.1007/978-3-642-22152-1_35
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-22151-4
Online ISBN: 978-3-642-22152-1
eBook Packages: Computer ScienceComputer Science (R0)