Abstract
Naive Bayes is a well known and studied algorithm both in statistics and machine learning. Although its limitations with respect to expressive power, this procedure has a surprisingly good performance in a wide variety of domains, including many where there are clear dependencies between attributes. In this paper we address its main perceived limitation - its inability to deal with attribute dependencies. We present Linear Bayes that uses, for the continuous attributes, a multivariate normal distribution to compute the require probabilities. In this way, the interdependencies between the continuous attributes are considered. On the empirical evaluation, we compare Linear Bayes against a naive- Bayes that discretize continuous attributes, a naive-Bayes that assumes a univariate Gaussian for continuous attributes, and a standard Linear discriminant function. We show that Linear Bayes is a plausible algorithm, that competes quite well against other well established techniques.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
C. Blake, E. Keogh, and C.J. Merz. UCI repository of Machine Learning databases, 1999.
Pedro Domingos and Michael Pazzani. On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning,29:103–129, 1997.
J. Dougherty, R. Kohavi, and M. Sahami. Supervised and unsupervised discretization of continuous features. In A. Prieditis and S. Russel, editors, Machine Learning Proc. of 12th International Conference. Morgan Kaufmann, 1995.
R.O. Duda and P.E. Hart. Pattern Classification and Scene Analysis. New York, Willey and Sons, 1973.
Nir Friedman, Moises Goldszmidt, and Thomas J. Lee. Bayesian network classification with continuous features: Getting the best of both discretization and parametric fitting. In Jude Shavlik, editor, Proceedings of the 15th International Conference-ICML’98. Morgan Kaufmann, 1998.
J. Gama. Iterative Bayes. In S. Arikawa and K. Furukawa, editors, Discovery Science-Second International Conference. LNAI 1721, Springer Verlag, 1999.
George John. Enhancements to the data mining process. PhD thesis, Stanford University, 1997.
J. Kittler. Combining classifiers: A theoretical framework. Pattern analysis and Applications, Vol. 1, No. 1, 1998.
R. Kohavi. Scaling up the accuracy of naive Bayes classifiers: a decision tree hybrid. In Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining. AAAI Press, 1996.
Ron Kohavi, B. Becker, and D. Sommerfield. Improving simple Bayes. In Maarten van Someren and Gerhard Widmer, editors, Poster Papers of ECML-97. Charles University, Prague, 1997.
I. Kononenko. Semi-naive Bayesian classifier. In Y. Kodratoff, editor, European Working Session on Learning-EWSL91. LNAI 482 Springer Verlag, 1991.
P. Langley. Induction of recursive Bayesian classifiers. In P. Brazdil, editor, Proc. of European Conf. on Machine Learning. LNAI 667, Springer Verlag, 1993.
Pat Langley. Tractable average-case analysis of naive Bayesian classifiers. In I. Bratko and S. Dzeroski, editors, Machine Learning, Proceedings of the 16th International Conference. Morgan Kaufmann, 1999.
D. Michie, D.J. Spiegelhalter, and C. Taylor. Machine Learning, Neural and Statistical Classification. Ellis Horwood, 1994.
Tom Mitchell. Machine Learning. MacGraw-Hill Companies, Inc., 1997.
M. Pazzani. Constructive induction of cartesian product attributes. In Proc. of the Conference ISIS96: Information, Statistics and Induction in Science, pages 66–77. World Scientific, 1996.
W. Press, S. Teukolsky, W. Vetterling, and B. Flannery. Numerical Recipes in C: the art of scientific computing 2 Ed. University of Cambridge, 1992.
G. Webb and M. Pazzani. Adjusted probability naive Bayesian induction. In 11th Australian Joint Conference on Artificial Intelligence. World Scientific, 1998.
Z. Zheng. Naive Bayesian classifier committees. In C. Nedellec and C. Rouveirol, editors, Proc. of European Conf. on Machine Learning ECML-98. LNAI 1398, Springer Verlag, 1998.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gama, J. (2000). A Linear-Bayes Classifier. In: Monard, M.C., Sichman, J.S. (eds) Advances in Artificial Intelligence. IBERAMIA SBIA 2000 2000. Lecture Notes in Computer Science(), vol 1952. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44399-1_28
Download citation
DOI: https://doi.org/10.1007/3-540-44399-1_28
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-41276-2
Online ISBN: 978-3-540-44399-5
eBook Packages: Springer Book Archive