Probalistic Networks and Fuzzy Clustering as Generalizations of Naive Bayes Classifiers

  • Christian Borgelt
  • Heiko Timm
  • Rudolf Kruse
Part of the Advances in Soft Computing book series (AINSC, volume 8)


Although at first sight probabilistic networks and fuzzy clustering seem to be disparate areas of research, a closer look reveals that they can both be seen as generalizations of naive Bayes classifiers. If all attributes are numeric (except the class attribute, of course), naive Bayes classifiers often assume an axis-parallel multidimensional normal distribution for each class as the underlying model. Probabilistic networks remove the requirement that the distributions must be axis-parallel by taking the covariance of the attributes into account, where this is necessary. Fuzzy clustering is an unsupervised method that tries to find general or axis-parallel distributions to cluster the data. Although it does not take into account the class information, it can be used to improve the result of naive Bayes classifiers and probabilistic networks by removing the restriction that there can be only one distribution per class.


Bayesian Network Fuzzy Cluster Membership Degree Class Attribute Multivariate Normal Distribution 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    S.K. Andersen, K.G. Olesen, F.V. Jensen, and F. Jensen. HUGIN — A Shell for Building Bayesian Belief Universes for Expert Systems. Proc. 11th Int. J. Conf. on Artificial Intelligence (IJCAI’89, Detroit, MI, USA), 1080–1085. Morgan Kaufman, San Mateo, CA, USA 1989Google Scholar
  2. 2.
    M.J.A. Berry and G. Linoff. Data Mining Techniques — For Marketing, Sales and Customer Support. J. Wiley and Sons, Chichester, England 1997Google Scholar
  3. 3.
    J.C. Bezdek. Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press, New York, NY, USA 1981MATHCrossRefGoogle Scholar
  4. 4.
    J.C. Bezdek and S.K. Pal. Fuzzy Models for Pattern Recognition — Methods that Search for Structures in Data. IEEE Press, Piscataway, NJ, USA 1992Google Scholar
  5. 5.
    J. Dougherty, R. Kohavi, and M. Sahami. Supervised and Unsupervised Discretization of Continuous Features. Proc. 12th Int. Conf. on Machine Learning (ICML ‘85, Lake Tahoe, CA, USA), 194–202. Morgan Kaufman, San Mateo, CA, USA 1995Google Scholar
  6. 6.
    R.O. Duda and P.E. Hart. Pattern Classification and Scene Analysis. J. Wiley and Sons, New York, NY, USA 1973MATHGoogle Scholar
  7. 7.
    K.J. Ezawa and S.W. Norton. Knowledge Discovery in Telecommunication Services Data Using Bayesian Network Models. Proc. 1st Int. Conf. on Knowledge Discovery and Data Mining (KDD’95, Montreal, Canada), 100–105. AAAI Press, Menlo Park, CA, USA 1995Google Scholar
  8. 8.
    I. Gath and A.B. Geva. Unsupervised Optimal Fuzzy Clustering. IEEE Trans. Pattern Anal. Mach. Intelligence 11:773–781. IEEE Press, Piscataway, NJ, USA, 1989Google Scholar
  9. 9.
    J. Gebhardt and R. Kruse. The Context Model — An Integrating View of Vagueness and Uncertainty Int. Journal of Approximate Reasoning 9:283–314. North-Holland, Amsterdam, Netherlands 1993Google Scholar
  10. 10.
    J. Gebhardt and R. Kruse. POSSINFER — A Software Tool for Possibilistic Inference. In: D. Dubois, H. Prade, and R. Yager, eds. Fuzzy Set Methods in Information Engineering: A Guided Tour of Applications, 407–418. J. Wiley and Sons, New York, NY, USA 1996Google Scholar
  11. 11.
    I.J. Good. The Estimation of Probabilities: An Essay on Modern Bayesian Methods. MIT Press, Cambridge, MA, USA 1965Google Scholar
  12. 12.
    D. Heckerman. Probabilistic Similarity Networks. MIT Press, Cambridge, MA, USA 1991Google Scholar
  13. 13.
    F. Höppner, F. Klawonn, R. Kruse, and T. Runkler. Fuzzy Cluster Analysis. J. Wiley and Sons, Chichester, England 1999MATHGoogle Scholar
  14. 14.
    R. Kruse, E. Schwecke, and J. Heinsohn. Uncertainty and Vagueness in Knowledge-based Systems: Numerical Methods (Series Artificial Intelligence). Springer, Berlin, Germany 1991Google Scholar
  15. 15.
    R. Kruse, J. Gebhardt, and F. Klawonn. Foundations of Fuzzy Systems. J. Wiley and Sons, Chichester, England 1994Google Scholar
  16. 16.
    P. Langley, W. Iba, and K. Thompson. An Analysis of Bayesian Classifiers. Proc. 10th Nat. Conf. on Artificial Intelligence (AAAI’92, San Jose, CA, USA), 223–228. AAAI Press and MIT Press, Menlo Park and Cambridge, CA, USA 1992Google Scholar
  17. 17.
    P. Langley and S. Sage. Induction of Selective Bayesian Classifiers. Proc. 10th Conf. on Uncertainty in Artificial Intelligence (UAI’94, Seattle, WA, USA), 399406. Morgan Kaufman, San Mateo, CA, USA 1994Google Scholar
  18. 18.
    S.L. Lauritzen and D.J. Spiegelhalter. Local Computations with Probabilities on Graphical Structures and Their Application to Expert Systems. Journal of the Royal Statistical Society, Series B, 2 (50): 157–224. Blackwell, Oxford, United Kingdom 1988MathSciNetGoogle Scholar
  19. 19.
    D. Nauck, F. Klawonn, and R. Kruse. Foundations of Neuro-Fuzzy Systems. J. Wiley and Sons, Chichester, England 1997Google Scholar
  20. 20.
    J. Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (2nd edition). Morgan Kaufman, San Mateo, CA, USA 1992Google Scholar
  21. 21.
    A. Saffiotti and E. Umkehrer. PULCINELLA: A General Tool for Propagating Uncertainty in Valuation Networks. Proc. 7th Conf. on Uncertainty in Artificial Intelligence (UAI’91, Los Angeles, CA, USA), 323–331. Morgan Kaufman, San Mateo, CA, USA 1991Google Scholar
  22. 22.
    G. Shafer and P.P. Shenoy. Local Computations in Hypertrees (Working Paper 201). School of Business, University of Kansas, Lawrence, KS, USA 1988Google Scholar
  23. 23.
    P.P. Shenoy. Valuation-based Systems: A Framework for Managing Uncertainty in Expert Systems (Working Paper 226). School of Business, University of Kansas, Lawrence, KS, USA 1991Google Scholar
  24. 24.
    J. Whittaker. Graphical Models in Applied Multivariate Statistics. J. Wiley and Sons, Chichester, England 1990MATHGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Christian Borgelt
    • 1
  • Heiko Timm
    • 1
  • Rudolf Kruse
    • 1
  1. 1.Dept. of Kowledge Processing and Language EngineeringOtto-von-Guericke-University of MagdeburgMagdeburgGermany

Personalised recommendations