Multivariate Cluster-Based Discretization for Bayesian Network Structure Learning

  • Ahmed Mabrouk
  • Christophe GonzalesEmail author
  • Karine Jabet-Chevalier
  • Eric Chojnaki
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9310)


While there exist many efficient algorithms in the literature for learning Bayesian networks with discrete random variables, learning when some variables are discrete and others are continuous is still an issue. A common way to tackle this problem is to preprocess datasets by first discretizing continuous variables and, then, resorting to classical discrete variable-based learning algorithms. However, such a method is inefficient because the conditional dependences/arcs learnt during the learning phase bring valuable information that cannot be exploited by the discretization algorithm, thereby preventing it to be fully effective In this paper, we advocate to discretize while learning and we propose a new multivariate discretization algorithm that takes into account all the conditional dependences/arcs learnt so far. Unlike popular discretization methods, ours does not rely on entropy but on clustering using an EM scheme based on a Gaussian mixture model. Experiments show that our method significantly outperforms the state-of-the-art algorithms.


Multivariate discretization Bayesian network learning 



This work was supported by the French Institute for Radioprotection and Nuclear Safety (IRSN), the Belgium’s nuclear safety authorities (Bel V) and European project H2020-ICT-2014-1 #644425 Scissor.


  1. 1.
    Boullé, M.: MODL: a Bayes optimal discretization method for continuous attributes. Mach. Learn. 65(1), 131–165 (2006)CrossRefGoogle Scholar
  2. 2.
    Boullé, M.: Khiops: a statistical discretization method of continuous attributes. Mach. Learn. 55(1), 53–69 (2004)zbMATHCrossRefGoogle Scholar
  3. 3.
    Dempster, A.P., Laird, N., Rubin, D.: Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Stat. Soc. 39(1), 1–38 (1977)zbMATHMathSciNetGoogle Scholar
  4. 4.
    Dougherty, J., Kohavi, R., Sahami, M.: Supervised and unsupervised discretization of continuous features. In: proceeding of ICML 1995, pp. 194–202 (1995)Google Scholar
  5. 5.
    Fayyad, U., Irani, K.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceeding of IJCAI 1993, pp. 1022–1029 (1993)Google Scholar
  6. 6.
    Friedman, N., Goldszmidt, M.: Discretizing continuous attributes while learning Bayesian networks. In: proceeding of ICML 1996, pp. 157–165 (1996)Google Scholar
  7. 7.
    Ide, J.S., Cozman, F.G., Ramos, F.T.: Generating random Bayesian networks with constraints on induced width. In: Proceeding of ECAI 2004, pp. 323–327 (2004)Google Scholar
  8. 8.
    Jiang, S., Li, X., Zheng, Q., Wang, L.: Approximate equal frequency discretization method. In: proceeding of GCIS 2009, pp. 514–518 (2009)Google Scholar
  9. 9.
    Kerber, R.: ChiMerge: Discretization of numeric attributes. In: Proceeding of AAAI 1992, pp. 123–128 (1992)Google Scholar
  10. 10.
    Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press (2009)Google Scholar
  11. 11.
    Kwedlo, W., Krȩtowski, M.: An evolutionary algorithm using multivariate discretization for decision rule induction. In: Żytkow, J.M., Rauch, J. (eds.) PKDD 1999. LNCS (LNAI), vol. 1704, pp. 392–397. Springer, Heidelberg (1999) CrossRefGoogle Scholar
  12. 12.
    Lauritzen, S., Wermuth, N.: Graphical models for associations between variables, some of which are qualitative and some quantitative. Ann. Stat. 17(1), 31–57 (1989)zbMATHMathSciNetCrossRefGoogle Scholar
  13. 13.
    Monti, S., Cooper, G.: A multivariate discretization method for learning Bayesian networks from mixed data. In: Proceeding of UAI 1998, pp. 404–413 (1998)Google Scholar
  14. 14.
    Monti, S., Cooper, G.: A latent variable model for multivariate discretization. In: proceeding of AIS 1999, pp. 249–254 (1999)Google Scholar
  15. 15.
    Moral, S., Rumí, R., Salmerón, A.: Mixtures of truncated exponentials in hybrid Bayesian networks. In: Benferhat, S., Besnard, P. (eds.) ECSQARU 2001. LNCS (LNAI), vol. 2143, pp. 156–167. Springer, Heidelberg (2001) CrossRefGoogle Scholar
  16. 16.
    Ratanamahatana, C.: CloNI: Clustering of sqrt(n)-interval discretization. In: proc. of Int. Conf. on Data Mining & Comm Tech. (2003)Google Scholar
  17. 17.
    Ruichu, C., Zhifeng, H., Wen, W., Lijuan, W.: Regularized Gaussian mixture model based discretization for gene expression data association mining. Appl. Intell. 39(3), 607–613 (2013)CrossRefGoogle Scholar
  18. 18.
    Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)zbMATHCrossRefGoogle Scholar
  19. 19.
    Shachter, R.: Evaluating influence diagrams. Oper. Res. 34(6), 871–882 (1986)MathSciNetCrossRefGoogle Scholar
  20. 20.
    Shenoy, P., West, J.: Inference in hybrid Bayesian networks using mixtures of polynomials. Int. J. Approximate Reasoning 52(5), 641–657 (2011)zbMATHMathSciNetCrossRefGoogle Scholar
  21. 21.
    Song, D., Ek, C., Huebner, K., Kragic, D.: Multivariate discretization for Bayesian network structure learning in robot grasping. In: proceeding of ICRA 2011, pp. 1944–1950 (2011)Google Scholar
  22. 22.
    Zighed, D., Rabaséda, S., Rakotomalala, R.: FUSINTER: a method for discretization of continuous attributes. Int. J. Uncertainty Fuzziness Knowl. Based Syst. 6(03), 307–326 (1998)zbMATHCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Ahmed Mabrouk
    • 1
  • Christophe Gonzales
    • 2
    Email author
  • Karine Jabet-Chevalier
    • 1
  • Eric Chojnaki
    • 1
  1. 1.Institut de Radioprotection et de Sûreté NucléaireCadaracheFrance
  2. 2.Sorbonne Universités, UPMC Univ Paris 06, CNRS, UMR 7606, LIP6ParisFrance

Personalised recommendations