Parallel cycle-based branch-and-bound method for Bayesian network learning

  • Youcef BenmounaEmail author
  • Mohand Said Mezmaz
  • Said Mahmoudi
  • Med Amine Chikh
Short paper


Bayesian networks (BNs) are one of the most commonly used models for representing uncertainty in medical diagnosis. Learning the exact structure of a BN is a challenging problem. This paper proposes a multi-threaded branch-and-bound (B&B) method, called parallel cycle-based branch-and-bound (parallel CB-B&B). On the one hand, CB-B&B improves the standard B&B method by leveraging two heuristics, namely the branching strategy and the bounding operators; on the other hand, the learning procedure is alleviated by executing CB-B&B over a set of parallel processors. In comparison with conventional exact structure learning approaches for BN, the obtained results demonstrate that the proposed CB-B&B is efficient. On average, it produces the exact structure for BN three times faster than the standard B&B version. We also present simulations on parallel CB-B&B which show a significant gain in terms of execution time.


Medical diagnosis Bayesian network Structure learning Branch-and-bound Optimization Cycle-based Parallel computing 



  1. 1.
    Li J, Fu AWC, He H, Chen J, Jin H, McAullay D, Kelman C (2005) Mining risk patterns in medical data. In: Proceedings of the eleventh ACM SIGKDD international conference on knowledge discovery in data mining. ACM, pp. 770–775Google Scholar
  2. 2.
    Lavrac̃ N, (1999) Selected techniques for data mining in medicine. Artif Intell Med 16(1):3–23MathSciNetCrossRefGoogle Scholar
  3. 3.
    Pearl J (2014) Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, San FranciscozbMATHGoogle Scholar
  4. 4.
    Hurn MA, Mardia KV, Hainsworth TJ, Kirkbride J, Berry E (1996) Bayesian fused classification of medical images. IEEE Trans Med imaging 15(6):850–858CrossRefGoogle Scholar
  5. 5.
    Arias J, Martínez-Gómez J, Gámez JA, de Herrera AGS, Müller H (2016) Medical image modality classification using discrete Bayesian networks. Comput Vis Image Underst 151:61–71CrossRefGoogle Scholar
  6. 6.
    Al-Aidaroos KM, Bakar AA, Othman Z (2012) Medical data classification with Naive Bayes approach. Inf Technol J 11(9):1166CrossRefGoogle Scholar
  7. 7.
    Wolfson J, Bandyopadhyay S, Elidrisi M, Vazquez-Benitez G, Vock DM, Musgrove D, O’Connor PJ (2015) A Naive Bayes machine learning approach to risk prediction using censored, time-to-event data. Stat Med 34(21):2941–2957MathSciNetCrossRefGoogle Scholar
  8. 8.
    Wang KJ, Makond B, Wang KM (2014) Modeling and predicting the occurrence of brain metastasis from lung cancer by Bayesian network: a case study of Taiwan. Comput Biol Med 47:147–160CrossRefGoogle Scholar
  9. 9.
    Klement W, Wilk S, Michalowski W, Farion KJ, Osmond MH, Verter V (2012) Predicting the need for CT imaging in children with minor head injury using an ensemble of Naive Bayes classifiers. Artif Intell Med 54(3):163–170CrossRefGoogle Scholar
  10. 10.
    Chickering DM, Heckerman D, Meek C (2004) Large-sample learning of Bayesian networks is NP-hard. J Mach Learn Res 5:1287–1330MathSciNetzbMATHGoogle Scholar
  11. 11.
    Silander T, Myllymaki P (2012) A simple approach for finding the globally optimal Bayesian network structure. arXiv preprint arXiv:1206.6875
  12. 12.
    Singh AP, Moore AW (2005) Finding optimal Bayesian networks by dynamic programmingGoogle Scholar
  13. 13.
    Cussens J (2012) Bayesian network learning with cutting planes. arXiv preprint arXiv:1202.3713
  14. 14.
    Cussens J (2014) Integer programming for Bayesian network structure learning. Qual Technol Quant Manag 11(1):99–110CrossRefGoogle Scholar
  15. 15.
    Yuan C, Malone B, Wu X (2011) Learning optimal Bayesian networks using A* search. In: IJCAI proceedings-international joint conference on artificial intelligence, vol 22, no. 3, p 2186Google Scholar
  16. 16.
    Yuan C, Malone B (2012) An improved admissible heuristic for finding optimal Bayesian networks. In: Proceedings of the 28th conference on uncertainty in artificial intelligenceGoogle Scholar
  17. 17.
    Yuan C, Malone B (2013) Learning optimal Bayesian networks: a shortest path perspective. J Artif Intell Res (JAIR) 48:23–65MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    De Campos CP, Ji Q (2011) Efficient structure learning of Bayesian networks using constraints. J Mach Learn Res 12:663–689MathSciNetzbMATHGoogle Scholar
  19. 19.
    Chow CK, Liu CN (1968) Approximating discrete probability distributionswith dependence trees. Inf Theory 14:462–467CrossRefzbMATHGoogle Scholar
  20. 20.
    Meila M, Jaakkola T (2000) Tractable Bayesian learning of tree belief networks. In: Proceedings of the sixteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann Publishers Inc., pp 380–388Google Scholar
  21. 21.
    Spirtes P, Glymour C (1991) An algorithm for fast recovery of sparse causal graphs. Soc Sci Comput Rev 9(1):62–72CrossRefGoogle Scholar
  22. 22.
    Kalisch M, Buhlmann P (2007) Estimating high-dimensional directed acyclic graphs with the PC-algorithm. J Mach Learn Res 8:613–636zbMATHGoogle Scholar
  23. 23.
    Cooper GF, Herskovits E (1992) A Bayesian method for the induction of probabilistic networks from data. Mach Learn 9(4):309–347zbMATHGoogle Scholar
  24. 24.
    Liu F, Zhu Q (2007) The max-relevance and min-redundancy greedy Bayesian network learning algorithm. In: International Work Conference on the Interplay Between Natural and Artificial Computation. Springer, Berlin, Heidelberg, pp 346–356Google Scholar
  25. 25.
    Daly R, Shen Q (2009) Learning Bayesian network equivalence classes with ant colony optimization. J Artif Intell Res 35(1):391MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Zhang Q, Li Z, Zhou CJ, Wei XP (2013) Bayesian network structure learning based on the chaotic particle swarm optimization algorithm. Genet Mol Res 12(4):4468–4479CrossRefGoogle Scholar
  27. 27.
    Dos Santos EB, Hruschka Jr ER, Hruschka ER, Ebecken NF (2010) A distance-based mutation operator for learning Bayesian network structures using evolutionary algorithms. In: 2010 IEEE congress on evolutionary computation (CEC). IEEE, pp 1–8Google Scholar
  28. 28.
    Schwarz G (1978) Estimating the dimension of a model. Ann Stat 6(2):461–464MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Akaike H (1974) A new look at the statistical model identification. IEEE Trans Autom Control 19(6):716–723MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Teyssier M, Koller D (2005) Ordering-based search: a simple and effective algorithm for learning Bayesian networks. In: Proceedings of the 21st conference on uncertainty in artificial intelligence, UAÍ05, pp 584–590Google Scholar
  31. 31.
    Melab N (2005) Contributions à la résolution de problèmes d’optimisation combinatoire sur grilles de calcul. LIFL, USTL, Novembre. Thèse HDRGoogle Scholar
  32. 32.
    Cung VD et al (1994) Parallel and distributed branch-and-bound/A* algorithms. Laboratoire PRISM, Universite de Versailles, Techncial report, p 94Google Scholar
  33. 33.
    Gendron B, Crainic TG (1994) Parallel branch-and-branch algorithms: survey and synthesis. Oper Res 42(6):1042–1066MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Miller Donald L, Pekny Joseph F (1993) Commentary—the role of performance metrics for parallel mathematical programming algorithms. ORSA J Comput 5(1):26–28CrossRefGoogle Scholar
  35. 35.
    Janakiram Virendra K (1988) A randomized parallel branch-and-bound algorithm. Int J Parallel Program 17(3):277–301CrossRefzbMATHGoogle Scholar
  36. 36.
    Krivelevich M et al (2007) Approximation algorithms and hardness results for cycle packing problems. ACM Trans Algorithms (TALG) 3(4):48MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Cooper GF (1984) NESTOR: a computer-based medical diagnostic aid that integrates causal and probabilistic knowledge. No. STAN-CS-84-1031. STANFORD UNIV CA DEPT OF COMPUTER SCIENCE, 1984Google Scholar

Copyright information

© Springer-Verlag London Ltd., part of Springer Nature 2019

Authors and Affiliations

  • Youcef Benmouna
    • 1
    Email author
  • Mohand Said Mezmaz
    • 2
  • Said Mahmoudi
    • 2
  • Med Amine Chikh
    • 1
  1. 1.University of TlemcenTlemcenAlgeria
  2. 2.University of MonsMonsBelgium

Personalised recommendations