Determination of Variables for a Bayesian Network and the Most Precious One

  • Esma Nur CiniciogluEmail author
  • Taylan Yenilmez
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 610)


To ensure the quality of a learned Bayesian network out of limited data sets, evaluation and selection process of variables becomes necessary. With this purpose, two new variable selection criteria N 2 S j and N 3 S j are proposed in this research which show superior performance on limited data sets. These newly developed variable selection criteria with the existing ones from prior research are employed to create Bayesian networks from three different limited data sets. On each step of variable elimination, the performance of the resulting BNs are evaluated in terms of different network performance metrics. Furthermore, a new variable evaluation criteria, IH j , is proposed which measures the impact of a variable to all the other variables in the network. IH j serves as an indicator of the most important variables in the network which has a special importance for the use of BNs in social science research, where it is crucial to identify the most important factors in a setting.


Bayesian networks Variable selection in Bayesian networks Importance hierarchy of variables in network Variable evaluation scores Limited data sets 


  1. 1.
    Koller, D., Sahami, M.: Toward optimal feature selection. In: 13th International Conference on Machine Learning, pp. 284–292 (1996)Google Scholar
  2. 2.
    Fernández, A., Gómez, Á., Lecumberry, F., Pardo, Á., Ramírez, I.: Pattern recognition in latin America in the “big data” era. Pattern Recogn. 48(4), 1185–1196 (2015)CrossRefGoogle Scholar
  3. 3.
    Reunanen, J.: Overfitting in making comparisons between variable selection methods. J. Mach. Learn. Res. 3, 1371–1382 (2003)zbMATHGoogle Scholar
  4. 4.
    Sawalha, Z., Sayed, T.: Traffic accident modeling: some statistical issues. Can. J. Civ. Eng. 33(9), 1115–1124 (2006)CrossRefGoogle Scholar
  5. 5.
    Drugan, M.M., Wiering, M.A.: Feature selection for Bayesian network classifiers using the MDL-FS score. Int. J. Approximate Reasoning 51(6), 695–717 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Cooper, G.F., Aliferis, C.F., Ambrosino, R., Aronis, J., Buchanan, B.G., Caruana, R., Janosky, J.E.: An evaluation of machine-learning methods for predicting pneumonia mortality. Artif. Intell. Med. 9(2), 107–138 (1997)CrossRefGoogle Scholar
  7. 7.
    Cheng, J., Greiner, R.: Comparing Bayesian network classifiers. In: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 101–108. Morgan Kaufmann Publishers Inc. (1999)Google Scholar
  8. 8.
    Hruschka Jr., E.R., Hruschka, E.R., Ebecken, N.F.: Feature selection by Bayesian networks. In: Tawfik, A.Y., Goodwin, S.D. (eds.) Canadian AI 2004. LNCS (LNAI), vol. 3060, pp. 370–379. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  9. 9.
    Castro, P.A., Von Zuben, F.J.: Learning Bayesian networks to perform feature selection. In: International Joint Conference on Neural Networks, IJCNN 2009, pp. 467–473. IEEE Press (2009)Google Scholar
  10. 10.
    Cinicioglu, E.N., Büyükuğur, G.: How to create better performing Bayesian networks: a heuristic approach for variable selection. In: Laurent, A., Strauss, O., Bouchon-Meunier, B., Yager, R.R. (eds.) IPMU 2014, Part I. CCIS, vol. 442, pp. 527–535. Springer, Heidelberg (2014)Google Scholar
  11. 11.
    Tonda, A.P., Lutton, E., Reuillon, R., Squillero, G., Wuillemin, P.-H.: Bayesian network structure learning from limited datasets through graph evolution. In: Moraglio, A., Silva, S., Krawiec, K., Machado, P., Cotta, C. (eds.) EuroGP 2012. LNCS, vol. 7244, pp. 254–265. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  12. 12.
    Cinicioglu, E.N., Shenoy, P.P.: A new heuristic for learning Bayesian networks from limited datasets: a real-time recommendation system application with RFID systems in grocery stores. Ann. Oper. Res. 1–21 (2012). doi: 10.1007/s10479-012-1171-9
  13. 13.
    Marcot, B.G., Steventon, J.D., Sutherland, G.D., McCann, R.K.: Guidelines for developing and updating Bayesian belief networks applied to ecological modeling and conservation. Can. J. For. Res. 36(12), 3063–3074 (2006)CrossRefGoogle Scholar
  14. 14.
    Cinicioglu, E.N., Önsel Ekici, Ş., Ülengin, F.: Bayes Ağ Yapısının Oluşturulmasında Farklı Yaklaşımlar: Nedensel Bayes Ağları ve Veriden Ağ Öğrenme. In: Halil Sarıaslan Armağan Kitap, pp. 267–286. Siyasal Kitabevi, Ankara (2015)Google Scholar
  15. 15.
    Akaike, H.: Information theory and an extension of the maximum likelihood principle. In: Petrov, B.N, Csaki, F. (eds.) Second International Symposium on Information Theory, pp. 267–281. Akademiai Kiado, Budapest (1973)Google Scholar
  16. 16.
    Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, Burlington (2005)zbMATHGoogle Scholar
  18. 18.
    Cooper, G.F., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)zbMATHGoogle Scholar
  19. 19.
    Su, C., Andrew, A., Karagas, M.R., Borsuk, M.E.: Using Bayesian networks to discover relations between genes, environment, and disease. BioData Min. 6(1), 1 (2013)CrossRefGoogle Scholar
  20. 20.
    Druzdzel, M.J.: SMILE: structural modeling, inference, and learning engine and GeNIe: a development environment for graphical decision-theoretic models. In: AAAI/IAAI, pp. 902–903 (1999)Google Scholar
  21. 21.
  22. 22.
    Scutari, M.: Learning Bayesian networks with the bnlearn R package. J Stat. Soft 35, 1–22 (2010)CrossRefGoogle Scholar
  23. 23.
    Schwab, K.: The global competitiveness report 2009–2010. World Economic Forum, 2009, 2010, 2011, 2012. World Economic Forum, GenevaGoogle Scholar
  24. 24.
    Cinicioglu, E.N., Ulusoy, G., Önsel, S., Ülengin, F., Ülengin, B.: The basic competitiveness factors shaping the innovation performance of countries. In: Proceedings of the International Conference of Institute of Industrial Engineers, Istanbul, pp. 1–5 (2013)Google Scholar
  25. 25.
    Gaulier, G., Zignago, S.: BACI: international trade database at the product-level. The 1994–2007 Version. Working Papers 2010–23, CEPII Research Center (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  1. 1.Quantitative Methods Division, School of BusinessIstanbul UniversityIstanbulTurkey

Personalised recommendations