A systematic literature review on the applications of Bayesian networks to predict software quality

Abstract

Bayesian networks (BN) have been used for decision making in software engineering for many years. In other fields such as bioinformatics, BNs are rigorously evaluated in terms of the techniques that are used to build the network structure and to learn the parameters. We extend our prior mapping study to investigate the extent to which contextual and methodological details regarding BN construction are reported in the studies. We conduct a systematic literature review on the applications of BNs to predict software quality. We focus on more detailed questions regarding (1) dataset characteristics, (2) techniques used for parameter learning, (3) techniques used for structure learning, (4) use of tools, and (5) model validation techniques. Results on ten primary studies show that BNs are mostly built based on expert knowledge, i.e. structure and prior distributions are defined by experts, whereas authors benefit from BN tools and quantitative data to validate their models. In most of the papers, authors do not clearly explain their justification for choosing a specific technique, and they do not compare their proposed BNs with other machine learning approaches. There is also a lack of consensus on the performance measures to validate the proposed BNs. Compared to other domains, the use of BNs is still very limited and current publications do not report enough details to replicate the studies. We propose a framework that provides a set of guidelines for reporting the essential contextual and methodological details of BNs. We believe such a framework would be useful to replicate and extend the work on BNs.

This is a preview of subscription content, access via your institution.

Fig. 1
Fig. 2

Notes

  1. 1.

    One of the papers is accepted for publication in 2014 (so online access is possible), and the paper will officially be published in 2015.

References

  1. Abran, A., Bourque, P., Dupuis, R., & Moore, J. W. (2001). Guide to the software engineering body of knowledge-SWEBOK. Piscataway: IEEE Press.

    Google Scholar 

  2. Bev Littlewood, C., & Strigini, L. (1993). Validation of ultra-high dependability for software-based systems. Communications of the ACM (CACM), 36(11), 69–80.

    Article  Google Scholar 

  3. Clifton, D. A., Gibbons, J., Davies, J., & Tarassenko, L. Machine learning and software engineering in health informatics. In Realizing Artificial Intelligence Synergies in Software Engineering (RAISE), 2012 first international workshop on, 2012 (pp. 37–41), IEEE.

  4. Daly, R., Shen, Q., & Aitken, S. (2011). Learning Bayesian networks: Approaches and issues. The Knowledge Engineering Review, 26(02), 99–157.

    Article  Google Scholar 

  5. Darwiche, A. (2010). Bayesian networks. Communications of the ACM, 53(12), 80–90.

    Article  MATH  Google Scholar 

  6. Doguc, O., & Ramirez-Marquez, J. E. (2009). A generic method for estimating system reliability using Bayesian networks. Reliability Engineering & System Safety, 94(2), 542–550.

    Article  Google Scholar 

  7. Febrero, F., Calero, C., & Moraga, M. Á. (2014). A systematic mapping study of software reliability modeling. Information and Software Technology, 56(8), 839–849.

    Article  Google Scholar 

  8. Fenton, N., Marsh, W., Neil, M., Cates, P., Forey, S., & Tailor, M. (2004). Making resource decisions for software projects. In Proceedings of the 26th international conference onsoftware engineering, 2004. ICSE 2004 (pp. 397–406), IEEE.

  9. Fenton, N., & Neil, M. (1999). A critique of software defect prediction models. Software Engineering, IEEE Transactions on, 25(5), 675–689.

    Article  Google Scholar 

  10. Fenton, N., & Neil, M. (2012). Risk assessment and decision analysis with Bayesian networks. Boca Raton: CRC Press.

    MATH  Google Scholar 

  11. Fenton, N., Neil, M., & Marquez, D. (2008). Using Bayesian networks to predict software defects and reliability. Proceedings of the Institution of Mechanical Engineers, Part O: Journal of Risk and Reliability, 222(4), 701–712.

    Google Scholar 

  12. Fineman, M., Fenton, N., & Radlinski, L. (2009).Modelling project trade-off using Bayesian networks. In International conference on computational intelligence and software engineering, 2009. CiSE 2009 (pp. 1–4), IEEE.

  13. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76(5), 378.

    Article  Google Scholar 

  14. Hall, T., Beecham, S., Bowes, D., & Gray, D. (2011). A systematic literature review on fault prediction performance in software engineering. Software Engineering, IEEE Transactions, 38(6), 1276–1304.

    Article  Google Scholar 

  15. Harman, M. (2012). The role of artificial intelligence in software engineering. In 2012 First international workshop on realizing artificial intelligence synergies in software engineering (RAISE) (pp. 1–6), IEEE.

  16. Heckerman, D. (1998). A tutorial on learning with Bayesian networks. New York: Springer.

    Book  MATH  Google Scholar 

  17. Kitchenham, B. A., Budgen, D., & Pearl Brereton, O. (2011). Using mapping studies as the basis for further research–a participant-observer case study. Information and Software Technology, 53(6), 638–651.

    Article  Google Scholar 

  18. Kitchenham, B. A., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering. Technical report, EBSE Technical Report EBSE-2007-01.

  19. Kitchenham, B. A., Sjøberg, D. I., Dybå, T., Pfahl, D., Brereton, P., Budgen, D., et al. (2012). Three empirical studies on the agreement of reviewers about the quality of software engineering experiments. Information and Software Technology, 54(8), 804–819.

    Article  Google Scholar 

  20. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174.

    Article  MATH  Google Scholar 

  21. Littlewood, B., & Verrall, J. (1973). A Bayesian reliability growth model for computer software. Applied Statistics, 22(3), 332–346.

    MathSciNet  Article  Google Scholar 

  22. Lucas, P. J., van der Gaag, L. C., & Abu-Hanna, A. (2004). Bayesian networks in biomedicine and health-care. Artificial Intelligence in Medicine, 30(3), 201–214.

    Article  Google Scholar 

  23. Lunn, D. J., Thomas, A., Best, N., & Spiegelhalter, D. (2000). WinBUGS-a Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10(4), 325–337.

    Article  Google Scholar 

  24. Mendes, E. Predicting web development effort using a bayesian network. In Proceedings of EASE, 2007 (Vol. 7, pp. 83–93).

  25. Mendes, E., & Mosley, N. (2008). Bayesian network models for web effort prediction: A comparative study. Software Engineering, IEEE Transactions on, 34(6), 723–737.

    Article  Google Scholar 

  26. Misirli, A. T., & Bener, A. (2014a). Bayesian networks for evidence-based decision-making in software engineering. IEEE Transactions on Software Engineering, 40(6), 533–554.

    Article  Google Scholar 

  27. Misirli, A. T., & Bener, A. (2014b). Mapping study on Bayesian networks for software quality prediction. In RAISE 2014 Proceedings of the 3rd international workshop on realizing artificial intelligence synergies in software engineering (pp. 7–11).

  28. Musa, J. D., & Okumoto, K. (1983). Software reliability models: Concepts, classification, comparisons, and practice. In Electronic Systems Effectiveness and Life Cycle Costing (pp. 395–423), Springer.

  29. Neil, M., Tailor, M., & Marquez, D. (2007). Inference in hybrid Bayesian networks using dynamic discretization. Statistics and Computing, 17(3), 219–233.

    MathSciNet  Article  Google Scholar 

  30. Neil, M., Tailor, M., Marquez, D., Fenton, N., & Hearty, P. (2008). Modelling dependable systems using hybrid Bayesian networks. Reliability Engineering & System Safety, 93(7), 933–939.

    Article  Google Scholar 

  31. Radlinski, L. (2010). A survey of bayesian net models for software development effort prediction. International Journal of Software Engineering and Computing, 2(2), 95–109.

    Google Scholar 

  32. Schutt, R. (2012). 10 Important data science ideas. http://columbiadatascience.com/2012/10/15/10-important-data-science-ideas/.

  33. Spiegelhalter, D. J., Best, N. G., Carlin, B. P., & Van Der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 64(4), 583–639.

    MathSciNet  Article  MATH  Google Scholar 

  34. Wilkinson, D. (2007). Bayesian methods in bioinformatics and computational systems biology. Briefings in Bioinformatics, 8(2), 109–116.

    Article  Google Scholar 

  35. Winkler, R. L. (2001). Why Bayesian analysis hasn’t caught on in healthcare decision making. International Journal of Technology Assessment in Health Care, 17(01), 56–66.

    Article  Google Scholar 

Download references

Author information

Affiliations

Authors

Corresponding author

Correspondence to Ayse Tosun.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Tosun, A., Bener, A.B. & Akbarinasaji, S. A systematic literature review on the applications of Bayesian networks to predict software quality. Software Qual J 25, 273–305 (2017). https://doi.org/10.1007/s11219-015-9297-z

Download citation

Keywords

  • Software quality
  • Software reliability prediction
  • Systematic literature review
  • Bayesian network
  • Applications of AI