Abstract
Existing methods for predicting reliability of software are static and need manual maintenance to adjust to the evolving data sets in software organizations. Machine learning has a potential to address the problem of manual maintenance but can also require changes in how companies works with defect prediction. In this paper we address the problem of identifying what the benefits of machine learning are compared to existing methods and which barriers exist for adopting them in practice.
Our methods consist of literature studies and a case study at two companies – Ericsson and Volvo Car Group. By studying literature we develop a framework for adopting machine learning and using case studies we evaluate this framework through a series of four interviews with experts working with predictions at both companies - line manager, quality manager and measurement team leader.
The findings of our research show that the most important perceived benefits of adopting machine learning algorithms for defect prediction are accuracy of predictions and ability to generate new insights from data. The two most important perceived barriers in this context are inability to recognize new patterns and low generalizability of the machine learning algorithms.
We conclude that in order to support companies in making an informed decision to adopt machine learning techniques for software defect predictions we need to go beyond accuracy and also evaluate factors such as costs, generalizability and competence.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Fenton, N.E., Neil, M.: A critique of software defect prediction models. IEEE Trans. Softw. Eng. 25, 675–689 (1999)
Lyu, M.R.: Handbook of software reliability engineering. IEEE Computer Society Press, CA (1996)
Nagappan, N., Ball, T., Zeller, A.: Mining metrics to predict component failures. In: Proceedings of the 28th International Conference on Software Engineering, pp. 452–461 (2006)
Staron, M., Meding, W., Söderqvist, B.: A method for forecasting defect backlog in large streamline software development projects and its industrial evaluation. Inf. Softw. Technol. 52, 1069–1079 (2010)
Li, M., Smidts, C.S.: A ranking of software engineering measures based on expert opin-ion. IEEE Trans. Softw. Eng. 29, 811–824 (2003)
Fenton, N., Neil, M., Marsh, W., Hearty, P., Radliński, Ł., Krause, P.: On the effective-ness of early life cycle defect prediction with Bayesian Nets. Empir. Softw. Eng. 13, 499–537 (2008)
Zhang, D., Tsai, J.J.: Machine learning and software engineering. Softw. Qual. J. 11, 87–119 (2003)
Fenton, N., Neil, M., Marsh, W., Hearty, P., Marquez, D., Krause, P., Mishra, R.: Predict-ing software defects in varying development lifecycles using Bayesian nets. Inf. Softw. Technol. 49, 32–43 (2007)
Gondra, I.: Applying machine learning to software fault-proneness prediction. J. Syst. Softw. 81, 186–195 (2008)
Menzies, T., Greenwald, J., Frank, A.: Data mining static code attributes to learn defect predictors. IEEE Trans. on Softw. Eng. 33, 2–13 (2007)
Rana, R., Staron, M., Nilsson, M.: A framework for adoption of machine learning in industry for software defect prediction. Presented at the Submitted to ICSOFT-EA, 2014 , Vienna, Austria (2014)
Menzies, T., Ammar, K., Nikora, A., DiStefano, J.: How simple is software defect detec-tion. Submitt. Emprical Softw. Eng. J. (2003)
Chau, P.Y., Tam, K.Y.: Factors Affecting the Adoption of Open Systems: An Explora-tory Study. Mis Q. 21 (1997)
Wallace, L.G., Sheetz, S.D.: The adoption of software measures: A technology accep-tance model (TAM) perspective. Inf. Manage. 51, 249–259 (2014)
Ajzen, I., Fishbein, M.: Understanding attitudes and predicting social behaviour (1980)
Davis Jr., F.D.: A technology acceptance model for empirically testing new end-user information systems: Theory and results (1986), http://www.researchgate.net/publication/35465050_A_technology_acceptance_model_for_empirically_testing_new_end-user_information_systems__theory_and_results_/file/9c960519fbaddf3ba7.pdf
Hardgrave, B.C., Johnson, R.A.: Toward an information systems development acceptance model: the case of object-oriented systems development. IEEE Trans. on Eng. Manag. 50, 322–336 (2003)
Gefen, D., Straub, D.W.: Gender Differences in the Perception and Use of E-Mail: An Extension to the Technology Acceptance Model. MIS Q. 21 (1997)
Yi, M.Y., Hwang, Y.: Predicting the use of web-based information systems: self-efficacy, enjoyment, learning goal orientation, and the technology acceptance model. Int. J. Hum.-Comput. Stud. 59, 431–449 (2003)
Tornatzky, L.G., Fleischer, M., Chakrabarti, A.K.: Processes of technological innovation (1990)
Runeson, P., Höst, M.: Guidelines for conducting and reporting case study research in software engineering. Empir. Softw. Eng. 14, 131–164 (2009)
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer, New York (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Rana, R., Staron, M., Berger, C., Hansson, J., Nilsson, M., Meding, W. (2014). The Adoption of Machine Learning Techniques for Software Defect Prediction: An Initial Industrial Validation. In: Kravets, A., Shcherbakov, M., Kultsova, M., Iijima, T. (eds) Knowledge-Based Software Engineering. JCKBSE 2014. Communications in Computer and Information Science, vol 466. Springer, Cham. https://doi.org/10.1007/978-3-319-11854-3_23
Download citation
DOI: https://doi.org/10.1007/978-3-319-11854-3_23
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-11853-6
Online ISBN: 978-3-319-11854-3
eBook Packages: Computer ScienceComputer Science (R0)