Joint optimization of software time-to-market and testing duration using multi-attribute utility theory

  • P. K. KapurEmail author
  • Saurabh Panwar
  • Ompal Singh
  • Vivek Kumar
S.I.: Statistical Reliability Modeling and Optimization


An optimal software release strategy is a well-investigated issue in software reliability literature. Comprehensive testing is expected before releasing the software into the market to enhance the reliability and security of the software device. In recent years, few analysts have recommended the scheme for software projects that support releasing the software early in the market and continue the testing process for an added period in the field environment even after the software is distributed. These studies are based on one common assumption that the efficiency of the software engineers in detecting the faults occurs at a consistent rate throughout the testing phase. However, bug-identification rate may experience discontinuity at the software release time. In software engineering, the time-point at which fault detection rate changes is termed as change-point. Consequently, an alternative software release policy is proposed in the present paper, which offers a generalized framework for fault detection phenomenon using the unified approach. An extensive analysis of software time-to-market and testing duration based on cost-efficiency and reliability measures is discussed by considering the change in tester’s fault detection rate. A multi-criteria decision making technique known as multi-attribute utility theory is applied to optimize the software release policy under field-testing (FT) and no field-testing (NFT) frameworks. The relevance of the optimization problem is illustrated using a numerical example, comprising both the exponential and S-shaped bug-detection process.


Software reliability Field-testing Software reliability growth models (SRGMs) Change-point Multi-attribute utility theory (MAUT) Testing termination point Software distribution time Bug-identification rate 



  1. Arora, A., Caulkins, J. P., & Telang, R. (2006). Research note—Sell first, fix later: Impact of patching on software quality. Management Science,52(3), 465–471.CrossRefGoogle Scholar
  2. Chiu, K. C., Ho, J. W., & Huang, Y. S. (2009). Bayesian updating of optimal release time for software systems. Software Quality Journal,17(1), 99.CrossRefGoogle Scholar
  3. Dalal, S. R., & Mallows, C. L. (1988). When should one stop testing software? Journal of the American Statistical Association,83(403), 872–879.CrossRefGoogle Scholar
  4. Garmabaki, A. H., Aggarwal, A. G., Kapur, P. K., & Yadavali, V. S. S. (2012). Modeling two-dimensional software multi-upgradation and related release problem (a multi-attribute utility approach). International Journal of Reliability, Quality and Safety Engineering,19(03), 1250012.CrossRefGoogle Scholar
  5. Goel, A. L., & Okumoto, K. (1979). Time-dependent error-detection rate model for software reliability and other performance measures. IEEE Transactions on Reliability,28(3), 206–211.CrossRefGoogle Scholar
  6. Huang, C. Y. (2005a). Cost-reliability-optimal release policy for software reliability models incorporating improvements in testing efficiency. Journal of Systems and Software,77(2), 139–155.CrossRefGoogle Scholar
  7. Huang, C. Y. (2005b). Performance analysis of software reliability growth models with testing-effort and change-point. Journal of Systems and Software,76(2), 181–194.CrossRefGoogle Scholar
  8. Huang, C. Y., & Lyu, M. R. (2005). Optimal release time for software systems considering cost, testing-effort, and test efficiency. IEEE Transactions on Reliability,54(4), 583–591.CrossRefGoogle Scholar
  9. Huang, C. Y., & Lyu, M. R. (2011). Estimation and analysis of some generalized multiple change-point software reliability models. IEEE Transactions on Reliability,60(2), 498–514.CrossRefGoogle Scholar
  10. Huang, C. Y., Lyu, M. R., & Kuo, S. Y. (2003). A unified scheme of some non-homogenous poisson process models for software reliability estimation. IEEE Transactions on Software Engineering,29(3), 261–269.CrossRefGoogle Scholar
  11. Inoue, S., Ikeda, J., & Yamada, S. (2016). Bivariate change-point modeling for software reliability assessment with uncertainty of testing-environment factor. Annals of Operations Research,244(1), 209–220.CrossRefGoogle Scholar
  12. Inoue, S., Taniguchi, S., & Yamada, S. (2015). An all-stage truncated multiple change point model for software reliability assessment. International Journal of Reliability, Quality and Safety Engineering,22(04), 1550017.CrossRefGoogle Scholar
  13. Inoue, S., & Yamada, S. (2008, December). Optimal software release policy with change-point. In IEEE international conference on Industrial engineering and engineering management, 2008. IEEM 2008 (pp. 531–535). IEEE.Google Scholar
  14. Jiang, Z., & Sarkar, S. (2003, December). Optimal software release time with patching considered. In Workshop on Information Technologies and Systems, Seattle, WA, USA.Google Scholar
  15. Jiang, Z., Sarkar, S., & Jacob, V. S. (2012). Postrelease testing and software release policy for enterprise-level systems. Information Systems Research,23(31), 635–657.CrossRefGoogle Scholar
  16. Kapur, P. K., & Garg, R. B. (1991). Optimal release policies for software systems with testing effort. International Journal of Systems Science,22(9), 1563–1571.CrossRefGoogle Scholar
  17. Kapur, P. K., & Garg, R. B. (1992). A software reliability growth model for an error-removal phenomenon. Software Engineering Journal,7(4), 291–294.CrossRefGoogle Scholar
  18. Kapur, P. K., Goswami, D. N., Bardhan, A., & Singh, O. (2008a). Flexible software reliability growth model with testing effort dependent learning process. Applied Mathematical Modelling,32(7), 1298–1307.CrossRefGoogle Scholar
  19. Kapur, P. K., Khatri, S. K., Tickoo, A., & Shatnawi, O. (2014). Release time determination depending on number of test runs using multi attribute utility theory. International Journal of System Assurance Engineering and Management,5(2), 186–194.CrossRefGoogle Scholar
  20. Kapur, P. K., Kumar, S., & Garg, R. B. (1999). Contributions to hardware and software reliability (Vol. 3). Singapore: World Scientific.CrossRefGoogle Scholar
  21. Kapur, P. K., Pham, H., Aggarwal, A. G., & Kaur, G. (2012). Two dimensional multi-release software reliability modeling and optimal release planning. IEEE Transactions on Reliability,61(3), 758–768.CrossRefGoogle Scholar
  22. Kapur, P. K., Pham, H., Anand, S., & Yadav, K. (2011a). A unified approach for developing software reliability growth models in the presence of imperfect debugging and error generation. IEEE Transactions on Reliability,60(1), 331–340.CrossRefGoogle Scholar
  23. Kapur, P. K., Pham, H., Gupta, A., & Jha, P. C. (2011b). Software reliability assessment with OR applications. London: Springer.CrossRefGoogle Scholar
  24. Kapur, P. K., Shrivastava, A. K., & Singh, O. (2017). When to release and stop testing of a software. Journal of the Indian Society for Probability and Statistics,18(1), 19–37.CrossRefGoogle Scholar
  25. Kapur, P. K., Singh, V. B., Anand, S., & Yadavalli, V. S. S. (2008b). Software reliability growth model with change-point and effort control using a power function of the testing time. International Journal of Production Research,46(3), 771–787.CrossRefGoogle Scholar
  26. Kapur, P. K., Singh, V. B., Singh, O., & Singh, J. N. (2013). Software release time based on different multi-attribute utility functions. International Journal of Reliability, Quality and Safety Engineering,20(04), 1350012.CrossRefGoogle Scholar
  27. Keeney, R. L. (1971). Utility independence and preferences for multi attributed consequences. Operations Research,19(4), 875–893.CrossRefGoogle Scholar
  28. Li, X., Xie, M., & Ng, S. H. (2012). Multi-objective optimization approaches to software release time determination. Asia-Pacific Journal of Operational Research,29(03), 1240019.CrossRefGoogle Scholar
  29. Majumdar, R., Shrivastava, A. K., Kapur, P. K., & Khatri, S. K. (2017). Release and testing stop time of a software using multi-attribute utility theory. Life Cycle Reliability and Safety Engineering,6(1), 47–55.CrossRefGoogle Scholar
  30. Marquardt, D. W. (1963). An algorithm for least-squares estimation of nonlinear parameters. Journal of the Society for Industrial and Applied Mathematics,11(2), 431–441.CrossRefGoogle Scholar
  31. Minamino, Y., Inoue, S., & Yamada, S. (2015). Multi-attribute utility theory for estimation of optimal release time and change-point. International Journal of Reliability, Quality and Safety Engineering,22(04), 1550019.CrossRefGoogle Scholar
  32. Minamino, Y., Inoue, S., & Yamada, S. (2016). NHPP-based change-point modeling for software reliability assessment and its application to software development management. Annals of Operations Research,244(1), 85–101.CrossRefGoogle Scholar
  33. Musa, J. D., & Okumoto, K. (1983). Software reliability models: concepts, classification, comparisons, and practice. In J. K. Skwirzynski (Ed.), Electronic systems effectiveness and life cycle costing (pp. 395–423). Berlin: Springer.CrossRefGoogle Scholar
  34. Myers, G. J. (1976). Softwear reliability: Principles and practices. New York: Wiley.Google Scholar
  35. Ohba, M., & Yamada, S. (1984). S-shaped software reliability growth models. In International colloquium on reliability and maintainability, 4th, Tregastel, France (pp. 430–436).Google Scholar
  36. Okumoto, K., & Goel, A. L. (1980). Optimum release time for software systems based on reliability and cost criteria. Journal of Systems and Software,1(4), 315–318.Google Scholar
  37. Pham, H., Nordmann, L., & Zhang, Z. (1999). A general imperfect-software-debugging model with S-shaped fault-detection rate. IEEE Transactions on Reliability,48(2), 169–175.CrossRefGoogle Scholar
  38. Pham, H., & Zhang, X. (1999). Software release policies with gain in reliability justifying the costs. Annals of Software Engineering,8(1–4), 147–166.CrossRefGoogle Scholar
  39. SAS, S. (2004). STAT User guide, Version 9.1.2.. Cary, NC: SAS Institute Inc.Google Scholar
  40. Singh, O., Kapur, P. K., Shrivastava, A. K., & Kumar, V. (2015). Release time problem with multiple constraints. International Journal of System Assurance Engineering and Management,6(1), 83–91.CrossRefGoogle Scholar
  41. Subburaj, R., & Kapur, P. K. (2014). Two practical software reliability growth models for software project management. In P. K. Kapur, et al. (Eds.), Quality, reliability and Infocomm technology and industrial technological management. I.K: International Publishing House Pvt. Ltd.Google Scholar
  42. Tickoo, A., Verma, A. K., Khatri, S. K., & Kapur, P. K. (2016). Modeling Two-Dimensional Framework for Multi-Upgradations of a Software with Change Point. International Journal of Reliability, Quality and Safety Engineering,23(06), 1640008.CrossRefGoogle Scholar
  43. Wood, A. (1996). Predicting software reliability. Computer,29(11), 69–77.CrossRefGoogle Scholar
  44. Yamada, S. (2014). Software reliability modeling: Fundamentals and applications (Vol. 5). Tokyo: Springer.CrossRefGoogle Scholar
  45. Yamada, S., Ohba, M., & Osaki, S. (1983). S-shaped reliability growth modeling for software error detection. IEEE Transactions on Reliability,32(5), 475–484.CrossRefGoogle Scholar
  46. Yamada, S., & Osaki, S. (1985). Software reliability growth modeling: Models and applications. IEEE Transactions on Software Engineering,12, 1431–1437.CrossRefGoogle Scholar
  47. Yamada, S., & Osaki, S. (1987). Optimal software release policies with simultaneous cost and reliability requirements. European Journal of Operational Research,31(1), 46–51.CrossRefGoogle Scholar
  48. Yamada, S., & Yamaguchi, M. (2016). A method of statistical process control for successful open source software projects and its application to determining the development period. International Journal of Reliability, Quality and Safety Engineering,23(05), 1650018.CrossRefGoogle Scholar
  49. Zhang, X., & Pham, H. (2002). Predicting operational software availability and its applications to telecommunication systems. International Journal of Systems Science,33(11), 923–930.CrossRefGoogle Scholar
  50. Zhao, M. (2003). Statistical reliability change-point estimation models. In H. Pham (Ed.), Handbook of Reliability Engineering (pp. 157–163). London: Springer.CrossRefGoogle Scholar
  51. Zhu, M., & Pham, H. (2018). A multi-release software reliability modeling for open source software incorporating dependent fault detection process. Annals of Operations Research,269, 773–790.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2019

Authors and Affiliations

  • P. K. Kapur
    • 1
    Email author
  • Saurabh Panwar
    • 2
  • Ompal Singh
    • 2
  • Vivek Kumar
    • 2
  1. 1.Amity Center for Interdisciplinary ResearchAmity UniversityNoidaIndia
  2. 2.Department of Operational ResearchUniversity of DelhiDelhiIndia

Personalised recommendations