Evolving Systems

, Volume 10, Issue 2, pp 179–204 | Cite as

Performance of Maintainability Index prediction models: a feature selection based study

  • B. Ramachandra ReddyEmail author
  • Aparajita Ojha
Original Paper


Numerous design metrics have been studied in the literature to assess software performance and future maintainability issues. Maintainability Index (MI) is an indicator of software maintenance efforts, but that can be computed only after the complete code is developed. MI is complex to calculate and hence has been estimated by several researchers using design metrics. Various prediction models are built using different sets of design metrics as independent variables. Further, researchers have used different prediction models for predicting software maintainability. To identify most prominent design metrics and the prediction models that show consistent performance irrespective of the choice of design metrics, the present empirical study is performed on datasets of 26 open source Java projects with more than one thousand and six hundred class hierarchies. Seven feature selection methods are applied to find out the most prominent metrics that are selected by majority of methods. Then four most frequently used prediction models namely, multi linear regression, multilayer perceptron, support vector regression and M5P regression tree are analyzed for their performance with respect to initial selection of design metrics. The study concludes that MIF, MaxDIT, LCC, TCC are the most significant predictors for Maintainability Index of class hierarchies. Among the all the prediction models, support vector regression model exhibits the best performance both with respect to the choice of metrics and irrespective of initial metric selection.


Object oriented software Design metrics Maintainability Index Feature selection Regression models 


  1. Agarwal S, Tomar D (2014) A feature selection based model for software defect prediction. Int J Adv Sci Technol 65:39–58CrossRefGoogle Scholar
  2. Aggarwal K, Singh Y, Kaur A, Malhotra R (2006) Empirical study of object-oriented metrics. J Object Technol 5:149–173CrossRefGoogle Scholar
  3. Ahmed MA, Al-Jamimi HA (2013) Machine learning approaches for predicting software maintainability: a fuzzy-based transparent model. IET Softw 7:317–326CrossRefGoogle Scholar
  4. Al Dallal J (2011) Measuring the discriminative power of object-oriented class cohesion metrics. IEEE Trans Softw Eng 37:788–804CrossRefGoogle Scholar
  5. Al Dallal J (2013) Object-oriented class maintainability prediction using internal quality attributes. Inf Softw Technol 55:2028–2048CrossRefGoogle Scholar
  6. Almugrin S, Albattah W, Melton A (2016) Using indirect coupling metrics to predict package maintainability and testability. J Syst Softw 121:298–310CrossRefGoogle Scholar
  7. Alshayeb M (2013) On the relationship of class stability and maintainability. IET Softw 7:339–347CrossRefGoogle Scholar
  8. Anda B (2007) Assessing software system maintainability using structural measures and expert assessments. In: 2007 IEEE International Conference on Software Maintenance. IEEE, pp 204–213Google Scholar
  9. Banker RD, Datar SM, Kemerer CF, Zweig D (1993) Software complexity and maintenance costs. Commun ACM 36:81–95CrossRefGoogle Scholar
  10. Basili VR, Briand LC, Melo WL (1996) A validation of object-oriented design metrics as quality indicators. IEEE Trans Softw Eng 22:751–761CrossRefGoogle Scholar
  11. Briand LC, Wüst J, Daly JW, Porter DV (2000) Exploring the relationships between design measures and software quality in object-oriented systems. J Syst Softw 51:245–273CrossRefGoogle Scholar
  12. Catal C, Diri B (2009) Investigating the effect of dataset size, metrics sets, and feature selection techniques on software fault prediction problem. Inf Sci 179:1040–1058CrossRefGoogle Scholar
  13. Chen J-C, Huang S-J (2009) An empirical analysis of the impact of software development problem factors on software maintainability. J Syst Softw 82:981–992CrossRefGoogle Scholar
  14. Chidamber SR, Kemerer CF (1994) A metrics suite for object oriented design. IEEE Trans Softw Eng 20:476–493CrossRefGoogle Scholar
  15. Coleman D, Ash D, Lowther B, Oman P (1994) Using metrics to evaluate software system maintainability. Computer 27:44–49CrossRefGoogle Scholar
  16. Daly J, Brooks A, Miller J, Roper M, Wood M (1996) Evaluating inheritance depth on the maintainability of object-oriented software. Empir Softw Eng 1:109–132CrossRefGoogle Scholar
  17. e Abreu FB, Carapuça R (1994) Candidate metrics for object-oriented software within a taxonomy framework. J Syst Softw 26:87–96CrossRefGoogle Scholar
  18. El Emam K, Melo W, Machado JC (2001) The prediction of faulty classes using object-oriented design metrics. J Syst Softw 56(1):63–75CrossRefGoogle Scholar
  19. Elish MO (2014) A comparative study of fault density prediction in aspect-oriented systems using MLP, RBF, KNN, RT, DENFIS and SVR models. Artif Intell Rev 42:695–703CrossRefGoogle Scholar
  20. Elish MO, Aljamaan H, Ahmad I (2015) Three empirical studies on predicting software maintainability using ensemble methods. Soft Comput 19:2511–2524CrossRefGoogle Scholar
  21. Fenton N, Bieman J (2014) Software metrics: a rigorous and practical approach. CRC Press, Boca RatonGoogle Scholar
  22. Freitag D (2014) Greedy attribute selection. In: Machine learning proceedings 1994: proceedings of the eighth international conference. Morgan Kaufmann, Burlington. p 28Google Scholar
  23. Glover F (1989) Tabu search—part I. ORSA J Comput 1:190–206CrossRefzbMATHGoogle Scholar
  24. Gunn SR (1998) Support vector machines for classification and regression. ISIS technical report 14Google Scholar
  25. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182zbMATHGoogle Scholar
  26. Gyimothy T, Ferenc R, Siket I (2005) Empirical validation of object-oriented metrics on open source software for fault prediction. IEEE Trans Softw Eng 31:897–910CrossRefGoogle Scholar
  27. Hanley JA, McNeil BJ (1982) The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 143:29–36CrossRefGoogle Scholar
  28. Harrison R, Counsell S, Nithi R (2000) Experimental assessment of the effect of inheritance on the maintainability of object-oriented systems. J Syst Softw 52:173–179CrossRefGoogle Scholar
  29. Heiat A (2002) Comparison of artificial neural network and regression models for estimating software development effort. Inf Softw Technol 44:911–922CrossRefGoogle Scholar
  30. Heitlager I, Kuipers T, Visser J (2007) A practical model for measuring maintainability. In: 6th international conference on the quality of information and communications technology, 2007. QUATIC 2007. IEEE, pp 30–39Google Scholar
  31. Hernández-Orallo J (2013) ROC curves for regression. Pattern Recognit 46:3395–3411CrossRefzbMATHGoogle Scholar
  32. Hoque N, Bhattacharyya D, Kalita JK (2014) MIFS-ND: a mutual information-based feature selection method. Expert Syst Appl 41:6371–6385CrossRefGoogle Scholar
  33. ISO IOFS (2011) ISO/IEC 25010:2011. 2016
  34. Jabangwe R, Börstler J, Šmite D, Wohlin C (2015) Empirical evidence on the link between object-oriented measures and external quality attributes: a systematic literature review. Empir Softw Eng 20:640–693CrossRefGoogle Scholar
  35. Jorgensen M, Shepperd M (2007) A systematic review of software development cost estimation studies. IEEE Trans Softw Eng 33:33–53CrossRefGoogle Scholar
  36. Jović A, Brkić K, Bogunović N (2015) A review of feature selection methods with applications. In: 38th international convention on information and communication technology, electronics and microelectronics (MIPRO), 2015. IEEE, pp 1200–1205Google Scholar
  37. Kiewkanya M, Jindasawat N, Muenchaisri P (2004) A methodology for constructing maintainability model of object-oriented design. In: Fourth international conference on quality software, 2004. QSIC 2004. Proceedings. IEEE, pp 206–213Google Scholar
  38. Kim Y, Street WN, Menczer F (2000) Feature selection in unsupervised learning via evolutionary search. In: Proceedings of the sixth ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 365–369Google Scholar
  39. Kiran NR, Ravi V (2008) Software reliability prediction by soft computing techniques. J Syst Softw 81(4):576–583CrossRefGoogle Scholar
  40. Kitchenham BA, Pickard LM, MacDonell SG, Shepperd MJ (2001) What accuracy statistics really measure [software estimation]. IEE Proc Softw 148:81–85CrossRefGoogle Scholar
  41. Krogh A, Vedelsby J (1995) Neural network ensembles, cross validation, and active learning. Adv Neural Inf Process Syst 7:231–238Google Scholar
  42. Kumar L, Krishna A, Rath SK (2017) The impact of feature selection on maintainability prediction of service-oriented applications. Serv Oriented Comput Appl 11:137–161CrossRefGoogle Scholar
  43. Laradji IH, Alshayeb M, Ghouti L (2015) Software defect prediction using ensemble learning on selected features. Inf Softw Technol 58:388–402CrossRefGoogle Scholar
  44. Li W, Henry S (1993) Object-oriented metrics that predict maintainability. J Syst Softw 23:111–122CrossRefGoogle Scholar
  45. Lu Y, Mao X, Li Z (2016) Assessing software maintainability based on class diagram design: a preliminary case study. Lect Notes Softw Eng 4:53CrossRefGoogle Scholar
  46. Mishra D (2012) New inheritance complexity metrics for object-oriented software systems: an evaluation with Weyuker’s properties. Comput Inf 30:267–293MathSciNetGoogle Scholar
  47. Muthanna S, Kontogiannis K, Ponnambalam K, Stacey B (2000) A maintainability model for industrial software systems using design level metrics. In: Seventh working conference on reverse engineering, 2000. Proceedings. IEEE, pp 248–256Google Scholar
  48. Naboulsi Z (2011) Code metrics—Maintainability Index. Microsoft.
  49. Oh I-S, Lee J-S, Moon B-R (2004) Hybrid genetic algorithms for feature selection. IEEE Trans Pattern Anal Mach Intell 26:1424–1437CrossRefGoogle Scholar
  50. Ping L (2010) A quantitative approach to software maintainability prediction. In: International forum on information technology and applications (IFITA), 2010. IEEE, pp 105–108Google Scholar
  51. Quinlan JR (1992) Learning with continuous classes. In: 5th Australian joint conference on artificial intelligence. Singapore, pp 343–348Google Scholar
  52. Reddy BR, Khurana S, Ojha A (2015) Software maintainability estimation made easy: a comprehensive tool COIN. In: Proceedings of the sixth international conference on computer and communication technology 2015. ACM, pp 68–72Google Scholar
  53. Riaz M, Mendes E, Tempero E (2009) A systematic review of software maintainability prediction and metrics. In: Proceedings of the 2009 3rd international symposium on empirical software engineering and measurement. IEEE Computer Society, pp 367–377Google Scholar
  54. Shi Y, Eberhart R (1998) A modified particle swarm optimizer. In: Evolutionary computation proceedings, 1998. The 1998 IEEE international conference on IEEE world congress on computational intelligence. IEEE, pp 69–73Google Scholar
  55. Sjøberg DI, Anda B, Mockus A (2012) Questioning software maintenance metrics: a comparative case study. In: Proceedings of the ACM-IEEE international symposium on empirical software engineering and measurement. ACM, pp 107–110Google Scholar
  56. Smola A, Vapnik V (1997) Support vector regression machines. Adv Neural Inf Process Syst 9:155–161Google Scholar
  57. Thwin MMT, Quah T-S (2005) Application of neural networks for software quality prediction using object-oriented metrics. J Syst Softw 76:147–156CrossRefGoogle Scholar
  58. Van Koten C, Gray A (2006) An application of Bayesian network for predicting object-oriented software maintainability. Inf Softw Technol 48:59–67CrossRefGoogle Scholar
  59. Wang H, Khoshgoftaar TM, Napolitano A (2010) A comparative study of ensemble feature selection techniques for software defect prediction. In: 2010 ninth international conference on machine learning and applications (ICMLA). IEEE, pp 135–140Google Scholar
  60. Welker KD (2001) The software Maintainability Index revisited. CrossTalk 14:18–21Google Scholar
  61. Zhang W, Huang L, Ng V, Ge J (2015) SMPLearner: learning to predict software maintainability. Autom Softw Eng 22:111–141CrossRefGoogle Scholar
  62. Zhou Y, Leung H (2007) Predicting object-oriented software maintainability using multivariate adaptive regression splines. J Syst Softw 80:1349–1361CrossRefGoogle Scholar
  63. Zhou Y, Xu B (2008) Predicting the maintainability of open source software using design metrics. Wuhan Univ J Nat Sci 13:14–20CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany 2017

Authors and Affiliations

  1. 1.PDPM IIITDMJJabalpurIndia

Personalised recommendations