Comparing Pre-defined Software Engineering Metrics with Free-Text for the Prediction of Code ‘Ripples’

Abstract

An ongoing issue in industrial software engineering is the amount of effort it requires to make ‘maintenance’ changes to code. An equally relevant research line is determining whether the effect of any maintenance change causes a ‘ripple’ effect, characterized by extra, unforeseen and wide-ranging changes in other parts of the system in response to a single, initial change. In this paper, we exploit a combination of change data and comment data from developers in the form of free text from three ‘live’ industrial web-based systems as a basis for exploring this concept using IDA techniques. We explore the predictive power of change metrics vis-à-vis textual descriptions of the same requested changes. Interesting observations about the data and its properties emerged. In terms of predicting a ripple effect, we found using either quantitative change data or qualitative text data provided approximately the same predictive power. The result was very surprising; while we might expect the relative vagueness of textual descriptions to provide less explanatory power than the categorical metric data, it actually provided the approximate same level. Overall, the results have resonance for both IT practitioners in understanding dynamic system features and for empirical studies where only text data is available.