Keywords

1 Introduction

While many organizations embark on an agile transformation to make their businesses more agile and responsive, the actual impact of those transformation efforts is often not well understood. Agile transformations are a relatively new and complex organizational phenomenon. Initially developed and applied within individual teams and initiatives with a focus on software development, organizations have begun to apply it at an enterprise level, impacting multiple organizational layers such as teams [6, 27], programs and portfolios [28], as well as multiple business domains such as HR, finance and sales [21].

While agile transformations are frequently thought to provide better alignment with client needs, better involvement of business and users, as well as better and more transparent planning [28], their impact is historically understood from the perspective of an agile software development capability due to their roots in that domain [6]. Current studies consider their impact only on individual levels, mostly within teams and individual organizations. For practitioners, this is problematic, as it is difficult to understand the expected benefits and how those benefits relate to the necessary investments required to adopt agile ways of working.

In this paper we report on our international study, presenting for the first time a view on the impact of agile transformations across the domains of portfolio, programs and teams. To academia we present a model of agile maturity and organizational performance, depicting how organizational performance is impacted through growth of agile maturity across the levels of portfolios, programs and teams during agile transformations. To practice we provide examples of what impact organizations can expect from undertaking an agile transformation.

2 Related Work

In this section we will briefly elaborate on the definition and history of agile transformations with their associated challenges, and then discuss extant literature on the impact of agile transformations on organizational performance. This will transition into the research question as posed in the following section.

2.1 Agile Transformations and Their History

In order to overcome the challenges of adoption and achieve the benefits of agile methods in the context of larger enterprises, organizations embark on a Agile Transformation Process [8, 13]. The development and adoption of agile at scale, agile project and portfolio management can be historically described in at least four successive stages:

(1) Team-level agile: At first, in the late 1990 s, a number of frameworks and methods were created to deal with an increasing number of failing software development projects. The academic roots lie with Takeuchi and Nonaka [29], whose product development game was translated and expanded into frameworks such as Scrum [24] and XP. Today these frameworks are known under the umbrella of agile methods, facilitating shortened feedback loops and better aligning work with business needs.

(2) Cross-team and program-level agile: The success of Scrum leading up to 2000 brought about the desire to execute larger initiatives in an agile way. However, as Scrum was originally designed for initiatives or projects with an optimal team size of 5–9 individuals, organizations began experimenting with ways to coordinate several agile teams to deliver initiatives requiring larger workforces. This led to the creation of Scrum of Scrums. Other frameworks focusing on smaller organizations have been developed, such as Nexus, which limits itself to 80 people.

(3) Enterprise agile: An increasing adoption of Scrum in organizational settings, while successful on the one hand, challenged the existing roles and responsibilities in organizations applying these frameworks. As Scrum requires more interaction between users, sponsors and teams, it often clashes with standard organizational structures and workflow [5]. Hence, as of approximately 2010, frameworks began appearing that allowed the embedding of large-scale agile IT initiatives into enterprises, the most prominent being the Scaled Agile Framework (SAFe) [12], LeSS (Large-Scale Scrum) and the Spotify model.

(4) Business agility: From 2018 onwards, companies and framework creators adjusted their thinking from IT-driven to organization-wide agility [1]. While the term can be traced back to earlier academic literature [16], terms such as agile finance, agile marketing, agile sales and agile HR began appearing later [21].

2.2 Understanding Individual Transformation Journeys

Strategies Employed: Frameworks tend to have ideal implementation roadmaps, but the implementation strategies actually employed may vary. For example, there is a team-per-team transformation, a department-per-department implementation and a ‘big bang’ approach to full organizational change. Further, a new department or even company can be set up to be agile-native. Different companies approach this in different ways, and therefore achieve different results.

Challenges Involved: When undergoing agile transformations, commonly reported challenges are (1) resistance to change, (2) difficulty of implementation resulting from vague terminology and a lack of clear guidance from literature and (3) the integration of non-development functions, e.g. projects being iterative means funding needs to be iterative, which is not always the case [5, 18].

Agile Maturity Across Differing Organizational Layers: Companies have transformed to become agile to varying degrees across different organizational layers, including teams, programs and portfolios. All three layers are addressed by the SAFe framework [12], but other frameworks such as LeSS, the Spotify model and the business agility framework do not necessarily acknowledge them.

The SAFe layers can be summarized as follows [12]: Team level: a set of teams responsible for the development of User Stories based on Features identified at the Program level. Program level: a team of teams building upon a set of 5–12 teams, responsible for the development of Features to be developed by the underlying teams. Portfolio level: the portfolio management team defines the strategic themes, translates those into a portfolio-backlog, and allocates it to respective program layers as appropriate. Thus, based on the scaling principle, the program layer builds on multiple underlying teams. While portfolios build on multiple programs respectively, they apply a different workflow in which the programs within the portfolio might be competing for resources [28].

Progress in becoming agile can be measured in terms of agile maturity, using the five levels identified and proposed by Laanti [14], as shown in Fig. 1.

2.3 The Impact of Agile Transformations

Practitioner literature promises high-impact numbers in decent alignment with reported metrics. For example, SAFe presents the following benefits [12]: 10–50% happier, more motivated employees, 30–75% faster time-to-market, 20–50% increase in productivity, 25–75% defect reductionFootnote 1. However, these figures are currently supported only by anecdotal evidence from supplied case studies with limited reproducibility, as reported without e.g. sample size or calculation methodology. Additionally, practitioner frameworks generally report qualitative improvements as opposed to strict metrics.

More academically, previous research by Laanti [15] outlines a model to evaluate organizational improvements, and applies it to measure the perceived benefits of agile. In this research, Laanti collects agreement with statements that claim, for example, quality improvement as a result of agile transformations on a seven-point scale, as presented in Table 1. Reported values show an average reported mean of 5, suggesting some, though limited, agreement. The natural limitations of this are that degree of improvement is not measured, and a lack of differentiation is made between organizational levels. Olszewska [20] provides a quantitative analysis of performance in agile contexts. Both find positive results, and in combination with other studies, agile has started being associated with factors such as (1) improved quality [25], (2) added value [22], (3) faster time-to-market, (4) better responsiveness to change [15, 20] and (5) lower development cost [30]. Past studies have the shortcoming of being limited to case studies, or not adjusting for the penetration of agile throughout different organizational levels. Concretely, solely IT being agile as opposed to the entire organization being agile may result in different organizational improvements.

Fig. 1.
figure 1

Transformation maturity model based on Laanti [14]

Full size image

3 Research Question

Extant literature suggests adopting agile methods across the layers of an organization, meaning pursuing an agile transformation is associated with improved organizational performance. However, empirical support is scarce, primarily researching agile at team level. Past research has not considered agile presence at different organizational levels, and practitioner frameworks even go so far as to claim that adopting agile methods is associated with equal performance improvement at all levels, meaning the team, program and portfolio level. This research aims to fill that gap by investigating organizational performance associated with agile transformations at those differing levels. As such, the research question of this paper is: What are the benefits of undergoing an agile transformation?

4 Methodology

In order to both answer our research question and attain a suitable level of external validity, data from a wide range of agile practitioners was needed. Hence, we developed a survey to obtain a suitable sample size. Surveys are easy to distribute, thus facilitating sample size, and are self-administered, thus minimizing desirability biases through anonymity [11].

4.1 Survey Design

In order to answer our research question, we designed a survey in five segments: (1) Context; (2) Agile transformation; (3) Agile maturity; (4) Organizational performance; and (5) Satisfaction. To improve survey quality, multiple versions of the survey were tested and subjected to expert input. This resulted in adjusted questions to ensure all participants were able to answer.

(1) Context: The survey first gathers relevant descriptive information to verify representativeness of the sample. It includes questions on the participant’s role in the organization and its transformation, as well as the company’s size and industry.

(2) Agile transformation: This section first verifies whether the company has completed an agile transformation, or is in the process of one, or plans to transform in the future. Second, it explores what strategy was employed to initiate the transformation, and what transformation framework was utilized. Third, the scope of the initiative is assessed, verifying how much of the organization is transforming.

(3) Agile maturity: Laanti [14] presents a maturity model distinguishing three common organizational levels in agile: portfolio, program and team. If sufficient effort is put in, companies move through these levels from beginner, novice, fluent, advanced to eventually world-class. What needs verification is as follows: as companies scale their agility, will the benefits of agility scale as well? Similarly, as companies level up their agility, will the benefits follow? Participants were asked to rank their company in terms of proficiency for each organizational level.

(4) Organizational performance: To evaluate the impact of the agile transformation, participants were asked to input their perceived percentile improvements on the following metrics, as adopted from Laanti [15]: effectiveness of development; quality of the product; transparency of development; increased collaboration; work being more fun; work being more organized; work being more planned; autonomy of development teams; earlier detection of bugs/errors/defects; work being less hectic. These measures were employed to facilitate comparison with existing literature.

(5) Transformation satisfaction: Satisfaction with the transformation was evaluated on a seven-point Likert scale with the following question: How satisfied are you with the results of the transformation (so far)?

4.2 Data Collection

The survey was distributed through online communities on platforms such as LinkedIn and personal networks. Furthermore, to ensure sample representativeness, relevant practitioners were approached directly, simultaneously targeting relevant companies and ensuring a variety of seniority level. The collection period spanned three weeks from 11 July to 21 August 2018. During this time frame, 264 people started the survey, and 134 completed it, resulting in a response rate of 51%.

5 Results

5.1 Descriptive Statistics

Roles of Participants: The principal roles of participants within the different transformations as reported were Agile Program Coach (26.12%) and Transformation Manager/Lead (21.64%). Other roles were Team Coach (21.64%), Transformation sponsor (8.21%) and DevOps coach (5.22%).

Size of Organizations: Most participants are employed in large organizations. This means that most companies in this sample have more than 50,000 employees (38.06%). The next largest group of respondents (23.88%) work within companies with an employee number of between 1,001 and 5,000. The remaining respondents were from companies with fewer than 1,000 employees (19.4%), between 20,001 and 50,000 employees (9.7%) and between 5,001 and 20,000 employees (8.96%).

Industry: A large proportion of the participants come from three specific industries. These industries are software (21.64%), financial services (17.91%) and professional services (15.67%). The probable reason for these higher percentages is that the agile way of working is more prevalent within these industries. The remaining responses were from workers in the following industries: telecoms (6.72%), utilities (5.97%), health care (4.48%), retail (4.48%), government (3.73%), manufacturing (3.73%), consumer products (2.99%), public services (2.24%), transportation (2.24%), insurance (0.75%), media & entertainment (0.75%), internet services (0.75%) and education (0.75%).

5.2 Transformation Details

Transformation Strategy: Participants were asked about the strategy used to implement large-scale agile within their company. Most companies used the bottom-up (team-by-team) strategy (42.54%), whereas others used the department-by-department strategy (29.1%), the big-bang strategy (11.94%), the new department strategy (7.46%) and the new company strategy (0.75%). Eight per cent of the participants said they used another strategy or no strategy at all.

Agile Frameworks Applied: The largest group of participants reported using the Scaled Agile Framework®(SAFe®) (42.11%). Other participants reported the use of Scrum of Scrums (19.55%), internally created methods (14.29%), Enterprise Scrum (3.01%), Large-scale scrum (2.26%), Lean management (2.26%) and Nexus (0.75%). Other participants responded they used another or no framework for the transformation (15.79%).

Capital Investment: The largest group of participants indicated that the transformation had needed an investment of between €500,000 and €2 million (26.72%). The other participants responded that the investment needed was between €2 and €10 million (25.19%), between €100,000 and €500,000 (16.03%), less than €100,000 (14.5%), more than €50 million (9.16%) and between €10 and €50 million (8.4%).

Fig. 2.
figure 2

Agile maturity across portfolio, program and team layers

Full size image
Table 1. Perceived impact of agile transformation on various metrics associated with dimensions of organizational performance. Mean and three quartiles are reported for our own data. For comparison (discussed in Sect. 6.2), reported means are reproduced from Laanti [15] and Olszewska [20], and reported ranges from SAFe [12]. Laanti reported on a seven-point Likert scale, rather than improvement percentage.
Full size table

Maturity: As seen in Fig. 2, on team level the biggest group of participants assessed their company as being at advanced level (29%). The other participants estimated that their organization was fluent at team level (23%), novice at team level (20%) and beginner at team level (19%). A smaller group estimated it to be at world-class level (8%). On program level, the biggest group of participants estimated their company to be at novice level (32%). The other participants estimated that their organization was at beginner on program level (26%), fluent on program level (18%), advanced on program level (18%) and world-class on program level (7%). On portfolio level the biggest group of participants estimated their company at beginner level (38%). The other participants estimated that their organization was at novice level (25%), fluent at portfolio level (17%), advanced at portfolio level (14%) and world-class at portfolio level (7%).

Satisfaction with Transformation: The biggest group of participants was moderately satisfied with the results of the transformation (38.06%). Other groups of participants were slightly satisfied (20.9%), extremely satisfied (18.66%), slightly dissatisfied (8.21%), neither satisfied nor dissatisfied (7.46%), moderately dissatisfied (3.73%) and extremely dissatisfied (2.99%).

Organizational Impact: Our results regarding perceived impact of agile transformations on various dimensions of organizational performance are reported in Table 1. For each dimension, the perceived impact on one or more associated metrics was measured. For these, we show the mean impact, as well as the impact at first, second and third quartile of the distribution. The second quartile (median) reflects the central tendency, which is highest for Increases collaboration at \(75\%\) improvement and lowest for Makes work less hectic at \(50\%\) improvement. The first and third quartiles capture the inter-quartile range (IQR) showing smallest dispersion for Increases collaboration with IRQ \(=91\%-60\%=31\%\) and largest dispersion for Makes work less hectic with IQR \(=73\%-21\%=52\%\).

Fig. 3.
figure 3

Correlations between maturity at different organizational layers and metric improvement per dimension. In bold we show the strongest correlations (above 0.45).

Full size image

5.3 Correlation Analysis

The model shown in Fig. 3 shows the correlations of agile maturity on a particular layer with organizational performance per metric. Pearson correlations were used to allow for significance testing and due to the linearity of the expected relationship between maturity and performance. Except Makes work more fun, all metrics correlated with agile maturity are significant at P<0.05 across all organizational levels. The strongest correlations, and thus the largest improvements associated with improved agile maturity, are as follows: Increases the effectiveness of development with coefficients between 0.45 and 0.51; Enables the earlier detection of defects improving between 0.36 and 0.44; improvements in Makes work less hectic between 0.36 and 0.43. Notably, no metrics seem to respond uniformly across organizational levels. In terms of the proposed research question, all metrics seem to undergo relevant improvements as a result of undergoing an agile transformation.

6 Discussion

Our discussion starts with general observations (6.1), and then goes into separate dimensions (6.2) and organizational layers (6.3).

6.1 The Impact of Agile Transformations: General Observations

We now discuss the impact of agile transformations, comparing our data with existing literature, beginning with general observations, and continuing with a discussion of the individual dimensions in Sect. 6.2. To allow comparison of results across existing studies and our data, we grouped the metrics of Laanti [15], Olszewska [20] and SAFe [12] into a hierarchical classification in Table 1.

Our data suggests that agile transformations positively impact organizational performance across all employed metrics. An observation one can make while looking at Table 1 is that all studies report rather large improvements.

Further, one can observe that the impact differs per organizational layer. For example, an increased maturity on program level shows the strongest impact, correlating with the dimensions of Productivity, Quality and Workflow health (see Fig. 3). Most notably, increasing agile maturity at the program level has a correlation coefficient of 0.511 with Increases the effectiveness of development, and one of 0.450 with Increases the quality of product, as shown in Fig. 3. The weakest correlation is found with Makes work more fun at the program level at 0.212. Satisfaction with the transformation is positively associated with performance benefits, except for the Improves time-to-market metric. However, dissatisfied respondents still report an average performance improvement of 45%. The highest satisfaction shows an average performance improvement of 77%. This is in line with previous research finding that satisfaction with change predicts performance benefits of that change, while asserting that agile transformations in and of themselves are beneficial.

Comparing our data to the results of Laanti [15], Olszewska [20] and SAFe [12], one can observe that reported performance improvements vary significantly. We identify three potential interpretations of this cross-study variance. A rather obvious explanation is that the applied metrics differ and the measurements have been taken differently across the various studies. An increased acceptance and maturity of agile methods in practice could be another potential explanation. Yet another reason could be that the impact is contextual and varies greatly across the surveyed organizations. Previous studies and professional reports (cf. State of Agile Survey [26] and Business Agility Report [1]) indeed indicate an increase in acceptance and adoption of agile methods in practice.

6.2 Impacted Metric Dimensions

We will now continue to discuss the individual metrics presented in Table 1 and their measurements.

Overall, our results correlate with previous findings that organizational performance improves as a result of agile transformations. Differences can be found, however, in the magnitude of this improvement, with Olszewska occasionally reporting very high numbers [20]. These differences may be caused by the differences in sample, since both Laanti and Olszewska report findings from within a single organization. Furthermore, comparison is challenging due to different scales and operationalization of concepts. Notwithstanding these differences, the benefits of agile transformations are confirmed through improved replication across a larger sample.

We will now continue to discuss the impact of agile transformations across the five identified dimensions of (1) Productivity, (2) Responsiveness, (3) Quality, (4) Workflow health and (5) Employee satisfaction & engagement, with their respective metrics as illustrated in Fig. 3.

Productivity: Defined as total output divided by total input, improvements in productivity come from both efficiency and effectiveness [2]. Factors influencing productivity in software development can be categorized into product (e.g. complexity, size), process (e.g. maturity) and development environment (e.g. languages, development tools)  [17, 31]. SAFe reports a 50% improvement in productivity in general. On Increases the effectiveness of development, our reported improvements almost double Laanti’s results.

Responsiveness: Responsiveness here refers to quickness of response to either customer or market demand [2]. Responsiveness in software development is generally associated with a mature use of agile practices and processes [23] and team configuration (e.g. application of feature teams with an end-to-end responsibility over a (sub)product [19]). We report similar Time-to-market improvements at 67% as SAFe’s 20–70%, though without a range, which suggests our results may be higher. More concretely, Olszewska reports an improvement of 24% and 64% on Customer service request turnaround time and Lead time per feature respectively. This shows that responsiveness can be expected to improve, but depending on operationalization, different results may be achieved.

Quality: Defined as a measure of excellence, both product and development quality fall under this category [2]. In software development, the ISO 25010 standard delineates two overarching categories: product quality and quality in use. These have eight and five categories respectively, which in turn have 31 and 11 sub-categories [10]. While the complexity of quality may therefore not be fully covered in existing agile literature, results seem to unite in finding benefits of agile transformations. Laanti reports a fair degree of agreement on Increases the quality of the product (4.70/7) and Enables the earlier detection of defects (4.70/7), which is mirrored by our results of 61% and 67% respectively. The Increases the quality of the product and Enables the earlier detection of defects reported here fall into the range reported by SAFe under defect reduction, as does Olszewska’s Average days of open external reports (31%). Notably, the Number of external trouble reports worsened, meaning increased, with 188% as reported by Olszewska [19]. This may have been caused by the increased number of releases, since there would be more opportunities for trouble reports to be logged within the same time frame.

Workflow Health: Workflow refers to the way that work is organized, or the sequence of steps in a work process [2]. A workflow can be called healthy, when the work is well organized and planned, in which case individual tasks are executed and (intermediate) products are delivered at a steady pace. Thus, the notion of workflow health concerns the internals of the work process, which is linked to, but distinct from, the other categories such as employee satisfaction or productivity. The results presented here correlate with existing literature in emphasizing the benefits to be achieved by pursuing an agile transformation. A transformation Makes work more organized by 55%, and Laanti reports agreement on improvements here (4.50). Olszewska’s metric of the Number of days between commits reports a slightly lower improvement of 38%. Olszewska also reports an impressive improvement in Number of releases per time of 400%. Important to distinguish is that increases in Number of releases per time are affected not only by workflow health, but also by the size of an individual release. In agile software development, individual pieces of work are to be small, suggesting the workflow health improvement, its presence implicitly agreed upon, may not reach 400%.

Employee Satisfaction and Engagement: This section here follows the definition: employees being happy and actively engaged with their job due to the job itself and the overall working conditions [2]. Higher satisfaction and engagement leads to higher individual performance, and literature has found a positive relationship with firm growth as well as retention rates [3, 9]. Increases collaboration and Increases the transparency of development are the highest evaluated metrics in Laanti’s study as well as ours. Interestingly, all metrics except Makes work less hectic (49%) exceed the range given by SAFe of 10–50%. Makes work less hectic is also the only metric where participants reported a negative impact, with 3.61 [15].

Customer Satisfaction: The table notably does not include this concept, which is a limitation of our data set as well as of existing agile literature. A faster time-to-market is not valuable if what is delivered is not valued by customers.

6.3 The Relevance of Organizational Layers

In this subsection we will discuss the correlation of impact with reported maturity across teams, programs and portfolios. Figure 3 presents the results of the correlation analysis between the maturity model and the impact dimensions.

While the impact dimensions discussed by Laanti [15], Olszewska [20] and SAFe [12] generally do not discriminate between organizational layers, looking at the types of metrics used, it can be assumed that those metrics are most applicable at the program and team layers. Due to the absence of portfolio-level metrics (e.g. portfolio performance metrics such as maximizing the portfolio’s overall economic value, strategic alignment and portfolio balance, or satisfaction metrics such as decision effectiveness [4]), for the sake of comparability, we will therefore discuss the impact on the program and team layers.

Following our analysis, we see that the perceived impact on performance is not consistent across organizational levels, meaning that performance improvement at the portfolio level is not equal to that at the team or program levels.

Notably, only the Makes work less hectic improvement increases when scaling from program to portfolio level. All other metrics’ improvement is decreased to varying degrees. Since scaling agile to the portfolio level is a relatively new phenomenon, the associated complexity may not allow the benefits to actualize to their full extent. Another explanation may be that intrinsic differences in the benefits of agile at the portfolio and program level exist. The portfolio level oversees several programs whereas the program level is, simply put, a group of teams. Making a parallel with a portfolio of business units is useful. The coordination costs of having multiple business units is well established in literature, and significant changes in the business unit operation necessitate adjustments in how the portfolio is managed. This would suggest that scaling agile to the portfolio layer adds a degree of complexity that companies may not be equipped to manage. This interpretation is supported by the fact that the lowest agile maturities are reported at the portfolio level. Further, at the portfolio level, different metrics matter, e.g. Improves time-to-market is likely to be valued higher than Enables the earlier detection of defects. The program level is a group of teams, suggesting that improvements as a result of an agile transformation should follow the same mechanisms. However, the portfolio is not a sum of programs and follows a different scaling mechanism [28]. For these reasons, it is not unusual for performance improvement at the portfolio level to be perceived differently (i.e. lower) than at the program and team level.

6.4 Limitations

Although we applied a rigorous method when designing the questionnaire and collecting the data, multiple limitations are present in the current paper. Particularly, in survey-based research, three main types of bias can be found and are discussed here: sampling bias, response bias and non-response bias.

Sampling bias is the bias related to the way respondents are selected. We note that the majority of responses stem from participants who tend to be responsible for agile transformations (e.g. Agile Program Coaches and Transformation Managers and Leads), which may lead to self-selection bias, but also to more high-quality answers, as those participants are expected to have the best overview of the transformation. We addressed sampling bias by sharing the survey with different communities rather than those purely involved with agile methods. Nevertheless, self-selection bias might have occurred as a result of the research being an online survey. Responses from additional business stakeholders and software developers affected by the transformation would be a valuable addition to future research.

Response bias can be encapsulates friendliness bias and social desirability bias. As perceptions rather than concrete improvements (‘hard-data’) were collected, the results are open to potentially biased responses. The scale used for organizational performance could be a source of bias as it was presented as a 0%–100% range. This may have deterred participants from selecting 0, and also did not allow participants to indicate a decrease in performance. However, results are somewhat robust since participants who reported low satisfaction with their agile transformation still reported a minimum performance increase of 40%. As the survey was anonymous, we further believe that response bias due to socially desirable answers was mitigated.

Non-response bias is a bias where participants are unwilling to take or complete the survey, resulting in under-representation of specific viewpoints. With a rather high response rate of 51%, we believe that the non-response bias is limited. Still, negative opinions are under-represented, at under 10% of participants. This may suggest that overall, transformations go smoothly, but might also indicate that the dissatisfied group is under-represented in this paper.

7 Conclusions

In this paper we have presented the results of our empirical study on the impact of agile transformations on organizational performance. Based on an international survey with 134 participants from varying industries and nationalities, we discuss the perceived benefits.

The contribution of this paper is threefold: (1) we consolidated understanding of the benefits of agile transformations based on prior evidence and our data from a more diverse and larger sample; (2) we identified the dimensions impacted by agile transformations as being productivity, responsiveness, quality, workflow health and employee satisfaction; and (3) we traced specific benefits on those metric dimensions to individual organizational layers of teams, programs and portfolios, showing the magnitude of impact of each metric per layer.

First, by comparing existing quantitative results with both academic and practitioner literature, we conclude that agile transformations indeed result in widespread organizational performance improvements. Importantly, depending on the operationalization of a specific concept, exact results will vary between studies. However, the identified dimensions are confirmed to be positively impacted by the pursuit of an agile transformation with limited dependence on the selected metrics. Second, based on existing literature, including input for the described comparison, a comprehensive overview of the impacted dimensions is established. From an investigation of the non-agile literature, we posit the unsurprising notion that different metrics are appropriate at different layers. A particularly strong candidate is strategic alignment of projects at the portfolio level. Further, we point out the lack of further, customer-facing metrics, e.g. customer satisfaction. Third, besides the applicability of individual metrics, we confirm that the benefits of agile maturity on specific metrics differ according to the organizational layer. For example, workflow health sees practically equal improvements at the team and portfolio level, but notably higher improvements at the program level.

Overall, the level of granularity in understanding these performance benefits is improved by acknowledging organizational layers and the differences of performance benefits of agile transformations between them. A more fine-grained understanding of the impact at different dimensions and layers achieved through our research opens the possibility of building an integrated model of maturity, impacted metrics and organizational layers, where inter-dependencies within and across these topics can be investigated. This facilitates a potential adoption and growth model for organizational agility to optimize transformation paths for impact. A very recent publication [7] that presents a multi-factorial model of developer productivity seems to have independently taken a similar approach to ours, in the way it looks at various dimensions and organizational layers, and discusses similar metrics to ours. While the authors of that research take the notion of productivity as their focal point, our perspective is agile transformation, where productivity is just one of various dimensions.

We conclude that agile transformations positively impact organizational performance, with reported improvements in many cases going way beyond 30% across the reported dimensions.