Keywords

1 Introduction

Agile methods have been practiced for many years, and the challenges agile practitioners face have evolved over time. Initial concerns focused on how to adopt agile software development, and later, on how agile can be scaled to large IT projects [1]. More recently, challenges have moved towards business agility transformation [2], and how to remain agile in the long term [3, 4]. This paper focuses on this last concern, which we refer to as “sustaining agile”, i.e. the continuous process of maintaining and improving agility within an organization.

An organization that has transformed to agility, or in which only some part(s) of it have adopted agile practices may face different issues around sustaining agile [2] than one that has been agile from its inception. An organization that has adopted agile working from its inception may find the question of how to sustain agile puzzling because a key characteristic of agility is continuous adaptation and improvement, and one of the agile principles refers to sustainable pace: “The sponsors, developers, and users should be able to maintain a constant pace indefinitely”. However, by sustaining agile we are not referring to the continuous ongoing flexible adaptation of agile work but to how agile organizations deal with potentially disruptive change, yet continue to be agile.

Studies on organizational change tend to be long term, involving many participants and huge amounts of data collection and analysis. On the other hand, short interview studies can be limited in their depth of insight. To overcome these extremes, we set out to explore a different data collection approach, and to investigate the theoretical frameworks that might shed light on the question of how organizations sustain agility through disruptive change. We also wanted to identify concrete factors, specific examples and practical recommendations that practitioners demand [5]. To that end, we have conducted an initial study to identify factors considered when an organization needed to put effort into sustaining agility. We used a novel data collection approach, critical decision method (CDM) [6], and investigated three theoretical lenses, paradox theory [7], situation awareness [8] and shared mental models, [9] to explore the kind of practical consequences they help to uncover. This paper presents the findings of this initial study together with reflections on the data collection method and the three theoretical lenses; it contributes to the research on sustaining agile by identifying an initial set of factors considered when changes need to be made, and by proposing a data collection approach that focuses on real practice and is more targeted than the typical longitudinal studies.

The next section surveys literature in sustainability, resilience and organizational change. Section 3 describes the study design and Sect. 4 presents its results. Three theoretical lenses and their ability to generate concrete insights are considered in Sect. 5. Section 6 presents lessons learned from this initial study and concludes the paper.

2 Related Literature

The term ‘sustainability’ is used in a wide range of contexts. It is nowadays commonly associated with the UN’s sustainable development goals [10] where it is defined as “meeting the needs of the present without compromising the ability of future generations to meet their own needs”. However, a more general definition of sustainability is “the ability to be maintained at a certain rate or level” [11]. This more general definition is reflected in the literature on organizational change [12,13,14,15] and in software engineering literature focused on sustaining agile [16,17,18].

Buchanan et al. [12] reviewed studies on sustaining organizational change and defined it as “the process through which new working methods, performance goals and improvement trajectories are maintained for a period appropriate to a given context.” They proposed a provisional model for the processes influencing sustainability but the studies they reviewed focus on financial, political, and contextual factors affecting change rather than agile concerns such as customer focus, adaptability to uncertain environments and team empowerment. Holbeche [13], on the other hand, focuses on organizational agility. For this author, organizational agility is the “capacity to respond, adapt quickly and thrive in the changing environment”, and a resiliently agile organization has:

  • “an organizational culture and structure that facilitates change within the context of the situation that it faces;

  • staff who are willing and able to give of their best – in a sustainable way; and

  • a learning mindset in the mainstream business and underlying lean and agile processes and routines to drive innovation.”

This view resonates strongly with our notion of “sustaining agile”. In another work, Holbeche [14, p.54] lists the activities needed in the quest for business agility, gathering them into a circular model with 4 quadrants: agile strategizing, operations, linkages and people practices; at the centre and underlying everything are agile people and culture. But both [13] and [14] fail to address how organizational agility may be sustained. More recently, Miceli et al. [15] propose that sustainability and resilience are distinct but interdependent and that agility supports an organization in building resilience. However, they observe that research has still to identify the factors that are considered in the process of maintaining an organization’s agility.

So although there is a growing interest in agility within the organizational change literature, we found no studies that investigate how organizations sustain their agility.

There have been some investigations into sustaining agile within software engineering literature, but they are limited. Senapathi and Srinivasan [16] view sustaining agile with regards to the six stages presented in Rogers’ diffusion of innovation theory [19]. They see sustainability from a change management point of view and consider it as the post-adoptive stages where a change is accepted, routinised and infused into an organization. In this case, agile is being used, it becomes a normal activity, and it penetrates deeply and widely in the organization. Sedano et al. [17] investigate sustainability on the agile development team level. They define sustainable software development as the “ability and propensity of a software development team to mitigate the negative effects of major disruptions, especially team churn, on its productivity and effectiveness”. Barroca et al. [18], report on what practitioners thought agile sustainability meant. Four themes emerged from this study: being completely agile (whole organisation, mind-set, and principles); being independent (learning and self-sufficiency); being focused on business value and need (user/customer need, value, business need, and appropriate use); and being consistent across time (sustainable pace, and leadership and appropriate technical skills).

Today this is still an under-researched area with little in-depth study of agile sustainability. We therefore formulated our research question as: ‘What factors do IT organizations consider when making changes to sustain agile?’.

3 Study Design

The overall framework chosen for this initial study is the critical decision method (CDM) [20]. This approach uses multi-pass retrospection in which the same event is described more than once, but with differing levels of detail. It is an extension to the critical incident technique [21] to investigate how decisions are made in real situations. It has been used to research team decision-making [8], emergency ambulance control [22], and software flaws [23]; it is an interview method aimed at eliciting specific concrete experiences rather than idealized accounts. CDM’s anchoring in concrete experiences makes it appropriate for our focus on specific factors and practical insights; it allowed us to have a more targeted focus in data collection than that provided by longitudinal studies, while also providing the specificity that it can be hard to get from standard interview studies.

Participants were asked to identify “examples of particularly significant changes they needed to make in order to sustain agility”. More specifically, they identified a change that they believed was relevant to agile sustainability, described it and explained how it was effected. The CDM technique has not, to our knowledge, been applied to this kind of situation before. The study gained ethical approval from the relevant committee of one of our universities.

3.1 The Case Study: AgileCo

We sought a software development company that had experience of needing to make changes to their business in order to preserve their agility, and we were introduced to AgileCo through our collaboration with the Agile Business Consortium and the Business Agility Institute. AgileCo is a large video game developer with players all over the world. The company has been player-focused since their formation, and it is a desire to remain customer-centered that drives them, rather than an explicit need to sustain agility. But “to be customer-centric we have to be agile” (CTO). At the time of the study the organization employed around 3000 people and had 20 offices around the world. Most of their developers are also gamers themselves. Development takes place through iterations, with releases every 2 weeks. Each team is empowered to choose whichever approach they prefer (Kanban, Scrum, etc.), led by a delivery leader who is in charge of delivering the product and is also an agile expert supporting the team to work in the best agile way for them. This autonomy extended to how best to form teams, when to integrate teams, split teams or form new ones. This flexibility and the delivery-led structure extended to other parts of the business such as finance and HR where delivery leaders were also deployed. The company has a strong focus on education and all employees are taught the basics of agility from the moment they join the company.

AgileCo formed in the early 2010s. In the first 10 years they developed and maintained only one game and then the business decided to diversify and develop multiple games. The move to multiple games demanded the centralization of functions common to the different games, such as registration, but key decisions such as release cycles and feature development remained with the teams. This led to a situation in which development activities had to be co-ordinated more closely; teams were encouraged to release their games when ready rather than staggering them. This had implications for co-ordination and prioritization of centralized development. Moving to a multiple-game organization was a strategic decision taken in response to the changing external environment and was intended to maintain business value for the organization and its users, the players. At this time, the organization had highly distributed and empowered teams that knew, and had total ownership of, what they had to do.

This is the context within which we interviewed our participants.

3.2 Data Gathering

AgileCo were approached initially by our collaborators. Following a virtual introduction we interviewed the company’s CTO who put us in touch with further interviewees who were more closely involved in relevant business changes.

The CTO interview elicited the company’s background and was used to produce the description above. Our second interviewee had overall responsibility for two relevant business changes. Subsequently, our second interviewee recommended our third and fourth interviewees. Overall, we performed six interviews, four of which followed the CDM structure described above. The questions used during the CDM cycles are in Table 1. Note that the participants identified what constituted business change and that none of the questions ask directly about the factors considered during this experience. The second interview uncovered two changes, and the subsequent four interviews sought further detail and different perspectives on these two changes. Interviews were conducted and recorded through Microsoft Teams, and then transcribed.

Table 1. Prompting questions asked during the CDM cycles.
Full size table

3.3 Data Analysis

Data gathering resulted in six hours of transcribed interview data. Analysis was based on Wong’s [24] CDM analysis guidance, specifically using the thematic approach. First, descriptions of the two changes identified by interviewees were extracted. Then the specific examples of these changes were examined. Using a reflexive thematic analysis as described by Braun and Clarke [25], factors that the interviewees regarded as important in effecting the business changes were identified. Throughout, our aim was to characterize the situation from their point of view.

Three researchers were involved in the analysis, independently identifying themes and then discussing them. The findings were reported back to the organization for member checking and it was agreed that they reflected their experiences.

4 Study Results

The two changes identified were both in the context of changing from being a single-game company to a multiple-game company: changing the work system to balance individual team and central organization concerns; and prioritizing items proposed by senior managers of the organization. The first emphasizes agility closer to the coal face (the middle/lower layers of activity), while the second emphasizes agility at the higher levels of strategic value.

4.1 Changing the Work System

Changes to the work system were triggered from within the organization itself because of the need for centralization of some functions. The diversity of tools being used by the teams (spreadsheets, Jira, Favro, Kanban) made it difficult for progress towards milestones to be visualised. So, they decided to introduce a new work system that allowed work to be tracked centrally, and be a single point for measuring progress. The process to define this new system was slow, and was carried out in collaboration with the teams, bringing them along and avoiding creating overheads to their work.

The challenge to sustaining agility in making this change is summarized as: “not disrupting things at the coal face but still reap[ing] some benefits of predictability at the top” and the main aim was “finding the balance between empowering their [the teams’] agility and their best practices and giving the <…> large organization value”.

To make this change successfully the organization had to find “the pivot from individual teams owning and doing whatever they want into more of a web of collaboration between teams and discovering what that means and not having to argue from first principles that centralization is evil or wrongheaded every time an optimization is tried to be made for efficiency”. As an agile organization, being light touch and respectful of the autonomy of the teams was a factor that underlined any decisions taken: “we are here as more of a choreographer, or an orchestra leader to help you reveal the strengths of the things you’re building and engender the conversations to happen”.

4.2 Improving Prioritisation

The second change concerned how the organization prioritized work items coming from senior management, and integrated them into the development teams’ work. The example provided by our participants was that the company received an external demand regarding their compliance with a new regulation. This demand needed to be prioritized against other work at the organizational level. The compliance request came in when the teams were “6 to 9 months away from launching 3 to 4 major games”.

To respond to this external request, they broke it down into discovery chunks so it wasn’t “all or nothing” but became a set of tractable scenarios to assess and prioritize value: “make smaller pieces of value or smaller avoidance of pain”; “the discussion quickly boiled down to [..] what is the most clear value”; “we would cycle through those [..] topics until [..] we had an alignment on an absolute rank of [..] priorities”. The change was to approach the problem from a discovery perspective.

4.3 The Factors Considered

To answer our research question we looked for the main factors that were taken into account when making these changes.

The themes that emerged from the analysis of data around the first change were about prioritization, and negotiation of common work (“we turn over all the asks and centrally [..] we do a prioritization”; “we now broker discussions between the people that asked for the work”; “it’s very much a combination of a bottom-up and a top-down negotiation that happens”). The organization had to find a way that did not disturb the teams, was light touch, and also brought the teams along (“we saw it as a choreography of behaviours”).

For the second change, responding to external events by the top of the organization led to work breakdown; this needed to take into account business value, users and risk to the organization. “born a prioritization system [..] pull all the work and all the asks[..]by engaging in this central ritual, this central overhead which gets us no value to the players and no output from your teams the value you get is stability.”

Achieving business value and stability through prioritisation was a slow process requiring negotiation of common work and brokering discussions across the organization (“focusing on choreographing all the work towards a single purpose”; “So in this way we, kind of slowly and iteratively, pulled detail out from the depths of teams, and teams of teams and started to reveal those and have discussions. In doing so, we slowly evolved standards, [..], and in that way we didn’t come in, and just say ‘thou shalt fill out this form with all this detail and then we’ll disappear into our ivory tower and tell the customer what’s going on”).

Having these conversations eventually led to a state where there was alignment, knowability and transparency across teams that could support predictability for central business (“finding the balance between empowering their agility and their best practices and giving the [..] large organization value“; “the value in an organization is not the number of great ideas it has, it’s ability to win all those and make sure only the most valuable ideas actually come down to the teams [..] so giving visibility into that and enforcing the right conversations far enough ahead in the executive team”; “going back to the balance of value versus predictability, what we were trying to do is negotiate for the internal customer, the game team, and align[..] on what were the major chunks of things they needed”).

The themes identified from analysing the whole data set are shown in Table 2. Some of these factors resonate with the themes identified by practitioners when asked about sustaining agile [18], e.g. the focus on business value and user need, and being independent in the sense of respecting teams’ autonomy. Other factors relate specifically to the agile philosophy, e.g. being lightweight and encouraging discussion. Yet others consider the organization as a whole and the need to maintain a common goal, e.g. predictability and alignment.

The results highlight three perspectives that AgileCo took when making changes to sustain agility: an organization-wide perspective; an awareness of the external environment; and, an understanding of how external changes affect its agile teams. But sustaining agile is complex and this kind of analysis does not provide practical guidance and so we sought to extend our interpretation by investigating theoretical lenses.

Table 2. Factors considered when making changes to sustain agility
Full size table

5 Discussion: Three Theoretical Lenses

We consider three theoretical frameworks that reflect the three perspectives identified in our study, and explore whether they may help extract practical insights from our findings: paradox theory, situation awareness and shared mental models. These three were chosen as they have been used in previous studies of agility, and because they resonate with the changes AgileCo undertook, and the factors they considered.

Paradox theory [7] helps organizations to navigate the complexities of decision-making in the context of everyday contradictions. It has been used to understand the tensions encountered in organizational transformation to agility [2] – an organization-wide perspective. Situation awareness is “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future” [26] – which helps organizations understand the external environment and how it may affect them. Shared mental models “are team members’ shared, organized understandings and mental representation of knowledge about key elements of the team’s work environment” [27] – this provides an internal view of the impact of change and how external changes affect agile teams. Each of these lenses is explored in more depth below.

5.1 Balances and Paradox Theory

Strode et al. [2] argue that the tensions organizations faced in agile transformations are dilemmas (i.e. choices) that needed to be addressed at the beginning of a transformation, and paradoxes [7] that persisted beyond transformation and need to be balanced continually. The authors identified 13 tensions from three case studies, and suggested questions that leaders should answer to address these tensions. Paradoxes need attention and ongoing negotiation as part of the continuous improvement of the organization, and hence will be at the core of efforts to sustain agility.

Balances and Paradoxes in AgileCo

The two changes reported by AgileCo and the factors they considered characterise ongoing paradoxes in their desire to sustain agility.

The first change, the introduction of a new work system, resonates, in particular, with the paradox ‘Distributed authority vs macro-level goals’ as identified by Strode et al. [2]. AgileCo, when changing the work system, was faced with the question of ‘how do we guarantee the teams feel empowered and yet understand the organization’s goals?’. Their way of addressing this tension was to balance not disturbing the teams and being light touch with achieving predictability for the organization; this was achieved through collaboration and choreography, putting together quarterly meetings and being explicit about not wanting to disturb the teams.

The second change does not resonate so clearly with any of the paradoxes highlighted in Strode et al. [2]. Instead, it is an example of balancing business-as-usual and business value with the need to address an external demand. We suggest that this is a new paradox that was not encountered in the cases studied in Strode et al. [2]. AgileCo was faced with the question: ‘how do we keep our core business without ignoring unexpected external regulatory requests?’. They addressed this by breaking down the external request into discovery chunks that could be tractable and prioritized.

Using balances and paradoxes allowed us to confirm and extend by one the paradoxes and associated questions suggested in Strode et al. [2]. Using this lens to explore sustaining agility, gives an organization-wide perspective in the identification and understanding of paradoxes related to sustaining agility, and associated questions.

5.2 Situation Awareness (SA)

SA has been used to understand performance under difficult conditions [8], especially operational contexts such as piloting aircrafts, air traffic control, power plants, advanced manufacturing and medicine, where awareness of context is crucial for the tasks to be performed. Endsley [26] defines models for SA in a dynamic decision making context; SA is separate from decision making and consists of “perception of elements in the current situation, comprehension of current situation and projection of future status”. [28].

Several extensions to SA have been proposed. While SA focuses on individuals, Team Situation Awareness (TSA) extends SA to include the situation awareness of other team members and the whole team, and shared SA extends awareness across teams [28, 29]. Distributed SA [30] takes a socio-technical view and considers how systems may also have situation awareness.

Situation Awareness in AgileCo

SA is about the cognitive processes followed prior to decision taking. The context of an agile organization may seem very different from the critical contexts within which SA is typically applied; however, there was a high level of SA in understanding how to sustain agile in the face of change, in particular, in sensing the risk and in focusing on users and business value; in particular, knowing users/players is a strong value for AgileCo and its team members, most of whom are also users/players.

In the context of agile, and of AgileCo, the extensions to SA are particularly relevant as they highlight the interdependence and importance of SA between team members, between teams and across the organization. This was particularly the case when dealing with alignment, predictability and transparency in AgileCo; with all teams being aware of the priorities of others, and being aware of the organization’s goals and how their priorities fitted with these goals. Both of AgileCo’s changes relate to a move from team situation awareness to inter-team and distributed situation awareness. Within an agile team, situation awareness is maintained through various mechanisms including Scrum board, stand-ups and collective ownership [31]. Inter-team knowledge sharing can be supported through similar mechanisms to a degree [32] but AgileCo faced larger scale issues and needed to make collective decisions. In their case, distributed situation awareness needed to be established so that sensible decisions could be made. This relates in particular to the factors predictability and collaboration. However, SA pertains “in the moment” or over a timescale that is finite. For example “DSA can be defined as activated knowledge for a specific task within a system”[30]. In AgileCo, and for other situations of agile sustainability, this would have to be embedded in order to keep the awareness up-to-date and relevant.

Researchers have been looking into what helps develop TSA and shared SA; Bolstad and Endsley [33] mention “requirements, devices, mechanisms and processes”. For AgileCo, as an agile organisation, these were present; team members were made aware of information needed by others through predictability and transparency (requirements), “devices” such as communication and collaboration were also present as were effective team processes. One mechanism they mention, shared mental models, is discussed next. TSA and shared SA are important in sustaining agile, in having a good perception of the external environment. Using this lens to explore the factors being considered during change may help organizations determine how well they are supporting situation awareness in their bid to sustain agility.

5.3 Shared Mental Models

Shared mental models theory [9] explains how teams adapt to change and cope with changing demands [34] when they develop shared mental models. Schmidt et al. [27] use team adaptation theory [35] – “(1) assess situations appropriately and build a coherent understanding of a new situation, (2) adjust their plans accordingly, (3) coordinate their work to fit the new situation, and (4) learn by evaluating their effectiveness” – to propose that “team adaptation is positively related to a high sharedness of team members’ mental models.”.

Shared Mental Models in AgileCo

The way teams are empowered, autonomous and effective in AgileCo indicates that there is a high level of understanding and sharing amongst team members both regarding the tasks undertaken and the way the teams are organized.

The factors considered when dealing with the two changes made to sustain agility, indicate the importance of shared mental models at the team level, for example, the imperative to not disturb the teams indicates a shared understanding of how the teams are structured as being effective. The factors considered during these changes highlight that shared mental models also needed to stretch across teams; for example, negotiation of common work, conversation/discussion under collaboration, as well as alignment, knowability, transparency, and predictability for central business to guarantee predictability for the organization. These are important ingredients in achieving a shared understanding [9, 27].

It is also worth noting that, in AgileCo, negotiation and discussions happened both within the work system but outside it too. Shared understanding was supported in the processes and technical context but also needed to extend to the business context. This negotiation outside the work system relates to “articulation work” defined by Strauss as “the specifics of putting together tasks, task sequences, task clusters – even aligning larger units such as lines of work and sub-projects – in the service of work flow” [36].

With shared mental models consideration of sustaining agile needs to go beyond their original focus on individuals and teams, and focus on the wider organization and across teams. Using this lens to explore the factors being considered during change may help organizations ensure that shared mental model practices are supported both within teams and across the organization, hence leading to better shared understanding.

Figure 1 summarises the three theoretical frameworks and how they support different perspectives in sustaining agile.

Fig. 1.
figure 1

The three theoretical frameworks support different perspectives in sustaining agility

Full size image

6 Conclusions

Limitations

A key methodological issue for CDM studies relates to how closely the interviewee was engaged in the events described. In our study, the first interviewee was responsible for deciding the change needed, and in designing the new work system. S/he also was involved closely in prioritization improvement. In this sense s/he personally experienced the decision-making. The other two interviewees provided different perspectives on the situation and were also personally involved in the development of and working within the resulting system.

The data was collected by the two first authors, who also carried out the analysis and discussed it with the research team. The findings of this study have been presented and discussed with key members of the participating team.

This is an initial study focusing on one agile organization and the factors considered when it needed to sustain agility. The data collected and the conclusions drawn directly from it are therefore limited. But our aim was to trial a different data collection approach and analysis frameworks that could be used to research this phenomenon. We used CDM, a novel approach to data collection, and investigated how three theoretical lenses may be used to interpret the resulting data to uncover deeper insights and practical consequences. Our intention was exploratory aiming to identify possible ways for further research in an area that has not been widely researched.

Lessons Learned

In this work we used CDM as the data collection framework together with thematic analysis to identify factors that organizations consider when making changes to sustain agility. We then explored three theoretical lenses to see what kind of insights they may yield when interpreting the findings.

Using CDM supported our goal of finding the detail and specific examples of changes that AgileCo had to make. The multi-pass retrospection approach elicited detail, but was challenging to apply because of the perceived repetition from our participants. Although the questions targeted different aspects of the change at different times in the enquiry the same issues were visited repeatedly. Interviewing people from different roles provided different perspectives on the same events. We would recommend others to use this approach, but encourage them to train interviewers and prepare interviewees for the cyclical nature of the interrogation.

A thematic analysis identified factors that were being considered during changes needed to sustain agility, and based on this, we identified three potential theoretical lenses to extend our findings further. For each of these we have demonstrated the kind of practical insights that could be extracted from the data. These frameworks illuminated three key dimensions relevant to sustaining agility: teams vs organization, understanding the environment vs understanding how to deal with its impact inside the organization; and understanding the now vs looking into the future.

Based on this initial study we believe that approaching an investigation of sustaining agile in this way can yield interesting and practical results.