Keywords

1 The Next 20 Years in Agile

The first XP conference was held in 2000 in Cagliari, Italy – and that conference attracted many software practitioners who were excited about lightweight and iterative development methods like Extreme Programming. Today, at the twentieth XP conference, there is still a lot of interest and excitement about Agile among practitioners and researchers. Twenty years from now, will there still be the same level of enthusiasm for Agile software development? Maybe everything will be called Agile. Maybe all large software product development projects will use a single standard Agile scaling framework similar to one of the approaches being used today. Or maybe today’s Agile practices will have evolved into new and better development methods.

The members of the panel came from all corners of the world and all parts of the Agile community. Deepti Jain is an Agile practitioner and coach, and she is the founder of AgileVirgin, based in India. Nils Brede Moe is a research manager at SINTEF in Norway and a researcher at Blekinge Institute of Technology. Helen Sharp is a professor at The Open University in the United Kingdom, and her research interests include human and social aspects of software development. Ken Power is an independent consultant based in Ireland. Philippe Kruchten is a professor of software engineering at the University of British Columbia, in Vancouver, Canada. Steven Fraser, the panel impresario, is based in California where he advises on tech transfer and open innovation strategies for Innoxec. Previously he was the Director of the Cisco Research Center and the Lead for HP’s Global University Programs – and he has organized and delivered over 75 software engineering conferences, panels, and workshops. Philippe and Steve served as the Program Co-chairs for XP 2019.

The majority of the panelists agreed that Agile will be around 20 years from now, in some form:

  • Helen Sharp: “Agile will be around for quite a while to come yet. In 20 years, we may not be calling it that, but it will still be here.”

  • Deepti Jain: “I think that in 20 years, we will be working more independently. We will look forward to creating something that works best for us.”

  • Nils Brede Moe: “I have been doing research in Agile for the last 15 years. I’ve seen many different companies and many different cultures. It’s being practiced differently, and I think for the next 20 years, it will still be practiced differently.”

  • Ken Power: “In terms of the next 20 years – I have no idea… I’m optimistic we will see a lot of interesting things in the next 20 years.”

  • Philippe Kruchten: “Agile is there forever. It’s not going to be replaced. Maybe the adjective ‘Agile’ will diminish a little bit. But the mindset behind Agile, which is to constantly observe your environment and react or respond to it as fast as possible, that’s going to be there forever.”

2 The Next 20 Years Panel Discussion

At the end of a long and exhausting week, this panel still had the energy to discuss many interesting topics. Most of them agreed that Agile will be around in some form 20 years from now, but it might have a different name in the future. The panelists explained that the Agile Manifesto will have diminishing value to the Agile community over time. The panelists offered their opinions about the future of Agile scaling frameworks and related tools. They complained that the frameworks add too much complexity, and they also limit the ability of teams to tailor their way of working. Another important discussion was about how to increase the collaboration and communication between researchers and practitioners.

In their introductions, the panelists each presented their favorite issues relating to Agile methods in the future. Deepti Jain explained that 20 years from now, developers will be working more independently. Some of the current Agile frameworks will be too constraining: “Many folks will not want to be attached to an organization, or a framework or a structure.” Nils Brede Moe believed that there will be many useful applications for today’s research work on autonomous teams. But Nils saw technology transfer as a major issue: “As researchers, we need to be better at bridging research and practice.” Helen Sharp was sure that there is a future for Agile methods: “Agile was here before we called it Agile. And Agile will be here 20 years from now whether we call it agile or we call it something else.” Ken Power complained that he has no idea where things will be in 20 years, but he had advice for the near future. He claimed that the real innovators in Agile work will not waste their time arguing about labels. They will be exploring the interactions of Agile with other computing technologies: maybe flow-based development, maybe artificial intelligence and machine learning.

Philippe Kruchten injected a bit of humor into his introduction – but his serious point was that there will be a swinging pendulum of support for Agile. In the short term, he thought that the likeliest reaction to Agile will be popular obsession in the practitioner community. (“All business functions will become Agile: Agile accounting, Agile marketing, Agile kindergarten. SAFe becomes an ISO standard, Scrum is taught in grade 3.”) But in 10 years or so, this obsession may be followed by a revolution rejecting Agile.

Philippe also offered a list of interesting technology work in Agile, enough to keep researchers busy for 20 years. We have to learn to avoid a “one size fits all” process. We think we know what coordination, communication, and collaboration are – “but when you dig a little bit, you realize it is extremely different across the spectrum of the field into which we apply Agile.” We need to learn more about project governance and the concept of value.

There was a question about the future of the Agile Manifesto. “Is it a document that gets revered and a religion emerges around it? Or is it sent to the dustbin of history?” Helen asked “Shouldn’t it evolve? Shouldn’t it be Agile and be changed as time goes on?” Steve Fraser, the panel impresario, recalled the conclusions of the Agile Manifesto panel from earlier in the week. That panel had agreed that the Manifesto is not going to be changed. There had been a discussion about the origins of Agile and the significance of the Manifesto: “There was the observation that Agile existed before the Manifesto. But the Manifesto was seen as a defining moment, and it did both good and perhaps had some negative influence on the culture of the community.” Philippe looked to the future: “I think the Manifesto has served its purpose. We should stop referring to it. We know the flaws that it contains, and we should just move on. You cannot define that your organization is Agile by compliance with the Manifesto.”

Helen and Philippe were skeptical about claims that we will have robots doing all of the coding work in the next 20 years. Helen pointed to similar forecasts made 35 years ago, when “expert systems” were supposed to put programmers out of work. She thinks that “software development will change, but I don’t think it will change in that way… I don’t think it’s going to be automatic.” Philippe explained that AI and Machine Learning may bring some new tools to support programmers, such as tools to help them navigate large and complex code bases. But automated tools won’t be able to create new designs. “The real decisions about design, I don’t think they can be automated.”

Ken also hoped for technology to give us better programming tools. “The bottleneck in software development is not the interface between the developer and the keyboard and the computer. The bottleneck, especially as systems get larger, is learning and creating shared understanding. I think we’ll see a lot of innovation improving learning and tools to help with shared understanding. I think that’s where the innovation will be over the next 20 years.”

One questioner (Maria Paasivaara from IT-University Copenhagen and XP 2020 General Chair) asked about Agile scaling frameworks: “What is the future regarding scaling frameworks? Are we going to have all companies using some of the scaling frameworks? Or will they disappear?”

At the beginning of the XP 2019 conference, Scott Ambler had delivered a provocative keynote talk titled “#NoFrameworks: How We Can Take Agile Back!” [1]. In that talk, Scott explained that Agile teams should be allowed to choose their own way of working. But the current Agile scaling frameworks are simplistic solutions, promising great results after a few days of certification training.

Helen offered one benefit of Agile scaling frameworks: they help individuals structure their work. “The thing about frameworks is that they help you understand what kinds of things need to be done, and it gives you vocabulary and context that allow people to work.” But she pointed out that there is a danger of trying to solve all problems with the same cookie cutter.

Nils didn’t like the extra complexity that results from using the scaling frameworks. “You will see the frameworks for long time, because many people are making a lot of money out of them. But some people will raise questions. The frameworks are making things more complicated and more complex.” We need to make companies and projects less complex in the future to handle external complexity. Instead of simple jobs in complex organizations we need complex jobs in simple organizations. Nils thought that how to “scale up” Agile development is the wrong question: “People are asking how to scale, while research shows that companies are making things more and more complex, so the question for the future may be how to scale things down.”

Deepti added that “it’s about systems thinking.” She said that it can take generations to make a shift in how we think. She also explained that companies are reluctant to build their own framework. It is hard to spend the time to explore, unlearn, and relearn. “No one has time or energy for that. So they will go for the framework from the external expert or consultant.”

Philippe thinks that the frameworks will fade away. He recalled the object oriented wars in the 1990s, which was a competition between several major object oriented design methods that each had its own standard diagrams. (Philippe paused and added: “I still miss the little clouds of the Booch notation.”) But although today’s generation of software developers uses popular object oriented languages like Java and Python, they don’t use the diagrams. The same thing may happen to Agile scaling frameworks. “I think at some point in time, after some battles, some conferences, and a competition to be the first one to be an ISO standard, they will disappear, perhaps about 15 years from now.”

Other questioners explored the future opportunities for collaboration between companies and universities. “This conference has been a meeting place between practitioners and researchers. If you are a company seeking advice on development methods, you get most of the advice from practitioners or consulting companies. In the next 20 years, do you see anything researchers can do to get involved more involved in practice?”

Philippe explained that improving access to research publications can be a big help. He noted that the XP conference has started publishing its proceedings as Open Access. “Most independent developers, like my son here in Montreal, don’t have access to IEEE Xplore and ACM Digital Library.” (An individual subscription to ACM Digital Library or IEEE Xplore is relatively expensive.) Philippe also thought that consortia – groups of software companies who team up with academia – are a good forum to increase the interactions between researchers and practitioners. He praised the Scandinavian governments who have provided funding for long-term consortia.

Ken addressed this question from the practitioner viewpoint: “We practitioners have a responsibility to innovate and disseminate the results of our industry innovation through various channels.” But he challenged university researchers to do a better job of collaboration with industry: “There is an opportunity to treat practitioners not as research subjects but as research partners – to engage more in a partnership model. Co-author papers, co-lead studies – Agile research has a lot of promise for industry-academic collaboration.”

Nils agreed that collaborative work is good, but we may need to change the way we publish research work to make it more useful to practitioners. “Maybe we need to change the way we communicate our results – in a way that make more sense for practitioners, so we can have a better dialog between researchers and practitioners. Unfortunately, researchers just talk to researchers and practitioners to practitioners.” Philippe liked the idea and added his own suggestions: “Maybe we need also other kinds of publications than the relatively rigid style that we use in Journal of Systems and Software and Springer-Verlag journals. The industry people would like to read what we write, but maybe we need to make things a little bit more approachable. We could have the rigorous things, with all the evidence, research questions, and threat to validity, and then we write another simplified version with a little more guidance for the practitioners to work with and give us feedback.”

3 Summary

The panel members’ crystal balls might be a bit cloudy, but they are mostly optimistic about the future of Agile. Agile will be around in some form 20 years from now, although it may have a different name, but software development organizations will still need strong team collaboration and the ability to respond to change. It is unlikely that AI and Machine Learning will replace programmers, but new AI-based programming tools would be welcome. The demand for Agile scaling frameworks will grow for a while, but they will probably fade away over time. Researchers and practitioners will still need to communicate and share ideas: experimental results, empirical studies, new ideas for Agile practices, and new ways of teaching and learning about the Agile mindset.