Keywords

1 Stories Relating Research and Practitioner Experience

Throughout the 75-year history of software research and development, researchers and practitioners have advanced the field by sharing key learnings and best practices. Sharing is achieved through written documents: books and papers that document academic research, industrial experience, successes, failures, and retrospectives on the application of new methods and tools. Recently, sharing has also included internet-accessible audio and video recordings such as podcasts and YouTube videos. In both cases (oral and written), the software community’s goal is to contribute knowledge that others may leverage and extend.

The panel discussed three formats for sharing best practices: experience reports, research papers, and patterns. Each of the three formats is characterized by its own stylistic guidelines. Experience reports are personal stories written in the first person to document a personal journey. Research papers are accounts of researchers applying the scientific process to a specific technical investigation. Patterns are descriptions of problem and solution pairs within a general framework. Practitioners and researchers select an appropriate style to share their ideas and results with the software community.

2 Research Papers: Validating Research

Each academic field and many technical conferences have “guidelines” for research papers [1]. Casper Lassenius (Aalto University) related an informal set of guidelines for effective research papers. Casper (XP2021 academic track co-chair) shared his “COVID-D” research reporting model:

  • Context and Contribution

  • Objectivity

  • Validation

  • Interesting

  • Depth

  • Delightful

Each element of the model is an essential piece of a successful research paper’s content and structure. The Context and Contribution of a research paper explain the essential elements of the research work; Objectivity requires that both successes and failures are reported; Validation references data to support the paper’s claims and conclusions; a paper should be Interesting to both researchers and practitioners; Depth means that the paper contains enough detail for understanding and insight; Delightful means that the authors use an engaging style to attract and sustain readership.

In Casper’s opinion, the Context of a research paper should include both academic context and practitioner relevance. Authors hope to interest readers in the acceptance (and possible adoption) of their paper’s results, so the “academic context” is a matter of illustrating how results complement work reported by other researchers. Industry practitioners are motivated to read papers based on contextual research applicability.

Casper also noted that originality is an essential ingredient of a research paper. Authors must explain, identify, and distinguish (as unique) their contributions to the field. To validate research claims, the authors should include supporting data, from experiments – based on either new or community results. It was also observed that research papers generally focus on a very narrow set of questions. The requirements of validation and depth make it almost impossible for a paper to answer complex questions within journal or conference page limit constraints.

When we write research papers in the field of software engineering, we hope that our papers are interesting enough that others will build on our work. It is a continuing challenge to find an audience for research. The panel moderator, Steve Fraser (Innoxec), reflected on this issue. He recalled a complaint from Fred Brooks at an OOPSLA 2007 panel session on software engineering [2]. Brooks was worried that we don’t understand enough about others’ successes and failures. His point: “I know of no other field [software] where people do less study of other people’s work.”

3 Experience Reports: Personal Stories

Rebecca Wirfs-Brock (Wirfs-Brock Associates) and Ken Power (Independent Software Engineer) explained that experience reports are not as objective as a research paper, because an author relates their personal story – rather than reporting validated experimental results. Experience reports may also share emotional perspectives and feelings.

Ken believed that practitioners should write more experience reports. Ken noted, however, that all of us are challenged “finding the time to write down our stories and get them clear in our heads.” While the community benefits from experience reports, authors are often inexperienced writers, and they depend on conference assigned “shepherds” to relate and document their story. Shepherds are the authors’ guides though the storytelling challenges at conferences such as ACM SPLASH, XP, and Agile [3][4]).

A shepherd or writing coach is an essential catalyst as Ken explained, “As an author, [the shepherd is] somebody in your corner who wants you to succeed, who wants you to tell your story in the best possible way – so you’re not staring at that blank page on your own.”

4 Patterns: Distilling Many Stories

Mary Lynn Manns (Fearless Change) and Ademar Aguiar (University of Porto) explained the writing process for patterns. Patterns are short one- to two-page documents focused on practical solutions to a problem. A well-written pattern takes time to develop, and the writing task can be quite difficult. Mary Lynn observed, “A good patterns author will take years to get one good pattern written on a particular topic.” A draft pattern needs to be verified, and a solid process to check a pattern is to interview other experts and collect more stories. As a draft pattern evolves, it becomes more general and addresses a wider range of contexts.

Patterns are best if they are short and simple. Ademar emphasized two more important properties in telling good stories – Accessibility and Reusability – and in good patterns, both beginners and experts will have access to knowledge that is reusable.

A pattern is not based on novelty. Ademar quoted Brian Foote: “Patterns are a blatant disregard of originality.” A pattern author does not create new knowledge, rather their goal is to record existing knowledge in a useful and reusable way. The pattern writer’s role is to extract implicit knowledge from experts –and convert the knowledge into a short, useful document.

5 Pattern Evolution

The panelists discussed the challenges of updates to previously published narratives. Once a collection of patterns is published, these patterns become difficult to revise, even when a patterns update is needed to add new stories, new context, or new and improved approaches.

Everyone agreed that “out of date research papers” are a lesser problem, not the major crisis that out of date patterns could be. Casper noted that research papers are much lazier about updating their theories than pattern writers – he explained that “I think the half-life of a pattern is probably shorter than the half-life of an academic theory.” Rebecca observed that research papers contain their data and references, but patterns don’t. Rebecca explained her view of the difference: “When I write a research paper, it can stand on its own, it cites other things. When I write patterns, they go out in the wild. They are used by other people, they are misinterpreted by people, they are adapted by people.” So you never submit a “recall” of a research paper, you just write a new one. But a pattern needs to evolve to incorporate better techniques and better understanding.

Mary Lynn believed that good pattern writers must try to be humble. A pattern is never “final” and should not be “etched in stone.” A pattern must be open to revision by its author or by others.

Rebecca agreed, explaining that many patterns are frequently revised, and that patterns “… cannot stand without a curation and a community. Curation is something that takes time, energy, and passion.”

Mary Lynn added that the patterns community has the concept of a “proto-pattern,” a trial pattern that needs more validation. She explained that when you call your idea a proto-pattern, you indicate “I think this is true, and I’d like to open it for discussion.”

6 The Stories We Tell: The Agile Manifesto

The panel moderator, Steve Fraser (Innoxec), prompted the panelists to discuss the Agile Manifesto [5]: “Where does the Agile Manifesto fit into the stories we tell?” His question triggered several interesting comments, after he reminded panelists that Steve McConnell (Construx) was scheduled for a keynote talk the following day to advocate for revision and modernization of the Agile Manifesto.

Rebecca responded, “The authors [of the Manifesto] aren’t going to change it.” She added that the Manifesto captured the spirit of what agile was trying to do 20 years ago, but it should not be considered a set of unchanging principles for today. She admired the bravery of the pioneers of Agile because they decided to tell a bold story as a way to gain attention. She said, “How many times do you write a manifesto when you want to do something in research?” She compared the Manifesto authors to members of the early XP community who were gutsy enough to rally around the controversial name “Extreme Programming” coined by Kent Beck to describe a lightweight small-team development process incorporating many practices that we now consider “Agile.”

Ken shared his opinion on the Agile Manifesto: “It is a wonderful historical artifact,” and he suggested that many are unaware of the Manifesto and its values. Ken thought the most important part of the Manifesto’s “story” is its first line: “We are uncovering new and better ways of doing things.”

Ademar weighed in about the changing context of the agile world. “Maybe the generality [of the Manifesto] is too general today, we may need to be more specific.” Ademar related this to the problem of writing good patterns – patterns that are too generic fail to give useful advice, but patterns that are too specific may age quickly as developers move to new systems and new contexts. To this point, Steve McConnell’s XP 2021 keynote gave specific recommendations for updating the Manifesto. Steve indicated several Manifesto principles that should be updated in the light of 20 years of software engineering research and experience.

Mary Lynn thought that “getting your ideas out” for public discussion is the most important lesson from the Manifesto: “The important thing is that people just write.” Mary Lynn gives the Manifesto authors credit for saying what they thought. We should do the same – put out our opinions and let people discuss them.

Casper opined that he was ready to move on from the Agile Manifesto. “I’m interested in what works, what makes the software industry better. We shouldn’t start focusing on this like it’s a kind of sacred text. It should not be a test of your faith.”

7 The Future of Storytelling

Ken believed that social media tools are useful for sharing stories and information. He cited Discord, Twitch, and Twitter as tools to disseminate informal communications. Ken postulated that video, augmented reality, and virtual reality tools may become useful. Ken related how some agile teams record short videos to explain certain architecture decisions or to share other key information with teammates.

Casper observed that video is somewhat imperfect and inefficient. Developers can share information on StackOverflow which could be complemented by video chat. Casper thought that similar sites may emerge for patterns or organizational innovations.

Mary Lynn emphasized that short communications will be effective in the future. “We say that their [developer’s] attention span is shorter. I don’t know if I believe that. I just think we are being pulled in too many directions.” She advocated short videos, short blog entries, and other kinds of short presentations will be more valuable than full research papers and experience reports of 10 pages or more. Short presentations are more likely to be read and discussed, so they will have a greater impact.

Rebecca believed that all forms of storytelling will be relevant in the future. “I think that no form of communication is going to go away. We are just going to add to our storytelling bag of tricks. The challenge we have is picking the right mix.”

8 Other Insights from the Panel

Ademar shared the FAIR principles [6] from the Research Communications and e-Scholarship community: knowledge should be Findable, Accessible, Interoperable, and Reusable. E-Scholarship and patterns have communities that are advocates for efficient dissemination of information. Ademar believed that the FAIR principles apply to the creation of good patterns.

Casper asked some probing questions about the pattern writing process. He was curious how a pattern writer could use one or two stories to “abstract the context” of the problem well enough to write a good pattern. Casper asked, “How deep is your understanding?… to me, the patterns look kind of like witchcraft.”

Mary Lynn disagreed, explaining that if you could write a pattern from one story, that would be witchcraft. She explained that for years, the patterns community followed the “Rule of Three” – whatever you discovered might not really be a pattern until you hear it three times. She recommended that a pattern author record the sources of the stories that were incorporated into a pattern, because maintaining a pattern is easier if the author can say “this is where I heard this.”

The number of references to outside papers and experts should be much lower in experience reports than for research papers, according to Rebecca. “In a story about an agile experience, some people feel that they need to cite all the experts who have written the books.” For an experience report, the emphasis should be on what the authors have experienced and learned. “Be very direct about successes or failures, ‘aha moments’ you might have had… that’s what makes an interesting story.”

9 Summary

All storytelling formats are useful for sharing experiences and transferring knowledge and technology in the software community.

Ademar recommended trying to write patterns: “It’s not harder or easier than a research paper, it’s just different.”

Ken was quite willing to expand storytelling beyond the three main forms: “I find the practice of writing – whether it’s experience reports, research papers, tweets, blog posts, patterns, or anything else – to be a very useful exercise in coordinating my own thoughts, and in helping other people to coordinate and articulate their thoughts.”

Casper found that sharing stories, whether in written form or face-to-face over a beverage, can make a positive contribution, even if the stories talk about our problems: “So people don’t need to do the same mistakes, and at least we can laugh together.”

Mary Lynn and Rebecca both advocated writing as a good “thinking tool,” even for people who are ashamed of their own writing style. “You don’t have to classify yourself as a writer, just go out there and write,” Mary Lynn exhorted. Rebecca was also encouraging: “Not all of us are doing revolutionary things. But you can still seize the moment.”

Effective writing will continue to be a means for software professionals to document and share results to continue and advance innovation and technology.