Keywords

1 Introduction

Design science research (DSR) adopts a pragmatic research paradigm to develop artefacts that are innovative and solve real-world problems [12]. DSR is relevant for information systems (IS) research because it directly addresses two of the discipline’s key aspects, namely the central role of the IS artefact in IS research and the perceived lack of professional relevance of IS research [9]. The notion of ‘design as research’ in the IS domain is relatively new. The adoption of DSR in IS research is mainly due to [10], wherein an IS research framework (ISRF) is provided that emphasises the rigour and relevance of the research. At the same time, [23] introduced a process model for DSR with awareness, suggestion, development, evaluation and conclusion as subsequent phases. This model is discussed in more detail in Sect. 2. In IS, research often includes the construction of some kind of artefact. Exactly what such an artefact entails is often also the topic of rigorous debate [19, 22]. Although [7] provides a framework for reporting in a research project, it do not discuss the process model of DSR in the same way as [23]. Independently of the research approach followed for a research project, students find the process of structuring the research report in such a manner that it forms a valid argument to be a challenge. This is often especially true in postgraduate studies that include the construction of an artefact in the research project. In such cases, students need to develop a document structure that supports the research contribution, and which includes an artefact comprising more than one component. The purpose of our paper is to consider the DSR process model as in [23], and propose possible document structures to support the research contribution in the research report. Using an analysis of several postgraduate research reports that successfully adopted DSR, this paper develops a mapping between the proposed structure of a research report and the DSR process model of [23]. The mapping is presented in four scenarios and validated with two examples of completed research reports. The mapping and scenarios were subsequently validated in a workshop with doctoral degree candidates. The feedback results indicated that the mapping was useful for all DSR students who needed to structure their research reports.

2 Background: DSR

DSR is primarily concerned with research on design as science [1, 6, 7, 13,14,15, 23]. The intent of DSR is to create an artefact through a balanced process that combines the highest standards of rigour with a high level of relevance. One of the measures of DSR is whether the research resulted in a relevant artefact, but also whether the process was rigorous [8, 24]. Figure 1 depicts the ISRF of [8, 10]. Using this framework, DSR is described as research building and evaluating computing artefacts designed to meet identified needs [8]. The goal of the artefact is the fulfilment of a specific need or utility. The description of the needs would provide the requirements for the artefact. In the building of the artefact, knowledge from the applicable knowledge base is used. During evaluation, the artefact is measured against the needs to evaluate its utility [22]. One of the central discussions in DSR is what is recognised as an ‘artefact’ in the DSR paradigm [19, 22]. One of the reasons is that the artefact is not always tangible, but more often intangible (such as a model, software, a framework or architecture). In DSR, it is accepted that the artefact embodies or is part of the design theory [21]. The following three discussions on DSR artefacts appear in [18]:

Fig. 1.
figure 1

Information systems research framework according to [10]

Full size image

  • Constructs, models and methods are DSR ‘artefacts’ in [25], whereas constructs, models, methods, instantiation and better theories are DSR ‘outputs’ in [23].

  • DSR ‘outputs’ were analyzed in [21], identified as constructs, models, methods and instantiations, as well as social innovations or new properties of technical, social or informational resources. Accordingly, an ‘artefact’ is any designed object with an embedded solution to an understood research problem.

  • In [19] we can find an exhaustive list of acceptable DSR artefacts that included software, algorithms, methods, models, frameworks and architecture, grouped into eight types: system design, method, language or notation, algorithm, guideline, requirements, pattern and metric.

For the purpose of this paper, it is accepted that the IS DSR artefact is anything that is delivered by a rigorous research and development process and that can be shown to fulfil an identified need.

Fig. 2.
figure 2

Design science research process model (DSR cycle) according to [23]

Full size image

2.1 DSR Methodology

One of the most-cited methods accepted by DSResearchers is the design cycle or process model of [23], (see Fig. 2). The method was derived from other DSR advocates such as [8, 9, 16]. In [23] the following five phases for the execution of a typical DSR project are recommended:

  1. 1.

    Awareness of the problem: The awareness could be generated from practical experience or from related disciplines. The output from this phase is a proposal.

  2. 2.

    Suggestion: The suggestion is closely related to the awareness of the problem (as indicated by the dotted line). The suggestion is often included as a tentative design in the complete proposal as output. However, an approach to develop a suggestion might be included in the proposal if a possible solution is not immediately evident.

  3. 3.

    Development: The tentative design is implemented during this phase and the technique for implementation will differ depending on the artefact.

  4. 4.

    Evaluation: When the artefact has been developed, the evaluation of the artefact is mandatory, usually according to requirements and criteria specified during the suggestion phase (as part of the proposal). The result of the evaluation should be carefully noted and explained. This phase may result in the refinement of an awareness, a suggestion or a development, especially if the result of the evaluation is not satisfactory.

  5. 5.

    Conclusion: This is the final phase when the research results and contribution are identified. This not only includes the artefact, but all additional knowledge with regard to the process, construction and evaluation that were acquired. The output of this phase is an acceptable research contribution.

The notion of ‘iteration’, as indicated by the arrows on the left in Fig. 2, is embedded in the DSR method. It is possible to branch back to awareness during the execution of development, evaluation and conclusion. Several cycles of the abovementioned notions are often executed during the construction of a DSR artefact. Circumscription is due to the discovery of constraint knowledge about the theories gained through the detection and analysis of contradictions, but in practice these cycles also occur because development, evaluation and conclusion in DSR often expose new problems that could be entered into the DSR cycle at the awareness stage [23].

The DSR process model of [23] summarises the phases that are necessary to execute a DSR project. Nowadays, the approach is often adopted by researchers and postgraduate students in IS as an acknowledged, repeatable process for the construction of a useful artefact and research contribution. Given the adoption of this approach, it is possible to argue that the specific process model for DSR plays a valuable role in ensuring that computing research is more rigorous and repeatable, but also relevant and useful. For the remainder of this paper, reference will be made to the DSR process model of [23] (Fig. 2) simply as ‘the DSR process model’.

Fig. 3.
figure 3

Structure of a research report in [4]

Full size image

3 Typical Structure of a Research Report

Multiple considerations, such as subject chosen and research approach, drive the planning around the structure of a postgraduate study or research report [20]. In theoretical and textual studies the introduction is normally followed by chapters that reflect the analysis of secondary literature and the conclusion. In contrast, the structure of an empirical study is largely dictated by the methods utilised.

Generally, for most studies, the student will have to include a chapter that explains the research design, another on the analysis and presentation of findings, as well as a discussion and recommendations [17, 26]. Therefore, the structure of a research report (dissertation or thesis) followed by students in most disciplines usually consists of an introduction, background, body (core) and conclusion [4, 11]. As mentioned in [4] (see Fig. 3), all sections in a research report are connected. The report’s conclusion connects to the goal, and the background acts as input to the core or body of the work.

Fig. 4.
figure 4

Structure of a research paper or research report in [11]

Full size image

In [11] it was proposed that the text consists of an introduction, literature review, method, body and conclusion, (see Fig. 4). The purpose of the introduction is to give an overview of the report, and includes sections on the research questions, objectives, a problem statement and a brief chapter overview. The second section in this document structure is the literature review, where the theory base for the study is introduced: from a high-level or broad perspective, as well as an in-depth discussion of relevant topics. This section might consist of more than one chapter. The method chapter, which provides an overview of the research design, including the research instruments for data analysis and data collection, follows the literature review. The body section of the research report usually consists of the evidence of how the student either proved his or her hypothesis if working deductively, or how he or she derived at a contribution if working inductively. The body might be more than one chapter, depending on the format in which the student presents the work. Lastly, the conclusion of the work consists of a summary of the findings and the contribution. The questions that guide this structure, although not explicitly mentioned, are the following: what do I want to know? (Sect. 1); what do I know? (Sect. 2); what will I do? (Sect. 3); what did I find when I executed my plan? (Sect. 4); what is the solution or contribution? (Sect. 5).

Despite the proposed structure of a research paper or research report discussed above, students often find it difficult to structure a study that adopts a DSR method. It is, for instance, not clear how to integrate the DSR process model into the proposed document structure. Awareness of a problem could typically fit in Sect. 1 of the document structure, but where would the suggestion then fit in? The literature review should support the problem and therefore the awareness. So does the literature review follow the awareness or does the awareness description follow the literature review? How would the development phase be included given the proposed document structure, especially if more than one development cycle was included in artefact development? Including different theory sections for different development cycles to accommodate the rigour requirement of [10] is even more confusing.

Table 1. Publication scheme for DSR studies according to [7]
Full size table

4 Research Report Structures for DSR Process Models

Since the acceptance of DSR as an acceptable research approach for IS research, several discussions about the structure of a DSR research project have commenced. For example, [7] proposed the publication scheme summarised in Table 1 for DSR studies. Due to the wide-spread adoption of the process model of [23], the purpose of our paper is to extend the publication scheme by integrating the DSR process model and mapping it to the proposed research report structure (as discussed in the previous section). Postgraduate research reports (theses and dissertations) of students that successfully adopted the DSR process model were analysed and four different scenarios were identified. In the first scenario, the student only had one DSR design cycle, and a single artefact was constructed during the design. In the second, third and fourth scenarios, the students developed composite artefacts that consisted of more than one component. The DSR process therefore included more than one cycle.

4.1 Scenario 1: One Cycle of Design

Often in students’ curricula, especially at master’s degree level, a student might be involved in the design of a simple artefact with a single function, as opposed to a composite artefact that consists of more than one component. Examples of such artefacts include a system with a single defined function, such as a mobile application. It is also possible to have one single cycle when the artefact consists of more than one component, but the components are predeveloped and the designer is only involved in the assembly of existing components in the DSR process. Presenting research of this nature can then be presented using a single mapping from the DSR process model, as illustrated in Fig. 5. In Fig. 5 the mapping between the DSR process model and the research report structure is indicated with numbered arrows (1–8).

  • Mapping 1: Introduction and Awareness of the Problem. In Sect. 1 of a research report (the introduction), the student should already introduce the problem. This this correlates with the awareness phase of the DSR process model.

  • Mapping 2: Introduction and Suggestion. In Sect. 1 of the research report (the introduction), the student already provides an indication of the type of solution (artefact) for the problem (as discussed in Mapping 1).

  • Mapping 3: Literature Review and Awareness of the Problem. In the literature review, the student provides proof of the identified problem by, for instance, discussing a problem that is experienced in practice (relevance) and/or providing proof that the problem has not previously been resolved in literature. Usually, this section should include a comprehensive discussion of the existing related literature and should indicate the lack of a solution in literature.

  • Mapping 4: Literature Review and Suggestion. The student could already suggest an artefact that could provide a solution for the problem at the end of the literature review. This artefact could be a construct, model, method, instantiation or better theory. At this stage, the student might also introduce any theories that will be used if applicable (to address the rigour requirement of [10]). It is also possible to only provide the suggestion in the body of the research report.

  • Mapping 5: Method and Development. In the method section of the research report, the student includes a description of the adopted and adapted DSR process model, which includes the planning of the development of the proposed artefact. Depending on the type of artefact, the student includes how the artefact will be constructed. For an experimental study, the plan may include how the artefact will be built and tested. For a qualitative study, the student may include the questionnaires and analysis methods to be used to collect data to build the artefact, such as in the case of a construct, method or conceptual model. The study may also outline how the artefact will be tested. Depending on the scope of the study, this may include a proof of concept and not a full test.

  • Mapping 6: Body and Development. The student includes all the data relevant to the building of the artefact in the body of the report. For an experimental study, it might include data on the experiment conducted during the development. For the development of a software artefact, the body might include the description of the system itself and the different screens and functionality. For a qualitative study, the results might be the data that was collected, as well as the analysis of the data. The body section includes the results of the study, including the artefact itself. Generally, in a more inductive study, the artefact is presented at the end, while a more deductive study proposes the artefact in the beginning of the body section and then follows a more descriptive process of the development of the artefact.

  • Mapping 7: Body and Evaluation. The body section, usually consisting of several chapters, also includes the results of any evaluation that was done. In a more experimental study, such as the development of an application, this section includes the testing results of the application. For an inductive qualitative study (where the main component of the study consists of constructing the artefact), the evaluation may include a proof of concept or validation using data collected from focus groups or interviews, depending on the scope of the study.

  • Mapping 8: Conclusion. The last section of the research report summarises the study and research contribution, including how the artefact as research contribution has value from a rigour and relevance perspective [10].

Fig. 5.
figure 5

Mapping for a single design science research process cycle

Full size image
Fig. 6.
figure 6

Design science research process model with many cycles

Full size image
Fig. 7.
figure 7

Design science research process model in [2], with the student’s additional comment: “This research design consists of five steps: awareness of the problem, suggestion, development, evaluation and conclusion. For the awareness of the problem, a literature review was conducted, while theory analysis, theoretical study and artefact building were conducted for the suggestion and development. Verification using a proof of concept was conducted for the evaluation and conclusion”

Full size image

4.2 Scenario 2: DSR Process Model with Many Cycles of Design

In doctoral degree studies it is often the case that a composite artefact that consists of more than one component is constructed. In such a scenario, the same general format that was discussed in the previous section could be followed, with extensions in the method (Mapping 5, which influences the development) and the body (Mapping 6, which influences the development): see Fig. 6. As indicated in Fig. 6, the research report will still include a main DSR cycle as the main guiding structure, such as the one discussed in the first scenario. However, the method section might include a description of several subcycles that are then included in the body of the research report. The awareness of the first subcycle will form part of the development of the main DSR process when the student realises that the development of the main artefact consists of the development of subcomponents. For the components, there will then be separate cycles that follow the DSR process model, as indicated by the arrows in Fig. 2. These subresearch cycles could also extend into further cycles. Each artefact component may be evaluated separately, or the testing and evaluation could be included in the evaluation of the complete artefact in the main research cycle.

4.3 Scenario 3: Problem Establishment as Part of the Research Process

A variation of Scenario 2 occurs when the student cannot provide sufficient motivation for the research problem from the literature review and needs to provide additional evidence by, for instance, conducting a pre-study as part of his or her research study. In this case, the same structure as in Fig. 5 is proposed, however the awareness and suggestion phases of the DSR process model are included in the body of the research report, after the literature review.

4.4 Scenario 4: Change in the Research Report Structure

As a final scenario for a DSR research report, it is possible to include the method section before the literature review. This structure could be problematic and students are cautioned against its use. This structure might be confusing to the reader (examiner), as he or she will be confronted with the research design in the method section before the awareness of the problem (problem description) and the suggestion are presented.

5 Examples of the Use of the DSR Process Model

As discussed in the previous section, four possible scenarios were identified to map a research report structure to a study that adopted the DSR process model. In this section, some examples are discussed to illustrate the proposed mapping.

Fig. 8.
figure 8

Design science research process model as a strategy in [5]

Full size image

5.1 Example of Scenario 1: One Cycle of Design

An example of Scenario 1 can be found in [2] wherein a decision-making tool was develped to guide users when selecting a technology that integrates ontologies with relational databases. In this case the student did not map his chapters to the DSR process model of [23] diagrammatically, but his verbal description in [2] (chpt. 1) gives evidence of a mapping according to Scenario 1: see the caption text (in italics) to Fig. 7). In [2] (chpt. 1), the student already introduced the problem (awareness) and proposed a tool (suggestion), but only linked the DSR model to his document structure at the end of [2] (chpt. 1) after discussing the DSR process model.

Fig. 9.
figure 9

PRIF developed in [3]

Full size image

5.2 Example of Scenario 2: DSR Process Model with Many Cycles of Design

A conceptual framework for financial reporting was developed and presented in [5]. The chapter map of that doctoral thesis is depicted in Fig. 8. The development of the conceptual framework as an artefact consisted of one main research cycle and four subcycles. Figure 8 illustrates that the main cycle is documented in [5] (chpt. 1–3). The development phase branches off into four subcycles, which are reported on individually in the body [5] (chpt. 4–7). Cycles 1 and 4 include evaluation activities, however cycles 2-3 do not. These components are tested in the main research cycle, which is described in [5] (chpt. 8). The conclusion is in [5] (chpt. 9): it summarises the study’s contribution. The method used in [5] aligns with the discussion of Scenario 2 (see above) where a main DSR cycle is proposed with a number of subcycles as part of the development phase.

5.3 Example of Scenario 3: Establishment of the Problem as Part of the Research Process

As mentioned, it is possible to also include the awareness and suggestion in the body of the research report. In the another example [3] we can also find more than one cycle for the development of a process reuse identification framework (PRIF), as illustrated in Fig. 9. In contrast to [5], a survey was conducted in [3] to establish the research problem clearly. This problem is then discussed in the body of [3]. The problem is introduced in the introduction and literature review sections, but since the evidence was not conclusive, the problem was confirmed to align with the relevance requirement of [10] by providing the survey results after the method section in the body of the report [3] (chpt. 5). The subcycles are all reported on individually in [3] (chpt. 7–9), and the evaluation and conclusion are provided in [3] (chpt. 10). The structure in [3] is an example of Scenario 3, which provides additional evidence for the problem identification.

6 Discussion

In the past few years, we encountered several postgraduate students in computing (both under our supervision, as well as research reports examined externally) that adopted DSR, more specifically, the DSR process model of [23] as their preferred research approach. During a PhD workshop conducted with postgraduate second- and third-year DSR PhD candidates who were involved in the structuring of their research reports, the different scenarios presented in Sect. 5 were proposed. The discussions included the presentation of examples from completed research reports that adopted DSR. All postgraduate students (\(n = 7\)) found the scenario discussions, as well as the examples of other students’ work and DSR structures of value. The discussion on the DSR process model of [23] and how it relates to the students’ own cycles of design and development of the artefact resulted in enthusiastic interaction. Discussions during the session included the fact that each student’s work is unique and that it is necessary for each student to construct his or her own research report workflow. Students specifically emphasized the usefulness of the mapping and the scenarios, as they may be applied to both qualitative and quantitative research, and already accommodate some of the unique nuances of their research. In addition, students emphasised particular advantages of their applications of the mapping and scenarios:

  • The first advantage pertained to the approach to their research when executing multiple design cycles. As the challenge for a student often lies in the ‘how to’, students at the workshop reflected that the scenario of multiple cycles provided clear guidelines for their approach and illustrated how one cycle informed the next in the context of what they know to be (through theory) their required research output.

  • The second advantage students identified related to the write-up of their research reports. Students who were in the process of writing up their dissertations or theses, highlighted the fact that a major challenge they experienced was to write up their research in a way that an examiner or external stakeholder may follow the presented argument. In this instance, the mapping and scenarios provided a solid recommendation on structuring a DSR study in order to produce an organised research report that correctly reflected the results and outcomes of their research.

  • Lastly, students reflected that research papers, such as journal papers and conference proceedings, in the DSR literature accommodated and referred to multiple stances that were aligned to many research methodologies and approaches where they were applied—an incidence that makes it more difficult for the students to find their own way. The scenarios and mapping provided a perfect vantage point—like a one-stop shop—from where the students could make sense of the DSR body of knowledge and particular application in their research study before embarking on a study of the broader DSR domain. This pointed to an inside-out consideration, rather than an outside-in approach, which, with such a focused starting point, provided clear direction and saved time when dealing with DSR research reports.

7 Conclusion

We acknowledge that there will be slight variations in the structure, depending on the unique research problems and approaches. However, in general, it is possible for studies that adopt DSR to map the requirements of a research report to the DSR process model. Given the feedback that was discussed above, the scenarios presented in this paper should provide valuable guidance for students who need to structure a DSR research report.