Keywords

1 Introduction

In response to a growing demand from industry for individuals with business process capability, universities continue to invest in the design and delivery of Business Process Management (BPM) curricula [43]. Much effort has been devoted by the BPM academic community to innovating course curricula, textbooks and teaching materials. However, BPM remains a particularly challenging subject as students learn skills across the lifecycle of process identification, discovery, analysis, redesign, execution, monitoring and control [18].

Unsurprisingly, BPM educators have devoted much attention to the question of how such curricula can best be delivered. Efforts include presenting content and syllabi for BPM courses [4, 5, 43] and mapping course content to the BPM capabilities required in practice [16]. Importantly, past work has demonstrated that BPM education requires an emphasis on both theory and practical application [33, 42]. Authors discuss the use of case studies and tools [37], and the importance of ensuring students develop practical skills and ability to apply BPM knowledge to complex problems and modeling scenarios [39].

One stream of research has brought attention to the importance of using professional BPM tools in practical components of course delivery, e.g. ARIS [4]; MS Visio, ARIS and YAWL [43]; BPMS platforms such as Adonis, Tibco and Bizagi [5]; ARIS and WebSphere [16] and, in some cases, ERP software [22]. Industry software tools have clearly come to play an important role in the teaching of BPM concepts. Hands-on exposure to industry systems develops skills, helps student understand how software actually works and improves their business process knowledge [41]. They are clearly important to building skills, but little empirical research exists to identify best practice [5].

Another stream of research has focused on how educational games can be used to provide students an environment in which to practice BPM. IBM’s Innov8Footnote 1 has been argued as particularly useful given its professional orientation (e.g. [7, 27, 30, 50]). Students benefit from experiencing hands-on, role-play within the game environment [51]. Games might also stimulate interaction, teamwork and promote more social and communicative learning [51]. Games are argued to be useful because of their potential to illustrate the relevance of what students have been learning as they navigate the simulation to apply their knowledge in realistic scenarios.

Taken together, these two streams of research provide evidence that the incorporation of hands-on training in the use of industry software as well as the incorporation of digital games can be used to help students learn BPM. However, past work does not explore the relative efficacy of these tools for student outcomes. This presents a dilemma for BPM educators who must decide with the limited resources available, which types of intervention will yield the best returns for student outcomes.

In this study we thus sought to determine whether exposing students to additional workplace tools to enhance their knowledge of BPM practice would be more, or less, effective at influencing outcomes than exposing students to a digital BPM game. We focus on three outcomes, namely students’ perceived competence in performing BPM tasks, intrinsic motivation toward performing BPM and their occupational self-efficacy i.e. belief in their ability to succeed in a BPM career. We considered these outcomes appropriate because BPM curricula should not only develop student skills but also their confidence and motivation to pursue careers in BPM.

We have previously explored the effect of the inclusion of digital game-based learning on students’ motivation and perceived competence in an earlier offering of our BPM course [21]. This study extends that work to examine within a new cohort of students the relative efficacy of games versus industry tools in the teaching and learning of BPM.

2 Study Design

2.1 Context of the Study

Our study context was a core (required) undergraduate class in BPM for information systems majors. The students were enrolled in a seven-week BPM course in their third year of study. The course consisted on 28 lecture hours, 7 workshop hours, and 14 hands-on computer laboratory hours. The course uses project-based learning where students work on an assigned project case in small teams. The project reinforces concepts by requiring students to apply knowledge to assigned process modelling, analysis, redesign and automation tasks in the case study. In practical laboratory sessions, students use Bizagi to learn to model processes in BPMN and then to enact processes through Bizagi’s BPMS engine (Bizagi Suite).

2.2 Conceptual Background and Research Variables

Digital game-based learning (DGBL) can be defined as the use of computer games (or digital games) within an educational context to supplement learning and support a particular learning outcome [2, 49, 54]. Digital games have also been termed “serious games” [20], “simulation games” [29] and “digital learning games” [40].

Digital learning games can be effective in addressing knowledge and skill and motivational outcomes. They can be used to illustrate the relevance of what students have been learning, provide them an opportunity to apply their knowledge in realistic scenarios that mimic real world problems, practice skills in a “virtual” real life situation, experience hands-on role-play and immersion in the virtual world, stimulate interaction among students and promote more social and communicative learning. Moreover, they provide an environment where students can express their autonomy, make their own choices, try out new skills and feel a sense of mastery or competence [6, 12, 19, 28, 34, 38, 46, 48]. Digital games should allow users to experience an optimally challenging environment but to be effective must contain positive feedback that is free from judgmental evaluation [14]. However, the efficacy of DGBL is being questioned [10]. Several recent studies, reviews and meta-analyses of DGBL have shown no impact on motivation and learning achievement or it has been shown to perform worse than a traditional classroom setting [3, 17, 20, 24, 53, 54]. The impact of DGBL on outcomes thus remains an open question.

Industry software tools have come to play an important role in the teaching of BPM concepts. Emerging curricula are emphasizing practical application and use of modelling tools in course delivery (e.g. [33, 42]). The use of industry software platforms has been considered among the requirements for ‘state of the art’ in BPM education [5]. It is difficult to argue that a course in BPM is relevant if it does not ensure students develop practical skills in use of professional industry tools [37]. BPM educators are often making choices about which BPM tools to include, such as ARIS; MS Visio; YAWL; Adonis; Bizagi; WebSphere [4, 5, 16, 43].

The incorporation of tools helps students develop practical skills and ability to apply BPM knowledge to complex problems and modeling scenarios [39], improve their knowledge by understanding how software actually works [41], experience the modelling grammars and notation as implemented within marketing leading process tools [43], and obtain up-to-date skills highly valued by industry [42]. Undoubtedly the incorporation of professional tools into the curriculum can help students to assimilate concepts by connecting theory and practice. However, the question remains as to whether ‘more is better’ when it comes to the use of these tools in the classroom. Unfortunately, little empirical research exists to identify best practice [5]. More specifically, BPM educators are left wondering which specific tools are useful to include and how many tools should one incorporate and for which types of outcomes.

Consequently, we contribute by examining the relative effects of digital games versus exposure to additional industry tools. The study was grounded in social cognitive career theory [31] along with Deci and Ryan’s [15] self-determination theory. Together, they suggest an important set of constructs that represent higher-order outcomes that educators should look to achieve in a course on BPM.

Deci and Ryan [13,14,15] introduced their self-determination theory (SDT) and its underpinning of cognitive evaluation theory (CET) in an effort to explain and understand human motivation in any context. According to the theory humans actively seek to improve themselves by pursuing and engaging with challenges that allow them to realize their potential and capacity [15]. Moreover, the social environment that the individual is in will either support or diminish this process of self-realisation [15]. The theory views intrinsic motivation as the highest level of human motivation.

Intrinsic Motivation can be defined as an individual’s internal motivation towards performing a task. When someone is intrinsically motivated they are willing to devote extra effort to an activity because of the interest and enjoyment derived from its performance [15]. Students with higher levels of intrinsic motivation are more engaged in the learning process and put more effort into their academic activities to achieve higher levels of academic performance [23, 44]. According to SDT, intrinsic motivation is the most powerful form of motivation that an individual can feel towards an activity and should therefore affect behaviour in a far more powerful way than external rewards. Consequently, we considered that a student’s intrinsic motivation toward BPM as reflected in their expression of enjoyment and interest in BPM is an important outcome of interest when considering the efficacy of curriculum and classroom interventions.

According to SDT and CET, for an individual to be motivated, there are basic psychological needs that must be supported by the social environment. These needs can either be reinforced by the social environment to result in high levels of motivation or can be diminished by it. In an educational context, competence is considered the most influential of these psychological needs for developing intrinsic motivation. Prior work supports perceived competence as a predictor of intrinsic motivation [13, 15].

Perceived Competence can be defined as a person’s perception of how skilled they are at performing an activity or task. Previous studies have found that perceived competence felt towards a subject domain has a positive relationship with learning achievement in that particular subject domain [8, 25, 35]. When perceived competence is low, academic performance diminishes [36]. Therefore, BPM coursework should be designed to so that students perceive themselves as growing in competence and skill along the BPM lifecycle, and across the theoretical and practical components of BPM.

Social cognitive career theory (SCCT) purports that there are sociocognitive determinants of career and academic interests, where such interests subsequently promote career-related activity involvement and skill acquisition [31]. The core construct within SCCT is occupational self-efficacy, which is considered to mediate between occupationally relevant abilities and occupational interest. Drawing on the work of Bandura, SCCT defines self-efficacy as an individual’s judgment of their capabilities to successfully execute a course of action and is considered a central mechanism of personal agency determining one’s choices and effort expenditure [31]. According to SCCT, self-efficacy is prominent in the formation of career interest such that people form enduring interests in activities in which they view themselves as efficacious [31]. Moreover, individuals are likely to perceive greater rewards and anticipate greater satisfaction from pursuing those activities in which they consider themselves more able to succeed.

Occupational Specific Self-efficacy is considered to have effects on career choice [45] and is defined as an individual’s belief in their ability to succeed in a given career. Lent et al. [32] consider self-efficacy to influence choice of major directly and indirectly through effects on interest and outcome expectations. In the IT education context, past work has shown that feeling “not suited for IT type work” is among the reasons for students not wanting to pursue IT studies or an IT career [52]. Occupational self-efficacy has been shown to correlate with IT career intention and choice to major in IS [11] and is important to forming positive attitudes toward IT jobs [26]. This link between occupational self-efficacy and career interest and choice is likely to also hold in the more specific case of BPM. It is unlikely that students will pursue BPM as a career option or even learn more about the opportunities available to them for careers in BPM if they have low levels of BPM occupational self-efficacy.

2.3 Experimental Design

To achieve our research objective, our study followed a between group randomized experimental design [47]. This design is characterized by the comparison of multiple groups that are randomly assigned participants. Random assignment ensures groups are on average equal and that any effects observed in the study can be attributed to the interventions [47]. Data collection followed a pre-test post-test design, which allows for the comparison of groups before and after interventions have been implemented [1]. The comparison of scores from a pretest ensures that there are no differences between groups prior to the interventions, and the comparison of pre-test and post-test scores enables the calculation of any differences in each group after the interventions have taken place [1].

As our study was exploratory, we did not establish a priori hypotheses as to which between DGBL and industry tools would be more effective than the other. However, our general expectation, based on the literature presented above, was for both DGBL and the additional exposure to industry tools to have positive impacts on perceived competence, intrinsic motivation and occupational self-efficacy. This is because digital games have demonstrated potential to impact learning and motivational outcomes in an educational context [12], while exposure to industry tools may help signal career readiness and build relevant skills [33, 37, 42].

Although all students had the same exposure to lectures, project work and Bizagi laboratory sessions, the experimental design allowed one randomly selected group (Group B) to extend their knowledge of industry BPM tools by additional exposure to modelling and simulation tasks using the Signavio Editor and the online BIMP simulation tool. These students had knowledge of BPMN and modeling experience from Bizagi, but now learned to extend that to a new editor (Signavio). They could either use their own Signavio bpmn file for simulation or use an instructor provided solution. The second randomly selected group (Group I) played the IBM Innov8 game as an opportunity to apply their understanding of BPM in a virtual business environment. The call-centre scenario within the Innov8 game was used in the session.

Students were randomly allocated to one of the two groups at the beginning of the experimental session, at the end of 5-weeks into the 7-week course. A baseline questionnaire was administered to each group to establish their existing levels of motivation (5 items), competence (6 items) and occupational self-efficacy (3 items). All items were measured on 7-point Likert scales. The motivation and competence items were adapted from Deci and Ryan’s intrinsic motivation inventory, and occupational self-efficacy adapted from Lent [32]. Negatively phrased items were reverse-coded.

3 Results

We carried out pre (baseline) and post (after exposure) comparisons of both groups. Results are summarized in Table 1. Baseline tests show that the randomization resulted in two equivalent groups prior to our intervention with no significant differences across the 14 questionnaire items.

Table 1. Pre- and post-test comparisons.
Full size table

Within Group I, significant decreases were observed from pre-test to post-test in perceived competence, aspects of intrinsic motivation such as interest and enjoyment, and in occupational self-efficacy indicators (see Fig. 1).

Fig. 1.
figure 1

Pre- and Post-Test comparison of group I (Innov8).

Full size image

On the other hand, Fig. 2. illustrates no significant differences for Group B, except for a slight marginal decrease in enjoyment (p < 0.10), but a significant increase in career preference for BPM. An examination of post-test differences shows that Group I is lower than Group B on perceptions of competence. Their lower levels and drop in perceived competence could be due to the immediate feedback provided by Innov8 when students failed to adequately redesign the process models to achieve improvements in the KPAs as envisaged by the game. Games appear to embed feedback mechanisms that allowed students to recognize limitations and work to improve them, which is absent in industry toolsets. Frustration with success in the game also spilled-over into motivational and career outcomes. By contrast, Group B students believed post-test that they possessed more skills needed for a BPM career and reported increased feelings of competence. This suggests that exposure to industry tools signals workplace readiness to students far more than the game experience.

Fig. 2.
figure 2

Pre- and Post-Test comparison of group B (Signavio and BIMP).

Full size image

Figure 3 graphs the composite means for perceived competence, intrinsic motivation and occupational self-efficacy for the post-test comparison of the two groups. There were no significant differences in post-test motivation.

Fig. 3.
figure 3

Post-Test comparison of groups.

Full size image

4 Discussion and Conclusion

Given the demand for additional BPM skills in the market, educators must focus on strengthening students’ beliefs in their ability to succeed in BPM careers. BPM course evaluations should therefore collect data on outcomes such as occupational self-efficacy along with traditional motivation and competence-related factors.

In our study, the group selected to learn an additional set of industry-relevant tools scored higher on post-test perceptions of competence, motivation and occupational self-efficacy than those in the DGBL group. These results suggest that students appreciated the opportunity to gain hands-on experience with industry tools and that such exposure is important, particularly for increased occupational self-efficacy outcomes. However, industry tools may not increase outcomes beyond a specific point. Our course had introduced students to Bizagi and the subsequent exposure to additional tools did not significantly increase perceived competence from pre-test baseline of this group. There were small non-significant increases in most perceived competence items possibly because students could see how their skills transferred across tools. Motivations were not significantly affected, but were slightly lower possibly due to the experimental tasks adding little additional enjoyment.

In the DGBL group, we observed an unexpected general decrease in perceived competence along with indicators of motivation and self-efficacy among the DGBL group. One explanation for this finding could be a “Dunning-Kruger Effect” that occurs when individuals are unaware of their own incompetence, or are in a stage of unconscious incompetence, where they have a tendency to over-estimate their level of competence [9]. This means that perceived competence could be inflated before students have been truly ‘tested’ by the game. A digital game such as Innov8 can provide students the chance to apply skills and to move them from unconscious incompetence to conscious incompetence, observable in a reduction of their perceived competence. Based on this, we do not necessarily conclude that a drop in perceived competence resulting from digital game play is a negative outcome to be avoided, but rather we speculate whether the direct feedback mechanism embedded in the game allowed students an opportunity to reflect on their actual skills. These students were exposed to an initial ‘reality’ check that escaped their colleagues in the other group. Thus, although games should not substitute in the curriculum for exposing students to industry tools, games can usefully provide a feedback mechanism absent from BPM tools. However, it is important to note that digital games are considered to work best when they provide feedback that is free from negative judgements [46]. However, the Innov8 game resulted in many students being ‘fired’. Ending the game as a ‘loser’ will very likely having a negative effect on motivation and cause a student to question their self-efficacy. Moreover, because perceived competence is a predictor of motivation with SDT, the drop in perceived competence would subsequently result in reduced motivation.

An alternative explanation for our finding is that there was a confounding learning problem stemming from lack of sufficient background knowledge in the business case of the game [43]. Innov8 game was based on a call-centre context and students may have faced additional learning in attempting to understand and relate to the specific processes and key performance targets.

Taken together, we recommend the incorporation of both industry-relevant tools and games into the practical components of BPM courses as they have differential effects on competence, motivation and occupational self-efficacy. Industry tools increase occupational self-efficacy and games present a ‘reality’ check against perceived competence. Games must however be implemented to ensure students receive positive and constructive feedback. Where BPM educators do not have control over game design, they may wish to make use of a post-game briefing to deal with any unintended consequences of negative judgement experienced during game play. Enhancing student background knowledge of the context of the game prior to game play can also help to reduce any confounding learning problems.

Much work also remains to determine best-practices in the use of industry tools and future work should determine which tools produce the best outcomes and in what sequences. Mastery of one tool may be sufficient for development of interest and motivation toward BPM. However, if educators combine multiple professional tools along with digital games, they may wish to consider how to position game play within the course so that students have an added opportunity to build confidence in marketplace readiness and mastery over real-world BPM tools following the ‘reality’ check of game play.

A major limitation of our work is that it was restricted to a single session with a pre- and post-test design. Future work should consider comparing outcomes in longitudinal studies with ongoing exposure to different interventions.