Keywords

1 Introduction

1.1 Overview

Serious games are used for many purposes; throughout this paper reference to serious games will adopt a definition specific to the purposes of education and training: “A serious game is an experience designed using game mechanics and game thinking to educate individuals in a specific content domain” [1, p. 15]. It has been suggested that serious games provide a number of advantages over traditional learning or instructional approaches, including flexibility, creative problem solving, greater engagement and enjoyment in the material [2, 3] and the ability to produce metrics that are valuable to debriefing learners and informing ongoing serious game development [4, 5].

However, many serious games fail to leverage their full advantages, and this is particularly true in respect of adaptive mechanisms and dynamic difficulty adaption (DDA). Adaptive training in serious games has the benefit of being a cost-effective method of providing training that closely approximates one-on-one tutoring and increases the overall effectiveness of training [6]. Traditional games have explored DDA in 3D games for well over a decade, and have demonstrated the efficacy of these systems on player enjoyment and engagement [7, 8]. Serious games have an additional purpose to provide a learning outcome for players. Thus, the mechanisms for adaption may need to be different as the foci is on learning outcomes rather than purely entertainment, although enjoyment and engagement are crucial components of effective learning.

A range of approaches have the potential for being useful for adapting serious games. This includes, but is not limited to, inventory and pick-up adaption [7], pedagogical agents [9], and a wide range of AI implementations. Serious games that adjust the challenge level in-line with the growing capability and knowledge of the learner are ideal for maintaining engagement, motivation and learning outcomes [3]. In this paper, we discuss DDA for adapting serious games, and extend this to consider approaches for measuring cognitive load that could be used to drive the adaptive mechanism in serious games. From this, we present a conceptual framework for cognitive adaptive serious games based on cognitive load theory and discuss future research plans to develop and assess this approach.

2 Core Components for Cognitive Adaptive Serious Games

2.1 Dynamic Difficulty Adjustment and Adaptive Techniques

The purpose of an adaptive framework is to assess the current state of the individual learner and adjust the training to better suit their needs within the constraints of the training requirements. In a commercial video games context, there has been significant research into developing systems to implement DDA [10, 11]. The aim of DDA is to provide the player with the correct level of challenge in order to make the game neither too hard nor too easy, thus increasing enjoyment [11, 12]. This concept of optimal challenge for the purpose of enjoyment is implemented with the aim of assisting the player achieve a flow state [11, 13]. Ideally, the benefit of DDA is that it will continuously adapt to player skill in an appropriate and subtle manner.

As previously indicated, the primary aim of a serious game is to deliver instructional, learning, or development outcomes. As a result, DDA in this context needs to relate to the performance of the participant against the serious games purpose rather than solely to enhance entertainment through amplification of flow states. Our conceptual model adopts a three-part framework towards engagement that encompasses flow along with other aspects suited to serious games [14]. One of the crucial aspects we consider is the role of cognitive load in the learning process, acknowledging the role of cognitive load levels in providing insights into schema integration amongst learners [15].

2.2 Measuring Cognitive Load

Cognitive load is the degree to which a learning task meets, exceeds, or fails to reach the processing capacity of a learner’s cognitive system [16]. This is explained through Cognitive load theory (CLT), which originated in the 1980s [17]. It is a widely accepted concept that describes three states of the cognitive processes involved in learning; intrinsic, extraneous and germane cognitive load [18]. There are a number of methods of measuring cognitive load, principally subjective and objective measures. Subjective measures are typically completed by participants after undertaking an activity, for example the NASA Task Load Index (NASA-TLX) [19, 20]. In contrast, objective measures are undertaken during an activity, measuring an observable occurrence, affect, or physiological system, and interpreting that data for cognitive load.

The conceptual framework detailed in this paper proposes the use of an adapted version of the detection-response task (DRT) [21] embedded within a serious game. The DRT has been chosen as it is registered through the International Organization for Standardization in ISO 17488:2016 as a proven and effective measure [22]. A virtual DRT, termed the “Remote DRT”, has been previously tested in-simulation, and proven to be effective [21]. An updated version of the “Remote DRT” is proposed here, potentially making a cognitive load based adaption an accurate, effective and affordable method for wide scale adoption.

2.3 Adapting Serious Games Through the Frame of CLT

CLT provides three aspects of cognitive load that can be manipulated to establish a sophisticated and theoretically sound adaptive framework. A practical description of how these adaptions may be implemented serious game tasks is outlined in Table 1.

Table 1. Adaptive examples tailored to CLT
Full size table

Using CLT as a framework provides a structured way for a combination of cognitive load and various performance metrics to be used to dynamically adapt a serious game to optimize learning.

3 Conceptual Framework

The conceptual framework detailed here is an extension of the serious games conceptual model proposed by Yusoff et al. [24]. However, the Yusoff et al. model does not incorporate learner motivation, affect, and prior knowledge as described in the cognitive-affective theory of learning with media (CATLM) [9]. Yusoff et al. define a range of factors, and the conceptual model outlined below adopts their definitions and adds the following components by drawing on CATLM and others [25]:

  1. 1.

    Learner Knowledge & Experience – understanding the learner’s current state of intrinsic motivation, knowledge and experience is essential [6]. It is important to assess the player’s ability with the serious game controls to avoid the learner struggling with the controls, rather than the learning content within it.

  2. 2.

    Review, Iterate – post-game review otherwise termed after-action review. This is critical to the learning process and is often underutilized in serious games [4, 5]. It is important for the facilitator to assess the performance of the serious game itself.

  3. 3.

    Play Game – forms the start/finish of the cognitive and performance measure loop.

  4. 4.

    Cognitive load and performance measures – a constant loop driven by the cognitive measure. This process includes a performance measure, this is separate to the cognitive measure loop however they are combined to inform the DDA mechanism.

  5. 5.

    Feedback – is similar to Reflection described by Yusoff et al. [24] and by Moreno and Mayer [9]; this is in-game feedback to help and inform the player.

  6. 6.

    DDA ±/− – This is the point at which an adaption mechanism is implemented making the game easier, harder or the same as described in Table 2.

    Table 2. Performance and cognitive load (CL) adaption template
    Full size table
  7. 7.

    Game Performance – outputs the player performance and cognitive load measures to inform the debrief process and other course requirements [4, 5].

Table 2 presents the various proposed DDA adaptions including concepts discussed in the preceding section (Sect. 2.3). This follows on from the identification of how difficulty may be adjusted (Table 1). Together, the existing serious game conceptual models, extended to incorporate the inclusion of an adaptive mechanism based on cognitive load, results in a new Cognitive Adaptive Serious Game Framework (CASG-F) (Fig. 1). Performance measures and cognitive load will work together to adapt the in-game tasks. However cognitive load will form an adaption mechanism in its own right, e.g. making a simulated car automatic instead of manual to reduce interaction elements (intrinsic cognitive load), or increasing challenge by introducing additional tasks (intrinsic) or weather effects (extraneous cognitive load).

Fig. 1.
figure 1

The Cognitive Adaptive Serious Game Framework (CASG-F)

Full size image

4 Future Research

The CASG-F proposed here has three broad stages in order to encompass instructional design concepts [26] and the triadic theoretical framework [27]. The CASG-F provides a roadmap for future experiments to incorporate these principles as best practice, and to outline how they are being integrated into the design and development process. The overarching aim of this research is to create an adaptive framework that can work flexibly and effectively using cognitive load as a key adaption measure.

Moving forward, robust experimental studies to validate the proposed adaptive process, and also to provide efficacy for the approach, are planned. Firstly, an experimental study is required to assess the in-game use of the DRT cognitive load measure, and to further validate its authenticity using a triangulation approach with an EEG combined with the NASA-TLX. The first experiment enables statistical validation of the affect different interventions have on the participants’ cognitive load, providing a ‘toolbox’ of adaptive techniques for future experiments. Later experimental studies will then be required to compare adaptive serious game performance to non-adaptive variants in order to corroborate the assumption that adaptive serious games are more effective in enhancing human performance.

The first experiment involves a driving task in a 3D game environment. This has the benefit of using a task where the real-world efficacy of the DRT has already been established [21]. The proposed experiment will:

  • Present three levels with the same driving track layout; the surrounding virtual environment will be different in each layout minimizing repetition for participants, and also provide information on how different visual environments affect cognitive and visual load.

  • Using a randomized approach, participants will drive two loops of the track; one loop will have the DRT and the other will not, the EEG will remain on for both loops.

  • The levels will include challenge sections, for example navigating a narrow section of road, observation tasks, following a vehicle, and more.

  • Qualities of the visual environment will be manipulated in order to observe how these changes relate to visual load and cognitive load, e.g. altering in-game weather and lighting. Quantifying the effect of visual environment manipulations on cognitive load will then further contribute to the adaption ‘toolbox’.

A study following this design will facilitate evaluation of the effect of the DRT itself on the cognitive load of the player participants. Experimental designs based on the CASG-F should also consider the layering of primary and secondary tasks to allow for robust measurement of cognitive load using the DRT approach [28]. To this end, the player should be given an additional task to perform requiring different cognitive processes. For example, asking the participant to count or respond to certain assets in the environment, requiring visual and cognitive discrimination, will effectively alter cognitive load [28]. Together, the CASG-F and the proposed experimental design provides an authentic first step toward a generalizable framework for developing serious games that respond dynamically to improve learning outcomes.

5 Conclusion

In this paper we propose a framework to realize the value of DDA and cognitive measures as applied to serious games. We then discussed how an adaptive framework can be applied through the lens of CLT. Extending existing theoretical frameworks, a conceptual model for Cognitive Adaptive Serious Games was presented that incorporates and recognizes different learning design approaches. Additionally, the proposed model has the potential to deliver real-time personalized training environments to move serious game implementations closer to the overall aim of enhancing human performance. Lastly, a comprehensive experimental design for future studies was presented where in-game cognitive load will be measured and verified by a variety of methods against a range of stimuli. This experiment design provides a roadmap for validating the real-time DRT as a cognitive load measure, and to establish a toolbox of validated methods to manipulate cognitive load based on a robust understanding of the impact of different game mechanics and mechanisms on the cognitive load of end users. Once established, this toolbox can then be applied to serious game implementations to assess the efficacy of adaptive serious games for education and training.