Abstract
Why do people learn after playing a serious game versus a game for entertainment? Serious games impart knowledge because there is a pedagogy driving the learning process. Serious games must successfully employ pedagogical methods and theories to increase the likelihood that knowledge is. The process of learning is hindered when an unusable interface demands cognitive resources that should be allocated to learning. Despite the creation of a usable system, if the player’s interaction with the model is hindered, can real transfer of knowledge occur? Within the context of serious games that make use of model-based training, we suggest that a measure of pedagogical usability is warranted. The authors provide a conceptual basis for measuring pedagogical usability, specifically targeting serious games that employ modeling as the mechanism of action.
You have full access to this open access chapter, Download conference paper PDF
Similar content being viewed by others
Keywords
1 Introduction
Usability has been defined as the extent to which a player is able to understand, learn, and control the functions of a system [1]. More specifically, usability focuses on the ease of use of the independent functionalities within individual components of a game (e.g. functionality of displays, player input and usefulness of in-game feedback), making it a key feature of a system [2]. Usability in serious games is of particular importance; in accordance with cognitive load theory, the process of learning is hindered when an unusable interface demands cognitive resources that should be allocated to learning [2]. Moreover, usability can influence a variety of other outcomes, such as user satisfaction [3, 4], errors [4], and attitude [5, 6].
In order for a game to be usable, developers must consider human limits in terms of memory, perception, and attention, as well as anticipate probable user errors [7]. Thus, usability should allow users to successfully execute a given task as determined by salient objectives. Despite high performance, a system is unlikely to be utilized if users are generally dissatisfied with the mechanics or interface of the system [8]. It is important to quantify how efficiently players interact with the game to both validate gameplay and further reveal insight as to the mechanics underlying serious games. With this in mind, usability reflects the degree to which the system affords the user the ability to achieve objectives within a specific domain of use [9].
The last decade has seen a substantial rise in research on the emotional and affective aspects of the gaming experience. Traditional usability metrics are relevant, but must be adapted to the videogame context [10, 11]. Given the rise in interest in usability within the realm of HCI research, a variety of usability models and considerations of design principles have emerged with foci (e.g. emotion, affect, experience, pleasure, hedonic qualities) [12]. However, measures of these facets may not convey enough information about the efficacy of serious games.
2 Conceptual Basis of a New Approach Usability Evaluation
There are a variety of benefits to evaluating usability or user experience throughout the game development process [7]. Unfortunately, there are a multitude of shortcomings and challenges that inhibit the efficient and effective implementation of such evaluation. Most usability metrics are designed to evaluate traditional, task-oriented software. However, the nature of gameplay is quite different from that of traditional software [7]. Thus, this creates substantial difficulties for researchers investigating non-task-oriented software (i.e., the game industry). To overcome these difficulties, researchers have made efforts to adapt and create tools specific for the game industry. At this time, popular usability measures (e.g. efficiency) only convey how well a game is designed. These measures further fall short for the serious games designer. As such, it would be advantageous to utilize an underlying pedagogy to drive the game design process for designing serious games.
Why do people learn after playing a serious game versus a game for entertainment? Games for entertainment can be objective driven (think Call of Duty), but they are not designed to impart knowledge or skill outside of the game environment. Serious games, on the other hand, target the acquisition and transference of knowledge or skill outside the game environment. Serious games that achieve transference of knowledge do not work for the primary reason of having an eye-catching character or great theme music, rather it is the addition of an underlying pedagogy (activities that impart knowledge or skill) that is driving the learning process [13]. Therefore, serious games must successfully employ pedagogical methods and theories to increase the chance that knowledge is obtained [14]. In other words, in the field of clinical psychology, it is often said that treatment caused change; however, demonstrating cause does not explain why treatment produced change in behavior. Rather, it is a mechanism that explains how a treatment translates into events that lead to changes in behavior [15].
Given the educational nature of serious games, we suggest that usability in serious games must go above and beyond interface usability. This does not suggest that a sound underlying pedagogy will overcome all traditional UX principles; rather, despite the creation of a usable system, if the “player” does not evaluate the system as being usable can real transfer of knowledge occur? Thus, within the context of serious games that make use of model-based training, we suggest a need for a metric to assess whether a serious game integrates modeling in a user-friendly manner in serious games.
A variety of learning theories exist that have been applied to serious games [16]. Among the easiest and most effective pedagogical methods that can be implemented for serious games is behavior modeling. Modeling, as the pedagogical driver of learning and transference of knowledge, in the context described by Kazdin [15], is the mechanism of action that explains how a serious game translates into the transference of knowledge and skill. In order to effectively utilize modeling, we must first understand the history and principles of modeling.
2.1 Modeling as an Instructional Approach
Modeling has its roots in Albert Bandura’s version of Social Learning Theory [16], later renamed Social Cognitive Theory [17]. Bandura explains, “Learning would be exceedingly laborious, not to mention hazardous, if people had to rely solely on the effects of their own actions to inform them what to do. Fortunately, most human behavior is learned observationally through modeling: from observing others one forms an idea of how new behaviors are performed, and on later occasions this coded information serves as a guide for action.” [18, p.22] In other words, Bandura [17, 18] purported that the observer acquires an organized knowledge base, called a schema, of the modeled behavior. Later, the observer is able to drawn on the schema to execute novel behavior. With that said, it is essential to differentiate between acquisition and spontaneous performance of the observed behaviors [19].
How does modeling work? Modeling is comprised of salient behaviors to be learned, models exhibiting how to effectively execute these behaviors, opportunities for practice, feedback, and social reinforcement [18]. Specifically, Bandura [19] explains that modeling encompasses “various subsystems” [p.221]. These subsystems are: (1) attentional processes, (2) retention processes, (3) motoric reproduction processes, and (4) incentive or motivational processes.
-
1.
Attentional processes explains that it is unlikely that an individual could replicate the modeled behavior, or responses, if the individual does not, “attend to, recognize, and differentiate the distinctive features of the model’s responses.” [19, p.222]. Serious games designers must create an environment that enables the observer, or trainees, to replicate the target behavior. The observer must be able to easily identify the model and view the demonstration of the target behavior [17, 18]. Superfluous stimuli should be kept to a minimum because it may flood the learners’ attention processes, thereby misdirecting resources. In other words, the trainee is not able to devote attention to the reference stimuli being targeted for learning.
-
2.
Retention processes concerns the “…long-term retention of coded modeling events.” [19, p.222]. During exposure to the stimulus, the observer recodes, classifies, and reorganizes the observed elements of the target behavior into familiar and relevant representations. The translation into relevant representations allows the observer to create schemata that are more easily remembered. This is accomplished despite the lack of direct interaction with the target behavior. In other words, the observer has created a symbolic representation [19]. For these processes to occur, serious games designers must create powerful environments that enable the observer to attend to and reproduce thru covert rehearsal [19].
-
3.
Motoric reproduction processes utilizes the symbolic representation that the observer has created to guide overt performance [19]. The motoric reproduction process takes the form of rehearsal. Whether mental or physical, rehearsal is critical for retention and improved performance; however, learners may be unable to reproduce observed behaviors. The capability of the observer to reproduce the symbolic representations of the modeled behavior serves as a guide to appropriate action. Serious games designers need to provide an environment that enables the observer to rehearse the target behavior because symbolic representation alone will not result in success [19]. Thus, it is the combination of the quality of the cognitive representation and the environment that dictates whether the learner is able to reproduce the modeled behavior [20].
-
4.
Motivational or incentive processes determine whether the learner will willingly exhibit the modeled behavior [21, 22]. Incentive or motivational processes can affect learning, retention, and performance. The observer creates perceptions of either positive or negative outcomes regarding the implementation of the observed behavior. Depending on the observer’s conclusions regarding the outcome, the probability of the individual performing the behavior will either increase or decrease. Serious games designers need to create environments that foster motivation and self-efficacy. Bandura [22] illustrated that when the individual perceives the outcome of their behavior to produce positive incentives the observational learning promptly emerges in action.
2.2 Measuring Pedagogical Usability
The goal of a pedagogical usability metric is to evaluate the extent to which the system affords the player the opportunity to successfully complete each step of modeling. As such, each of the four subsystems of modeling as supported by theoretical and empirical work should be considered: (1) attentional processes, (2) retention processes, (3) motoric reproduction, and (4) motivational processes. Items should reflect the extent to which the model in the game was implemented into the game to accurately complete each stage of modeling.
3 Discussion
Usability testing is critical to the development of efficacious serious games. With the addition of measures that reflect underlying pedagogy, we purport that it is possible to improve the serious game design process, thereby increasing the efficacy of the learning tool. Serious games are unique in that they implement pedagogy to drive the learning process. Thus, the purpose of the present paper is to propose a theoretical basis for creating a measure of modeling in usability. The aim of the measure should be to identify the extent to which a serious game employs the pillars of modeling. We argue that games should be evaluated for the extent to which trainees have an understandable interaction with the system. More specifically, we suggest that games that utilize modeling must incorporate the four subsystems described by Bandura.
Our contribution is two-fold. First, we provide a novel notion of usability, which goes beyond traditional system usability by incorporating an evaluation of pedagogical usability. Secondly, we provide a theoretical basis for a measure that can be used at multiple points throughout the development process. The use of a measure that allows for multipoint assessment has the potential to save time and money, which is important considering the design process, is often costly. Moreover, we purport that this measure will allow designers and developers to pinpoint why a particular function does not successfully facilitate the learning process.
It is our hope to provide the serious games industry with a novel, useful, and practical measure of usability that can be utilized throughout the design process to increase the quality and efficacy of their product. By incorporating usability testing of the pedagogical approach underlying the training, the industry can ensure the production of well-designed games that have the potential to produce the anticipated learning outcomes.
References
Pinelle, D., Wong, N., Stach, T.: Heuristic evaluation for games: usability principles for video game design. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1453–1462. ACM, April 2008
Olsen, T., Procci, K., Bowers, C.: Serious games usability testing: how to ensure proper usability, playability, and effectiveness. In: Marcus, A. (ed.) Design, User Experience, and Usability. Theory, Methods, Tools and Practice, pp. 625–634. Springer, Heidelberg (2011)
Constantine, L.L., Lockwood, L.A.: Software for Use: A Practical Guide to The Models and Methods of Usage-Centered Design. Pearson Education, Upper Saddle River (1999)
Nielsen, J., Phillips, V.L.: Estimating the relative usability of two interfaces: heuristic, formal, and empirical methods compared. In: Proceedings of the INTERACT 1993 and CHI 1993 Conference on Human Factors in Computing Systems, pp. 214–221. ACM May 1993
Preece, J.: Book Review: Developing User Interfaces: Ensuring Usability Through Product and Process by Deborah Hix and H. Rex Hartson (John Wiley and Sons, Inc., 1993). ACM SIGCHI Bulletin, 26(1), 74-75 (1994)
Shackel, B.: Usability-context, framework, definition, design and evaluation. Human Factors for Informatics Usability, pp. 21–37 (1991)
Isbister, K., Schaffer, N.: Game Usability: Advancing the Player Experience. CRC Press, Boca Raton (2008)
Chin, J.P., Diehl, V.A., Norman, K.L.: Development of an instrument measuring user satisfaction of the human-computer interface. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 213–218. ACM, May 1988
ISO 9241-11: Guidance on Usability, also issued by the International Organization for Standardization (1998)
Nacke, L., Lindley, C., Stellmach, S.: Log who’s playing: psychophysiological game analysis made easy through event logging. In: Markopoulos, P., de Ruyter, B., IJsselsteijn, W.A., Rowland, D. (eds.) Fun and Games 2008. LNCS, vol. 5294, pp. 150–157. Springer, Heidelberg (2008)
Tychsen, A., Canossa, A.: Defining personas in games using metrics. In: Proceedings of the 2008 Conference on Future Play: Research, Play, Share, pp. 73–80. ACM, November 2008
Law, E.L.C., Roto, V., Hassenzahl, M., Vermeeren, A.P., Kort, J.: Understanding, scoping and defining user experience: a survey approach. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 719–728. ACM April 2009
Hassenzahl, M.: The interplay of beauty, goodness, and usability in interactive products. Hum.-Comput. Interact. 19(4), 319–349 (2004)
Jordan, P.W.: Pleasure with products: human factors for body, mind and soul. In: Green, W.S., Jordan, P.W. (eds.) Human Factors in Product Design: Current Practise and Future Trends, pp. 206–217. Taylor & Francis, UK (1999)
Harteveld, C., Guimarães, R., Mayer, I., Bidarra, R.: Balancing pedagogy, game and reality components within a unique serious game for training levee inspection. In: Hui, K.-c., Pan, Z., Chung, RC.-k., Wang, C.C., Jin, X., Göbel, S., Li, E.C.-L. (eds.) Edutainment 2007. LNCS, vol. 4469, pp. 128–139. Springer, Heidelberg (2007)
Kazdin, A.E.: Mediators and mechanisms of change in psychotherapy research. Annu. Rev. Clin. Psychol. 3, 1–27 (2007)
Egenfeldt-Nielsen, S.: Beyond edutainment: Exploring the educational potential of computer games (2005). Lulu.com
Bandura, A., Walters, R.H.: Social Learning and Personality Development. Holt Rinehart & Winston, New York (1963)
Bandura, A.: Social foundations of thought and behavior: A social-cognitive theory (1986)
Bandura, A.: Social Learning Theory. Prentice-Hall, Englewood Cliffs (1977)
Bandura, A.: Social-learning theory of identificatory processes. In: Goslin, D.A. (ed.) Handbook of Socialization Theory and Research, pp. 213–262. Rand McNally, Chicago (1969)
Bandura, A.: Influence of models’ reinforcement contingencies on the acquisition of imitative responses. J. Pers. Soc. Psychol. 1(6), 589 (1965)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Kreutzer, C., Marks, M., Bowers, C. (2015). A Pedagogical Approach to Usability in Serious Games. In: Stephanidis, C. (eds) HCI International 2015 - Posters’ Extended Abstracts. HCI 2015. Communications in Computer and Information Science, vol 528. Springer, Cham. https://doi.org/10.1007/978-3-319-21380-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-21380-4_8
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-21379-8
Online ISBN: 978-3-319-21380-4
eBook Packages: Computer ScienceComputer Science (R0)