Keywords

1 Introduction

According to the Diagnostic and Statistical Manual of Mental Disorders – DSM-5 [1], autism is characterized by persistent limitation in social communication/interaction, restrictive and repetitive behavioral patterns, and (verbal and non-verbal) communication deficits. As the autism diagnosis evolved to cover a broader spectrum of symptoms, there is emerging evidence of increasing epidemic rates. Numbers may vary depending on the metrics used for evaluation, but there are studies reporting on rates as high as 1 in 150 children being diagnosed with Autism Spectrum Disorder (ASD) [2].

Accessibility has played an important role in providing autistic individuals with opportunities to overcome their limitations by fostering independence and assisting them in social relationships [3]. Despite the fact that much information on autism etiology remains to be uncovered, accessibility advocates (researchers, psychologists, therapists, teachers, among others) have been thriving to propose feasible and interesting intervention solutions, which can be confirmed by the growing number of computer-based resources published and available to the community.

Serious games and gamification strategies have been largely used in the treatment of psychiatric conditions, such as depression, eating disorders, substance use, dementia, and, also, Autism Spectrum Disorder [4]. Gamification is a powerful resource to increase motivation and engage participants, thus favoring the teaching-learning process or training for specific situations. Design elements of serious games and/or gamified interfaces often include storylines, mid-term and long-term goals, increasing level of difficulty, feedback and/or rewards, and provision of choice [5].

Recent studies suggest improvements in handling disorder-related symptoms through the use of serious games or gamified interventions [6]. The same applies to autism: gamification may be a potentially effective approach due to the high visual processing skills the ASD individuals possess. On the other hand, developers may face challenges. According to [5], little evidence exists that ASD individuals subjected to computer-based interventions are able to apply such learning to real life. This can be partly explained by the lack of fundamental gamification elements during the design process.

In light of this situation, there is an undeniable need to understand the current status of serious gaming targeted to Autism Spectrum Disorder. Through a systematic review, this paper collects data on serious games and/or gamified interventions for autism, with a focus on the gamification elements used and their visual representation in terms of the User Interface design. We believe that this study may help uncover best practices used and shed light on the graphic design approach to deliver them at the same time.

This paper is organized as follows: in Sect. 2, the review protocol is presented, including research questions, inclusion criteria, search strings, and databases; in Sect. 3, results from the review are described and discussed; finally, Sect. 4 brings conclusions and future perspectives regarding the design of gamified interventions for the autistic audience.

2 Review Protocol

This systematic review applies a three-phase process, including the design of the review protocol, the conducting of the search, and the data extraction [7]. We search, identify, analyze, and summarize evidence on computer-based gamification resources targeted to the ASD audience. In order to prevent biased or mixed results, the scope is limited to resources in which the User Interface is the primary output channel. Robotics, hardware-only ubicomp solutions, or any interface-less devices are not included in this study, because their examination would require a completely alternative approach.

Preliminary studies and consultations with field experts helped pilot the review protocol. The protocol was designed to provide comprehensive guidance through the review process, with respect to the inclusion criteria and the nature of the results. It intends to promote the identification of all relevant primary sources and assist the data extraction process at the same time. The research questions are: what is/are the learning objectives? What is the development process/methodology/approach employed? Who is the target audience? What gamification elements are used to improve learning skills? What representation methods are applied in the User Interface design?

Then, a list of keywords was defined to support the searching process. Keywords should cover three aspects of major importance, namely: (1) the target audience, that is, autistic individuals; (2) the object of study – software development; and (3) the intervention approach – computer-based serious games or gamified interventions. Similar keywords were grouped to form a search string, using Boolean “ANDs” and “ORs”: (“Autism spectrum disorder” OR “autistic individuals” OR “autism” OR “high-functioning autism”) AND (“computer-assisted learning” OR “computer-assisted intervention” OR “computer-enhanced learning” OR “therapy software”) AND (“serious games” OR “gamification” OR “gamified resources”).

In order to be included, papers had to adhere to the sensitivity inclusion criteria: describe a novel contribution to computer-based serious games for ASD; have the software development as the primary object of study; provide relevant information on the gamification elements used during development; and rely on the User Interface as the main output channel. Secondary studies and hardware-only interventions were not considered. Also, publications older than 10 years were discarded, thus prioritizing the recent literature.

Six sources were selected in this study: (1) IEEE Xplore Digital Library, (2) ACM Digital Library, (3) SpringerLink, (4) SCOPUS, (5) ScienceDirect, and (6) Google Scholar. Together, they grant access to a large collection of relevant resources for Software Engineering. Besides, important journals on autism research can be accessed through these sources, such as the Journal of Autism and Developmental Disorders (Springer) and the Autism (SCOPUS).

Searching was performed using a combination of the search string described previously. Multiple trials of the keywords were performed in order to reach as many primary studies as possible. The search resulted in a combined total of 2,995 papers, which were downloaded. After the exclusion of duplicated papers, the total narrowed down to 2,642 studies. Then, two assessors examined titles and abstracts to determine the relevance of the subject matter. Papers that failed to meet the inclusion criteria were eventually withdrawn from this review, resulting in 30 eligible studies.

3 Results and Discussion

All 30 selected papers were carefully examined according to the review protocol. Data extracted was compiled in quality reports and double-checked whenever assessors had conflicting recommendations. Quality reports included data on the learning objectives, the team organization, the software development process applied, the target audience, gamification elements, and the User Interface design.

The selected papers present a variety of learning objectives. Many aim to improve communication skills (46.6%), mostly by teaching new vocabulary and phrases while improving semantic meaningfulness. Other learning objectives are social interaction (13.3%), basic education (10%), and daily activities (10%). Two studies focused on facial recognition [14, 18] and only one aimed at refining motor skills [10]. Six of them (20%) presented multiple learning purposes [9, 13, 17, 19, 25, 32]. All of them involved the major impairments faced by ASD individuals: difficulty of socialization, imagination impairment, communication deficit, and behavioral issues [12] (Fig. 1).

Fig. 1.
figure 1

Learning objectives extracted from primary studies.

Full size image

Popular development approaches are the Participatory Design [15, 17, 30, 32] and the User-Centered Design [22,23,24, 31]. Both are techniques that aim to involve external help (experts, parents, teachers, and/or end-users) to assist the development process. Other approaches mentioned are the Applied Behavior Analysis [12], the ADDIE approach (Analysis, Design, Development, Implementation, Evaluation) [33], and the Extreme Programming [20]. Some studies reported on the use of an “iterative” [13, 14, 16, 21, 27, 29, 35] and/or “agile” approach [21, 27, 36], although they did not provide detailed information on it.

The team organization is either cross-functional or participatory. Cross-functional teams are composed by the development staff and stakeholders, who are deeply involved in the development lifecycle. On the other hand, participatory teams also engage stakeholders, but they are involved only in specific moments during the process (requirements specification or testing, for example). In any case, this reveals an effort to understand the needs and preferences of the end-user. Two studies did not provide any information on the methodological approach or the team organization [10, 34].

The vast majority of studies analyzed (76.6%) refer to their target audience simply as “autistic children” or some variation of the term. Only four studies report on the development of software for severity-specific audiences – that is, high-functioning and/or low-functioning autism [24, 28, 32, 36]. Two studies (6.6%) referred to the audience as “autistic individuals”, claiming that the proposed intervention would serve for ASD adolescents or adults as well [16, 34]. There is also one study [20] that designed an intervention for children with “specific disorders”, such as ASD, Down syndrome, mental retardation, etc. According to the author, this is possible due to the use of the PECS (Picture Exchange Communication System), which is an educational resource largely used in interventions for several disturbances, including autism.

Table 1 summarizes data on the gamification elements applied in the software and the User Interface design, that is, what visual representation methods were used to deliver the interface. Also, it clusters the primary studies according to the learning objective(s) they aim to achieve. “C” stands for Communication; “M” is Multiple, meaning that the study intended to cover more than one learning objective at the same time; “SI” means Social Interaction; “BE” means Basic Education; “DA” stands for Daily Activities; “FR” means Facial Recognition; and “MS” is an abbreviation for Motor Skills.

Table 1. Gamification elements and user interface design.
Full size table

3.1 Gamification Elements

According to Table 1, many of the studies use feedback (50%) and/or rewarding systems (33%) to appeal to the audience. Those can be either virtual output stimuli, such as visual cues or auditory feedback, or real-life tokens (a fruit, a game, or anything that the user would associate with having positively accomplished a task). Rewards have been largely used in ASD-specific interventions, often applied immediately after the task completion, in order to reinforce certain behavioral patterns [36].

Four studies applied Virtual Reality environments within the software [9,10,11,12]. There has been increasing interest in using VR for autistic intervention, since this technology provides an immersion environment that improves interventionist control and promotes engagement [10]. In addition to that, many studies (60%) increased the level of difficulty as the user progresses. This feature includes either mid-term or long-term goals, and is used to foster motivation and interest in using the software.

Most of the studies also include some degree of personalization to the software they propose. This allows the child (or teachers, or parents) to customize the graphics and/or the content according to their particular needs and preferences. On the top level, it may allow for changes in the color scheme, graphics size, and layout arrangement. Besides that, it may include changes in content presentation: photographs instead of cartoon-like illustrations, text messages or picture-only content, background music on and off, etc. A few studies combine personalization with a multimedia approach, including text, audio and video options [8, 9, 11, 13, 15, 37]. These features are particularly interesting if we take into account that each ASD individual has unique interests and preferences, and that the autism covers a broad range of symptoms [8].

In terms of monitoring, many studies apply strategies to customize learning outcomes and provide parental control (56.6% combined). This is often reached by synchronizing child-parent/child-teacher devices or having multiple interface profiles. These studies report on the use of an integrated system that allows teachers, therapists or parents to monitor their children progress, and select custom learning objectives depending on individual cognitive responses. Again, ASD individuals can benefit from having tasks tailored specifically for each of them and, at the same time, teachers and parents are entitled with more control over the learning outcomes.

A few studies also reported on the use of storytelling [15, 17, 27, 29, 33]. This method uses multiple media formats (text, images, sounds, animation, and video) in order to encourage students to create their own narratives. In ASD children, this may help improve their communication skills and the symbolic function, thus promoting an autonomous attitude [27]. Other features found on the studies include familiarization (pictures of the surroundings and/or voice recording) [18, 34], information sharing [16], use of virtual avatars [13], active experimentation [27], and remote control [29].

By clustering the information in relation to the game objective, a few more insights emerge. When dealing with communication, there is a preference to work with personalization, monitoring, custom learning, and increasing level of difficulty. This may be explained by the communication process itself, which requires a certain vocabulary level in order to make sentences, and so on. The same applies to Social Interaction, since it involves cognitive skills rather than physical abilities. In fact, objectives that cope with physical development, such as Daily Activities and Motor Skills, tend to apply feedback, rewards, and also personalization strategies, which provides a more immediate response to the user. In turn, games that included multiple objectives often used a wide variety of strategies (although rewards and level of difficulty are more frequent), perhaps because a single approach would not be sufficient to comply with all the objectives at once.

3.2 User Interface Design

The User Interface design methods reported by the studies include visual presentation (colors, illustration, photographs, video), textual elements (words or phrases) and even sounds, voice/narration, and virtual 3D environments. The most popular approaches, however, are the use of illustrations (76.6%), words/text (80%), and colors (73.3%) to express feelings and guide the user through the interface. The items check-marked (refer to Table 1) represent the topics described in more detail in the papers, which does not mean unmarked items were not covered by the study.

In relation to illustrations, there are multiple approaches available. Reports range from the use of simple, sketchy, basic shapes (squares, triangles, and circles) [9, 16, 17, 22, 37] to more sophisticated illustrations [25, 26, 31]. None of them report on the use of realistic drawings. Instead, whenever the authors wanted a more realistic style, they would rely on real-life pictures or photographs. The main explanation provided in those cases is that real-life pictures would be easier to recognize, especially if the parents/teachers use the built-in camera to upload their own images. It would also look more familiar to the user, and may, in some cases, be more effective [38].

The use of colors appeared as one of the major concerns in the development process. Again, there are various approaches. Some of the studies inform on the use of more vibrant colors [16, 22, 25, 28], while others suggest a calm and half-toned color scheme [23, 36]. There is no apparent revaluation of colors based on users’ feedback. A similar situation arises from the use of words/text: there is not much information on why authors chose one strategy over another. The differences in communication deficits experienced by ASD individuals impose a challenge because some of them are able to read while others are not [1]. Thus, using an amount of written information that fits the communication skills of the target audience is of major importance to achieve success.

The use of sounds is also reported in many of the studies (56.6%), though voice recording features or narration are much scarcer (10%). Sounds, in this case, include background music and/or other sound effects (clicking a button, for instance). Sounds can be interesting decorative strategies to increase motivation and reinforce learning activities [39]. A few studies report on the use of voice recording or narration in order to add customization attributes or improve familiarization [14, 24, 30]. Around 30% of the studies also include video (self-recorded or not) and virtual environments as means to promote immersion experiences.

Once again, clustering the information according to the objectives leads to other insights. Games with communication objectives largely apply illustration, photographs, and words/text. Surprisingly, however, not many of the studies use sounds, voice recording or video modeling, which could potentially help users become familiar with the sounds of the words and practice pronunciation. The same applies to Social Interaction and Basic Education – they could greatly benefit from having sound reinforcing positive behavior. In turn, video modeling and Virtual Reality appears in all of the studies handling Facial Recognition and Motor Skills, probably because those strategies offer opportunities to simulate real situations – which could also be applied to Daily Activities.

4 Conclusions and Future Perspectives

The objective of this review was to investigate the current status of research on ASD-specific gamified interventions, emphasizing gamification elements and design representation methods applied in the software development process. Data were also collected on the target audience, the learning objectives, and the software development process. As results show, there are a good number of studies addressing the topic, which suggests more awareness of the disturbance and a willingness to move toward accessibility.

Clinically, the causes of autism are unclear, and as medicine advances more complexity is added to its diagnosis, treatment, and prognostic. On the one hand, parents find it difficult to handle autism inside their homes, because many times they have little or no assistance on how to behave with their autistic children. On the other, accessibility is often seen as unnecessary and expensive in the corporate environment. That is precisely why it became so important: development teams endure the task to develop effective solutions for autism and, at the same time, prove they are feasible from a productive, financial, and corporate perspective.

In this context, this review shows a wide collection of gamification strategies to cope with the development of autism-specific software. Despite that, many challenges emerge from dealing with the ASD audience. Studies included in this review share some common characteristics. Overall, they provide information on the development process, although the procedures are often not described in full detail (with only a few exceptions). We believe this does not mean the development process was neglected, but rather that the authors decided to highlight some of the aspects they assumed to be the most relevant. However, the lack of information is disappointing, since a more detailed description would add consistency to the studies, and also help readers understand and replicate the methods and strategies used.

The target audience is, most of the cases, referred to simply as “autistic children” or “autistic individuals”. As mentioned before, autism covers a broad spectrum, that is, different symptoms appear in different severity levels in ASD individuals [1]. For this reason, it is hard to assume a single software product would be equally effective to the entire spectrum. Having a specific target audience (including the comprehensive description of symptoms) would potentially help optimize the development process and also serve as guidance for therapists in managing the software. Furthermore, this has a negative impact on testing, since results may reflect mixed ASD profiles.

In relation to the development process/design approach, there is a clear focus on applying methodologies that prioritize the end-user. On the one hand, cross-functional teams often present a better performance, since their particular expertise helps improving the software product in terms of usability. On the other, involving external agents (teachers, parents, and the ASD users) is fundamental to collect feedback and confirm its efficiency on a User Experience level. Even though most studies did not present the design process comprehensively, they surely reveal an effort to meet the end-user’s needs and preferences.

Despite the issues with the process detailing, all of the studies show a good use of multiple techniques (gamification elements and UI design strategies). The combination of such methods for the development of serious gaming applications often results in better usability and acceptance levels [30]. We should remember, once more, that ASD individuals have complex clinical conditions. Thus, combining multiple strategies and forms of presentation may have a huge impact on how they use and benefit from the software.

Unfortunately, most of the studies analyzed did not provide consistent testing procedures or clear results achieved with the games. Many studies used indirect forms of assessment, which is valuable, but hardly represents the end-user realistically. In cases when authors did test the software with autistic children [8, 9, 11, 12, 22, 23, 25, 27, 28], samples were often small, revealing fragility in the results achieved. In addition to that, none of the studies reported having used control groups within the testing procedures. This leaves us with no reference for comparison on how the audience would have behaved with and without the serious game. Of course, the testing efforts are important but, most of the cases, it is hard to assume they are unquestionable evidence of success.

Results, in turn, are generally presented in a superficial manner. Mostly, they are described as “good/not good” or “positive/negative”, without providing further information on the analysis criteria, reference metrics, or a comprehensive review process. Only a small number of studies inform on the specific positive/negative feedback received, as well as how they would apply the suggestions on later improvements [9, 13, 16, 17, 26, 32]. Exclusively negative or unsuccessful outcomes have not been found.

The path towards accessibility for autism is obviously not straightforward. Since the clinical literature is still limited, practitioners are also learning from empirical evidence. In this sense, one of the main challenges in developing serious games for autism is to reach out the end-user, in order to understand the complexity of conditions they have. In addition to that, the development process must be scrutinized. As showed in this systematic review, there are issues related to the selection of the target audience (which may lead to a poor definition of learning objectives), and the conduction of testing procedures (which may lead to biased or mixed results). Finally, developers must be aware of the user-centered approaches – involving autism advocates along the process may be a good strategy to prevent usability disasters.

The studies included in this review are undeniably valuable, and we believe the authors did their best with the limitations they might have faced and with the resources they had available at the time. Results achieved and presented in the studies reinforce that. Notwithstanding, scientific and methodological procedures must be taken seriously given the complexity of the subject matter. This review also intends to open future discussions on scientific rigor and integrity involving the development of ASD-specific software. Efforts must continue toward accessibility, effective learning, and social inclusion.