Keywords

1 Introduction

The number of people with dementia, especially Alzheimer’s disease (AD), is increasing and no cure has yet been found [6]. In addition to medicine-based therapies, also non-medicine-based therapies exist to slow down the progress of the disease. Current studies present a meaningful integration of tablet computers, such as iPads, into the therapy context with the aim of benefiting cognitive abilities or acting as trigger for the patient [2, 8]. Particularly mobile games show great potential to stimulate dementia patients [1, 4]. An advantage of mobile technology is that the hardware is affordable and easy to handle as no external input control (such as a mouse or a keyboard) is needed. Studies often choose children games and educational games for evaluation in the dementia context [1, 2]. Although these types of games seem to be easy to use, they still often require too complex interactions or result in a cognitive overload for people with dementia. A major problem is that most technologies are not designed for people with cognitive disabilities and therefore making operating the device very hard to use.

Many dementia patients, especially people over an age of 65 years, also suffer from visual and/or hearing impairment due to their advanced age. However, mobile applications integrate assistive technologies very infrequently. These applications need to fulfill special requirements to ease the usage for people who are cognitively impaired. Basically, designing for dementia patients means moving back in time and thinking of events that happened in the past of one individual and the respective generation [1]. Metaphors, which are frequently used in mobile applications, have to tie in with the level of knowledge, the experiences and the cultural background of earlier years. Required gestures for touch displays have to be simple or alternatively be explained in a tutorial-like style. As dementia patients tend to have almost no short-term memory, they might need a tutorial with every time they use the application.

In this paper, we present preliminary findings of interaction techniques in the tablet game Mindtraining in terms of ease of use and how good dementia patients are able to remember different kinds of gestures. The application serves as an interactive playful photo album, which shows various content on each page and integrates multiple different gestures, such as tap, drag and drop, and swipe. As there seems to be an absence of best-practice approaches for interaction techniques on tablet devices we focus on a mixture of gestures in our application. Intuitive interaction techniques could support a more independent usage of applications on tablets. Caretakers would benefit from this, as they cannot afford the time to provide technical support and training for every single patient. Already a small amount of additional workload for nurses can be an obstacle for using a beneficial serious game in a nursing home.

2 Related Work

A dementia disease can affect multiple cognitive abilities, such as memory, attention, language, reasoning, judgment, reading, and writing [9]. Recent research shows that video games with cognitive content have the potential of delaying the cognitive decline [1]. But only a few games were created for the target group dementia patients in mind, addressing their specific health restrictions.

In a research project, a mobile app was developed that offered personalized music based on the patient’s life, with a special focus on usability [7]. Music and photos also played an important role in a pilot study, where tablets were used in dementia therapy for over three months [1]. In this study, they proposed listening and singing to music or watching photos (animals, patients themselves, family, children, babies, etc.) and videos as initiating and ending practice in therapy. This inspires communication and affects the patient’s mood in a positive way. In the same study they mentioned that the caretaker had to give continuous positive feedback, especially at the end of the tablet therapy to motivate and encourage the patient. Games, which integrate continuous positive feedback and encouragement, would relieve the caretaker [1].

Software, especially games, for dementia is a well researched area and there are many recommendations available for developing dementia games [5, 8]. However, there seems to be a lack of guidelines for interaction techniques - especially regarding touch interfaces. Senior people might not have any prior knowledge of operating touch devices. Hence, performing different gestures in an application might be already too complex for people with dementia. The developed prototypes in past research projects integrate different interaction techniques, either input with mouse and keyboard, WiiMote controller, touch or ecological gestures. In conclusion, all researchers of the prototypes tried to find a simple and natural way to interact with the respective system to reduce the cognitive overload for people with dementia. Either touch interaction or ecological gestures seemed to be the most successful interaction paradigms. These do not require any external controller, such as a mouse or keyboard. In a study with mobile games and elderly people, the participants reported that it is much easier to touch the objects on the screen than using a mouse [10]. Learning how to interact with a software system is a complex task, in particular for users, who never used computer systems before. Yet, studies have shown that elderly people with dementia are willing to learn new technologies [3, 11]. In another study with dementia patients, half of the participants had never used a mobile device before. However, all of them indicated that they would like to see more technology and believed they could benefit from it [4]. Moreover tablets cover many positive features like high-quality responsiveness multi-touch screen, mobility, accessibility and ease of aquition [8].

3 Game Prototype Mindtraining

Based on related work, we established the requirements for Mindtraining and extended the application with different interaction techniques. The requirements include: (1) providing visual and auditory help; (2) matching the mental model with the conceptional model; (3) giving motivational messages and positive feedback when achieving a goal or making a mistake; (4) providing short and simple messages; and (5) counteracting the declining cognitive abilities of the persona [1, 3, 5]. Mindtraining is an interactive photo album with a playful integration of cognitive games. A photo album is an item, which is also known from the earlier years of nowadays dementia patients, hence it should present a good mental model for the players. Visual aids, especially photos, can stimulate the memories and are therefore a proofed concept in dementia therapy [1]. Playing the game requires different gestures and interaction techniques. In the album you have to swipe through the pages, tap on photos to see them in full screen and use tap for turning on and off a music radio. The radio plays songs of the earlier years of the patients and should serve as alternate stimulation of the user. Also a cognitive game with completing of sayings is implemented on multiple pages with two different scenarios: (1) tapping on the right answer, (2) dragging the right answer to a target location. Wordplays, including sayings, are widely used in mind training therapies for dementia patients to exercise their mind.

The system is complemented with a logging module which tracks the interaction of the user and logs the data in JSON format to files on the device. The module logs single tap, double tap, long press, swipe from left to right and vice versa, swipe from top to bottom and vice versa and dragging interactions of draggable elements. The logging is applied to the following view types of Android: TextView, Image, Button, ImageView and ImageButton.

Our objective is to investigate how people with dementia cope with different gestures in a mobile playful application. On the one hand it is interesting how good people can remember gestures and how intuitive they are. On the other hand we want to find out if one of the gestures might be easier to perform than another one and give recommendations of applying gestures in a mobile application.

4 Preliminary Study

The application was tested by eight participants, see Fig. 1 (age: 56–90, gender: 3 male, 5 female). Three of the participants have early-stage dementia, one has middle-stage dementia, the others don’t have dementia. The results of these tests were used to optimize the application and eradicate remaining usability problems of Mindtraining. The device used is a 10.1” sized ASUS MeMO Pad 10 Android tablet. A protection case covers the back of the device, which firstly gives additional grip and secondly provides a stable base to position it on a table in the right angle.

Fig. 1.
figure 1

Typical setting of the field tests: Subject playing the game, a carer sitting next to her and supporting her when questions arise and the observer in the background, taking notes.

Full size image

In total, four main iterations, each included planning, implementing, testing and evaluation, took place. Besides adapting sizes for fonts and graphics, two important issues were optimized during these iterations. At first people were instructed which gestures had to be performed, such as tapping on a photo or swiping for flipping a page. When it came to dementia patients, switching between the different gestures caused problems. As they had learned that swiping from right to left resulted in the intended outcome, they also tried swiping on photos and buttons. To solve the cognitive overload of switching between different gestures, simple animations for tapping and swiping were implemented at the desired positions to provide a visual instruction how to perform the gesture on the element. This optimization showed that dementia patients needed fewer instructions from the caretaker on how to interact with the different elements.

The second main usability problem, which had been observed, is that a single tap is often difficult to perform by elderly people including dementia patients. The main issues are sideslipping of the finger and the duration of the tap. Also the user sometimes touches the screen with the fingernails, which has no effect on a capacitive display. With Android’s standard functionality of clickable elements the action is only executed when performing a single tap on the element. To solve this issue we extended Android’s standard functionality for clickable elements in a way that these elements also react to gestures, such as swipe, double tap and long press. The logging module tracks, which gesture has been performed on an element. This will allow an analysis of the usage of gestures afterwards. Due to the single tap problem, the scenario with dragging the right answer to a target location in the sayings game, could be performed much faster and more fluently than single tapping on the right answer.

As next step, a field test with three participants with early-stage dementia with 75–90 years in age, will be conducted to observe the interactions. The test will take place in a small day-care facility where dementia patients get cared for three times a week. To examine which gestures on a tablet device are best suited for this target group, we integrated an automatic logging module in the final prototype. The logging module tracks how a user interacts with visible elements on the screen. Additional to the logging module, an observation protocol will serve as a documentation of qualitative data during the field test.

5 Conclusion and Future Work

In this paper we presented the purpose of serious games in the dementia context and introduced Mindtraining, an Android application that facilitates an intuitive usage of gestures. Software design guidelines for people with dementia are a common topic in research, but there seems to be a lack of recommendations when it comes to touch devices and their interaction techniques. In our application we are tracking and observing the user’s interaction with the tablet to find out which gestures can be performed better than others. The next step will be conducting a field test with dementia patients and analysing the used interactions in the application Mindtraining.