1 Introduction

With the increase of age, the physical and mental health of human beings can be affected, resulting in various impairments, which make it difficult to perform the daily activities of older people [24, 25]. These impairments are grouped in visual, auditory, cognitive and movement problems. Cognitive impairment refers to the change that occurs in the way we process information, which creates difficulties for understanding data, difficulty in memorizing and making decisions, loss of skills such as perceptual speed, the ability to work memory, focus attention on something specific, reasoning and spatial thinking [24].

When a person has difficulty performing tasks such as map navigation, displacement, and/or positioning; problems to locate in the environment with respect to different objects; and is not able to solve problems of rotation of images in his mind, it is possible that he suffers from a difficulty in spatial thinking and may be a sign of dementia [20]. There are neuropsychological tests that allow evaluating the cognitive abilities of the patient.

Some neuropsychological tests have been virtualized due to the need that neuropsychologists have expressed, to have efficient and low-cost tools that are easy to use, practical, portable and with a design adapted to the needs of patients [23]. For example, the psychological tests of Poppelreuter and Raven, which help to analyze cognitive abilities and mental illnesses such as visual agnosia [3]. There is also the word-learning test, which evaluates the memory of older adults and with the results obtained, the patient’s memory curve can be generated [21]. The pictogram is a test that studies the memory and intellectual process of patients [22]. To carry out these tests, the haptic and voice interaction modes were used. The haptic is especially useful for making drawings or tasks in which it is necessary for the user to touch the screen. The voice mode was used to answer questionnaires.

Some tests for adults have been implemented in an augmented reality (AR) environment, mainly with augmented reality lenses [28]. These tests are aimed at improving the motor capacity of patients. Augmented reality is a technology that integrates images of virtual objects into images of the real world [12]. The essential elements that compose it are a label that identifies objects in a unique way, an image capturing element, which can be a cell phone camera and software that processes the images taken by the camera and superimposes the virtual objects [11]. To deploy and manipulate the RA, various devices are used, such as the augmented reality hulls [28], the CAVE [9], the computers [26], the volumetric projections [15], the kinect [8] and the mobile devices as the tablets [2, 22, 29]. The augmented reality, comparing them with traditional treatments, has certain advantages. To begin with, they can not cause physical harm to the patient. In addition, the behavior of virtual objects can be controlled to generate the desired stimulation in the user. Through augmented reality, you can recreate scenes that could hardly be generated without the help of a virtual environment [16].

2 Related Work

Several reported works show that software has been developed in mobile devices to perform neuropsychology tests in older adults [4, 14, 22]. To develop mobile applications for older adults should consider aspects of interface design, as well as selection of the appropriate interaction modality for the population that will use it. For augmented reality these aspects have not been studied, however, Yoo et al. [28] present tests with older adults using augmented reality lenses.

2.1 Neuropsychological Tests on Mobile Devices

With the aim of providing efficient and effective tools for the detection of cognitive problems, various mobile applications have been developed to perform neuropsychological tests, such as the following:

Test of the Drawing of the Clock [27]: This test evaluates some skills such as understanding, planning, visual memory and image reconstruction, spatial ability, numerical knowledge, abstract thinking, concentration and tolerance to frustration. It consists of giving the patient a mobile device, in which there are four circles previously drawn and asked to denote 4 different schedules. You are given a point if you correctly put the numbers and one point by correctly drawing the hour hand and another by the minute hand. Three points for each clock, giving a maximum of 12 points throughout the test.

Eurotest [5]: The objective of this test is to measure the memory and the calculation capacity of people. The affectation of these cognitive capacities are precedents in the development of dementia. It was created in 2004 in Spain. It has the advantage that the educational level of the people to whom it is applied does not matter. To apply this test, the patient is first indicated the denomination of the coins and bills with which they will work, then certain calculations are made with the money and finally the patient answers some questions using a smart cell phone. The questions that are asked to the patient are calculations with the coins.

Phototest [6]: This test evaluates visual memory, and the association of memory and verbal fluency. It was developed in Spain in 2007. Its main advantage is that it can be applied to people who can not read or write. The test is an application that shows six photographs of objects common to the patient. Then he is asked to name names of people of the opposite sex and later of people of the same sex for 30 s. Finally, the memory of the photographs shown at the beginning is evaluated. If there are problems to remember, the patient will be helped with a semantic key.

2.2 Mobile Applications for Older Adults

Several research works have been developed to implement Luria’s neuropsychological tests on mobile devices. The investigations are from the analysis, design of interfaces and usability tests aimed at the elderly. That is, emphasis has been placed on interaction with older adults and the applications of Luria’s tests.

Analysis of Luria Memory Tests for Development on Mobile Devices [22]: An analysis of the various mobile devices and of the different Luria tests is presented. The main objective of the work is to determine which can be implemented as applications. The result is an analysis of the tests considering that the ideal mobile device is the Tablet.

Memory Test “Word Learning” Implemented in Tablets for Older Adults [14]: In this paper we present the virtualization of the memory test, which has implemented several combinations of modalities for the realization of the test in tablets. For the word learning test the patient is shown multiple words or numerical figures not united, the series consists of ten or twelve words or numerical digits. The patient is then asked to repeat the series in any order.

Distributed User Interfaces for Luria’s Tests for Older Adults [4]: Luria’s tests can be used by mental specialists to analyze mental deterioration in older adults. This work presents a graphical user interface distributed for three Luria tests, in order to be used by older adults. The purpose was to create an interface based on the user’s ability to handle secondary screens.

Distributed User Interfaces for Poppelreuters and Raven Visual Tests [3]: The Poppelreuter and Raven tests are used by psychologists to analyze cognitive abilities, mental illnesses such as visual agnosia and even dementia syndromes such as Alzheimer’s. It was proposed to use a distributed user interface, using a tablet and a smartTV, to support users with visual problems to do Poppelreuter and Raven tests. The use of these tests on mobile devices could help patients who can not easily visit the consulting rooms and the specialist perform the battery tests remotely.

2.3 Design and Interaction Modalities for Older Adults in Mobile Devices

To design interfaces oriented to health applications for older adults, lines of design and analysis of tests were created, such as neuropsychological, as will be seen below.

Design for Mobile Phones Focused on Older Adults [18]: The use of mobile devices by older adults has several problems, as well as many benefits. Elderly people prefer mobile apps that help them remember, like the diary, the diary and the alarm clock. With these requirements, that created design lines with which, to help the creation of applications that help in the daily tasks of the elderly.

Evaluation of Tactile Interaction for Older Adults [17]: Smartphones with touch interfaces are increasingly used, however, older adults have problems using mobile applications, due to the poor designs they have. To prove it, people older than 65 years, performed various tasks how to drag, pinch, use keyboard and photo viewers. The results showed adversities of usability.

Design Strategies for Tactile Interaction Oriented to Older Adults [1]: An analysis was carried out that led to the creation of lines and design recommendations contemplating tactile interaction for seniors, in order to facilitate the development of interfaces in mobile devices. These recommendations were classified into: “Touch and feel”, “Interact” and “Functionality”.

2.4 Augmented Reality for Older Adults

Augmented reality is being used in various areas, such as in medicine, where it helps improve people’s health, for example to older adults.

Augmented Reality for the Otago Tests for Improvement of Balance and Gait, as well as Reduction of Falls in 21 Older Adults [28]: A study was carried out that determines the effects of augmented reality in the performance of exercises that help the balance and gait of older adults. To work with augmented reality, they used augmented reality lenses and a camera computer. The result was the increase in speed, improvement of step length and stride, as well as reduction of falls in patients.

3 Interaction Modalities for Older Adults

The best interaction modality with which the user can manipulate the augmented reality software depends on several factors, such as the type of device on which one is working and the characteristics of the target population [19]. In the tablets the modalities of interaction by voice, tactile, based on vision, keyboard and haptic can be used, however, not all are necessarily apt to be used by the elderly with augmented reality [13]. To select the appropriate modality for the elderly, the deteriorations of the elderly must be taken into account, which are divided into 4 groups: Vision, hearing, cognition and movement [20].

The Yerkes test application is designed for use by seniors on tablets. For this, it was necessary to make an analysis [13] to determine which is the most appropriate interaction modality. Through the analysis it was determined that the most suitable modalities to implement in the mobile application are the following:

Based on Vision: This modality has been successfully tested on other devices, such as augmented reality lenses [28] and Kinect [9]. Although there is no proof of its usability in tablets, the research indicates that following certain parameters, such as giving real behavior to virtual objects, this interaction modality can be used by older adults in augmented reality [13].

Voice: This modality has been tested on mobile devices with older adults [14], but not in virtual environments. The voice modality is recommended for patients with vision and movement problems, and presents problems in people with auditory and cognitive problems. The analysis indicates that there are a series of criteria that must be followed so that this modality is suitable for all the physical and cognitive problems of the elderly. These criteria include that the voice be issued at 85 db or greater and that the instructions consist of 4 options or less [24].

Fig. 1.
figure 1

Examples of figures that must be shaped in the Yerkes test.

Full size image

Haptic: This is the most recommended modality, due to the benefits that have been observed in the different tests performed by patients on mobile devices [1, 17]. The haptic modality is direct and provides feedback. Its usability with the problems of vision, hearing and cognition, seems to be very high, however, it also depends on the design of the interface that the mobile application has. This modality is not recommended for people with movement problems.

4 Mobile App Design for Yerkes Test

The usability of a mobile application depends on several factors, one of the main ones is the design of the user interface. A poorly designed user interface may cause the user not to properly view the objects, the user does not understand the use of the application or the application may fall into disuse because it is not efficient, among others.

To make an adequate design of the user interface, it is important to know the population to which the mobile application is aimed and the purpose of this [19, 24]. The main objective in this article, is to present the Yerkes test actually increases on a tablet. Due to the fact that the users to whom this application is addressed are elderly people, the possible deterioration of the people in this group should be considered.

4.1 Yerkes Test

The Yerkes test was created by Robert Mearns Yerkes. This test consists of showing the patient the image of a figure composed of small cubes. Subsequently, the patient is given cubes to the patient to form the figure he visualized (Fig. 1). The difficulty of this test is that patients can not always identify how and by how many cubes the figure is composed. With this test, constructive capacity, mathematical ability and spatial thinking can be evaluated [20].

Currently, we have the necessary tools to apply the Yerkes test, in an augmented reality environment and in tablets. To achieve this, it is necessary to define the interaction modalities that are suitable for moving virtual objects, as well as the design of the user interface that the mobile application will have.

4.2 Selection of Interaction Modalities

For this application, three interaction modalities will be used, in order to test them in seniors and check their usability for augmented reality. The first is the haptic interaction, with which users can touch the virtual object directly on the screen. Voice interaction will also be used, so that users can provide instructions to virtual cubes through speech, such as “left”, “right”, “up” or “down”. Finally, the vision-based modality is contemplated, it is possible to implement it through virtual buttons, which means that certain parts of the marker can be touched so that the user manipulates the figure.

4.3 User Interface Design Considerations

When knowing the interaction modalities, the design of the user interface must be carried out. This step is important as it determines the navigation within the application. The impairments in the elderly that affect the interaction with mobile applications are mainly vision, hearing, and mobility. Therefore, the design of the interface must be able to offer an appropriate interaction with the users, anticipating the possible deterioration they may have. People receive information mainly using the sense of sight, so designing the interface for people with visual problems is especially sensitive. Fortunately, studies have been conducted in this regard, which have provided guidelines to facilitate this task. The main objects must be highlighted so that the user can easily locate them [10, 24]. The main objects should be large and should use contrasted colors or representative figures, in the case of those users who are not able to distinguish and differentiate colors. It is recommended that the Gestalt’s laws be considered, since they refer to the perception and visualization of objects [24]. For text, the font will be Arial of 12 points or greater, black with a white background [7]. The ear is one of the senses ignored when designing the interfaces. The ear is one of the senses ignored when designing the interfaces. However, the use of interactions by ear is especially useful when people have visual and movement problems. With this sense, users can receive instructions or indications of the application. For this reason it is important that when implementing the interaction by hearing, users are allowed to adjust the desired volume [7], in this aspect it is important to consider that older adults may require 90 dB or more [24]. It is recommended to use a female voice for instructions issued by the interface, in addition to instructions that must be concise and brief [10, 24]. For the cases in which the instructions are of option, it is recommended to have at most four options [24]. To provide feedback of the actions performed by the user, sounds can be issued and these will have to be short and paused [24].

The design of user interfaces for patients with cognitive problems can be very complicated, because when a user can see perfectly the objects on the screen, but can not understand the task at hand, he usually blames himself. To avoid this kind of errors, mobile applications should consider that interaction by hearing is not recommended with users with acute cognitive problems [24]. In applications with augmented reality, it is important that the objects have the behavior of an actual object [24].

People express their thoughts through movement, converting them into actions that generate responses in the systems. Therefore, the design of the mobile application, follows certain parameters, with the intention of people who have movement problems, can use the system more easily. The recommendations are as follows: The objectives placed on the screen should not be too many. The objectives must be large, so that it is easy to touch them. Do not implement tasks in which it is necessary to use the “double click”, and when users do this action, may not give the desired result. When using the vision-based modality, the dragging of objects must be avoided, and the interaction must be done through the use of virtual buttons [1, 24]. Feedback must be given to the user, regarding the action they are taking [24].

5 Prototype

In this section we will explain the user interface design of the augmented reality mobile application for the simulation of the Yerkes test. The application was divided in three sections. The first is for user registration, the second is to select the interaction modality with which the user will work, the last is the simulation of the Yerkes test.

5.1 User Registration

The mobile application can be used in several mobile platforms, so the patient records will be stored in a database, where these can be consulted whenever needed. To sort the data and differentiate them, the registry will have 2 fields: Name and date of birth, because this is information that the patient has memorized throughout his life and not all users would be able to memorize new data, such as passwords, NIPs and user names, among others. The tasks necessary to carry out the registration are segmented so that the user concentrates in one at a time. Figure 2 shows the activity in which the user enters their full name and the activity in which the patient selects their date of birth.

Fig. 2.
figure 2

Section to enter name and date of birth.

Full size image

5.2 Modality Selection

Not all patients who use the mobile application will have the same disabilities, so it is important to have more than one interaction modality. In this version, the application consider that when the user will performing this test. The user should be accompanied all time by the specialist, so that the neuropsychologist help the patient to select the most appropriate interaction modality. In Sect. 4.2 the modalities were selected, which are haptic, voice and vision-based. In Fig. 3, is shown the activity to select the interaction mode, in which it can be noted that each of the modality options is presented with a text and an image to be easily identified. When the user touch either of these two views, both change to red and the modality is selected.

Fig. 3.
figure 3

Activity to select the interaction modality.

Full size image

5.3 Yerkes Test

The activity for the Yerkes test in augmented reality for tablets, consists of two sections, the first is the modeling area and in the second the user can select any of the various options of a menu, such as the display of instructions, add virtual cubes and exit the mobile application, among others.

5.4 Menu

It is important that the user has a lot of space to form the requested figure, so the menu will be hidden, unless the user makes a slide on the left border of the screen (Fig. 4). The menu will have the following options:

Fig. 4.
figure 4

Yerkes test activity with the menu.

Full size image

“Exit” Button: When pressed, all the user’s data is sent to the database, including the time it took for the patient to perform the test and the figure that the user has made.

Instructions: The user will have instructions written, so he can consult them whenever he want. This helps to the patient not have to force his working memory.

Image Icon: Pressing it will maximize the image that shows the user the figure that must shape. A caption will also be displayed to close the image and return to the Yerkes test activity, without pressing any other button.

Cube Generator: In this section the cubes that the user needs to shape the figure will be generated.

5.5 Modeling Area

In this section, the patient can shape the figure that he have been required with the virtual cubes, using the interaction modality that has been selected. This area will be the one that will use the augmented reality, so it will be visualized what the camera captures, which ideally should be the marker, so the virtual objects begin to unfold (Fig. 5).

Fig. 5.
figure 5

Yerkes test display without menu.

Full size image

6 Conclusion

The technologies of mobile devices make it feasible to develop the augmented reality application for the Yerkes test. They also allow to incorporate different interaction modalities. However, there is still a need to validate the test with specialists and perform usability tests with older adults. A latent worry is the vision-based modality because it can cause fatigue in the users. In the haptic modality must be careful with the response resolution of gestures on the touch screen. Finally, the voice interaction modality can facilitate interaction when the user has learned the commands to perform the test. It should be noted that the test is about the ability of spatial thinking, not movement, so that the selected modalities present a good segment of opportunity for interaction with older adults. Although there are different RA technologies where the Yerkes test could be implemented, the tablets are an appropriate option because they are economical and have a greater scope for the population that needs to perform these tests. This has a direct benefit in economics and attention to specialists and patients.