Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

1 Introduction

Cognitive assessment and screening is becoming a major challenge for the researchers to then adapt prevention and support of patients. For instance, to understand Mild Cognitive Impairment’s (MCI) difficulties in their everyday life, they can be assessed during the realization of Instrumental Action of Daily Living (IADL) where errors are analyzed and characterized [1, 2]. Belchior et al. [3] have listed 9 studies where MCI are assessed with IADL in real world situations. However, IADL can also be tested using virtual environment (VE) [4,5,6,7,8]. VEs are real time computer simulation in which users can navigate and interact with the use of specific 3D interaction technique [9, 10]. 3D VEs and 2D computerized systems are mostly used in cognitive assessment and screening in dementia or MCI because they offer the possibility to automatically record behaviours in a database and to propose standardized measures [11]. Thus, numerous variables can be monitoring very precisely (e.g. completion time) and collected data can be easily analyzed. Most of the recorded data are linked to participants performance and are related to the usability of the proposed interaction technique. 3D interaction techniques are mainly characterized by 3 terms: navigation, selection and system control [12]. Navigation allows the users to travel in the VE from places to places adjusting his/her view point (i.e. steering) [13] which is fundamental to explore the VE and to navigate toward the chosen direction [14, 15]. Navigation can be a goal itself or a way to managed a task, but in both cases, it has to be easy to use and to learn to reduce the cognitive resources associated with the motor component (i.e. travel) [16]. So that users can easily be focused on the cognitive component of navigation (i.e. wayfinding) [17]. The imbrication of the travel and wayfinding component foster the creation of a cognitive map of the environment. The cognitive map is a mental representation of spatial information needed to navigate and to orient oneself in space [18]. Thus, navigation data are linked to the action plan and to the interaction technique used for travelling. Using VE permits to progress in the understanding of navigation patterns thanks to the recording of behavioral data. VEs are useful to collect data but the related behavioral analyses are sometime difficult to carry. Indeed, a lot of variables can be selected for analyses and results may be difficult to understand. Even if it is admitted that the users performance is linked to the task completion time [19] others variables can be extracted and analyzed such as log or time passed in each place of the VE. Having a better understanding of each variables recorded in a VE will allow us to apprehend participant’s performance and adapt his support for instance by modified the interaction technique.

2 Related Work

2.1 Interaction Technique Used in Virtual Environment for Assessing Cognition

3D interaction techniques can be split in two categories according to the Jung et al.’s taxonomy [20]: egocentric and exocentric techniques. With exocentric techniques, the user’s point of view is outside the VE citePoupyrev1998. For instance, the use of Ray Casting [16, 21], mice or keyboard permits to easily interact with the VE. They are the most used because they are intuitive and easy to learn [20, 22]. In a scoping review of VE for neuropsychological assessment, Valladeres et al. [23] showed that 47.7% of their studies, use mice and keyboard to interact with the VE and 20.75% use a touchscreen and a few the joystick. Moreover, some of them does not provide any indication about the way to interact in the VE. Indeed, 8 studies assessing MCI participants in VE have been listed (cf. Table 1) and participants with MCI could navigated using mice, keyboard, touch screen, joystick or motion capture tools (Kinect and Leap Motion) which makes it difficult to compare users performance in the several studies. Motion capture tools allow users to interact with the VE through Naturel User Interface such as walk simulation (e.g. walking-in-place, walking on a treadmill). These techniques are ecological with gestures very closed to the reality but can make user tired because of physical involvement to interact with the VE [24]. According Kulik et al. [15] interaction technique does not have to mimic reality but have to efficient and may also be fun. So, it may be preferable to have less ecological technique but more efficient.

Table 1. Interaction technique used in VE designed for assessing MCI
Full size table

To understand the impact of interaction technique on the participants performance it is necessary to give an exhaustive description of the system setup and the interaction technique used. The view point orientation can miss in the system description whereas this characteristic is primordial. Indeed, when a participant is navigating in a VE he must be able to move easily, looking around him [28]. Moreover navigation must be very easy allowing the participant to completely focus on the tasks to be achieved within the VE [29].

The challenge of human-machine interaction studies is to developpe and evaluate interaction techniques with a high usability and to understand how cognition is involved in the use of these interaction technique. Indeed, studies have shown that elderly and MCI can be uncomfortable with the use of mice [30] and can feel anxious about using it [31]. However, if some elderly has difficulties with technology acceptance, increasingly are using computer for instance to keep social link thanks to e-mail [32]. The use of computer and associated interaction technique such as mice and pads, is becoming very common in work as in private context. Moreover, mice could be used for screening MCI. Seelye et al. [33] found, with mouse data monitoring during web searching, that MCI make more resting time between mouse movements. Taken together these studies indicates that elderly can use various interaction technique and that navigational data could help to qualify users performance.

2.2 Cognitive Assessment in Virtual Environments

The use of VEs to assess cognitive functions specify to pick up variables used to score the performance of the participant. Indeed, these new tests recquire to readjust the scoring system from traditional tests especially in IADL tests. VEs permit to record a lot of variables but investigations are still needed to detect which variables are sensitive to correctly qualify the performance. Moreover, the recorded data are related both cognition and the use of interaction techniques. Indeed, the main variables used to qualify performance are completion time and number of error which are modulated by navigation efficiency in the VE. Navigation efficiency is mainly assess by completion time and so, linked to the user’s performance [19] where a long completion time is associated to a poor performance. These results are corroborated with some studies where MCI took significantly more time to complete the task than experimental participants [5,6,7]. For example, the navigation patterns can be shown with the Visualization of Users Flow (VU-Flow), a tool [34] which allow to analyze navigation patterns in 3D VE. VU-Flow records users position and orientation in logs so users path can be visualized as well as most visited routes. The path visualization leads to descriptive and quantitative data of navigational efficiency with fluid or saccadic paths and distance travelled [19]. Even if tools exist to visualize navigation data, it is still difficult to understand why patients took more time to realize the task. Is it because they are inefficient in the task realization (i.e. cognitive difficulties) or because the use of the interaction technique?

Variables used to assess participants performance in the 8 VE (cf. Table 1) were listed. Two methods can be used to qualify performance: created an efficiency ratio which is the compilation of several variables [8, 25] or analyzed variables one by one where most variables are time completion and number of errors. Others collected data exist as number and rest periods [4, 5] or distance travelled [4,5,6]. So, variables used in VE to qualify participants performance are related to quantitative navigation data. Mainly because monitoring data are linked to the tracking of participants position. Even if these data are difficult to appreciate because they are related to the use of the interaction technique, they can be reliable to qualify participants performance. For instance, Zygouris et al. [6] proposed an algorithm to detect MCI participant from elderly according time completion and age of the participants where a longer completion time is a sensitive variable to detect participants with MCI.

The present studys goal is to apprehend participants performance focusing on navigational data in a 3D EV. Results could help to have a better understanding of elderly performance and later compared them to MCI performance.

3 Method

The study was conducted with 13 young adults (4 men and 9 women) and 13 elderly (3 men and 10 women). Young participants (mean age: 25.16, SD: 6.14) were recruited in general population and elderly (mean age: 67.09, SD: 2.91) at the Alzheimer association. All participants are French speaker and have the same year of scolarity (mean 16.12 years for young adults and mean 15.25 years for elderly). Older with a MoCA less than 26 were excluded on this study. The participants realized IADL tasks during multitasking in the Virtual Multitasking Task-2 (VMT-2). The VMT-2 is a virtual apartment where prospective memory and executive functions are assessed during IADL. All trials begin with a familiarization stage where participants could visit the VMT-2, displayed on a full HD 27” monitor, without tasks to manage using mouse and keyboard and head-mounted display (HMD) (i.e. Oculus Rift). The goal of this preliminary step is to allow participants creating a cognitive map of the virtual apartment and to learn how to navigate select 3D objects (e.g. open/close a door). Then, all participants realized VMT-2 while navigation data and action were recorded in a csv database. Participants position was monitored in the x, z and y axis and transcribed on the apartment map. Collisions are marked using a red triangle on the map each time the user touch a 3D object such as a wall or a furniture. Total completion time and the time passed in each room of the virtual apartment were also recorded. Each participants had to perform the familiarization step in both a non-immersive and an immersive configuration. In the former case, the participants hat to navigate using mouse (travel) and keyboard (steering and selection), in the later case they had to navigate using the keyboard (travel), the HMD (steering) and the mouse (selection).

When the participant begins the VMT-2, he knows that he must answer to the phone if this one is ringing and to store the grocery. One minute after the beginning the phone rings and a voice asks the user to go to the main bedroom to feed the fishes and to dry on a shirt. Two minutes after the beginning, the voice on the phone asks to fax a document. Finally, after 15 min it asks to check if the tickets are properly placed on a shelf. The experiment ends when the participant think he has finished the tasks.

4 Results

All analyses were conducted using R software, at a significance level of 0.05. T modified was used to qualify errancies and U Mann-Whitney to compare elderly from young adult group.

4.1 Descriptive Analysis

Descriptive analysis was conduct on the map of the VE. We compared path with mouse and keyboard and path with HMD during familiarization stage where participants could visit the virtual apartment. Trajectory of adults and elderly present the same particularity when they navigate with the HMD (Fig. 1). Indeed, the path is more saccadic with right angle where it is smoother when participants use the mouse and keyboard to navigate.

Fig. 1.
figure 1

Path of adults (a) and elderly (b) during familiarization step with mouse and keyboard and then with HMD.

Full size image

4.2 Quantitative Analysis

Time Passed in Each Zone. Time passed in each zone was calculated to identified errancies. Errancy can be defined as a long time passed in one room including goings and comings without apparent goal. To qualify errancies, T modified test was conduct so that participant resting significantly more time than others in a room was doing errancy. There is no significantly difference of errancies between the 2 groups (U = 83; p = 0.47) or between immersion conditions (i.e. HMD and non-immersive) (U = 41; p = 0.96).

Completion Time. Elderly group took significantly more time to achieve the VMT-2 (U = 126; p = 0.009). We did not observe any effect of immersion (U = 82; p = 0.40).

Performance’s Rate. To complete the VMT-2, elderly group took significantly more time to achieve the tasks in the VMT-2 (U = 126; p = 0.009). In addition, young adults realized more tasks than elderly (U = 32; p = 0.05). To know more about actions plan we analyzed logs recorded by the system and create activity diagram for adults(Fig. 2) and elderly (Fig. 3). In a first step, participants come in the apartment and go toward the kitchen to store the groceries. Only a few young adults begin to store the groceries when the phone is ringing to ask them to feed the fishes and dry the shirt which are in the main bedroom. Most young adults go in the main bedroom to do it and then go to the kitchen before answering on the second phone call whereas only half of elderly has the time to go in the bedroom or in the kitchen to begin some activities. The second phone call specify to go to the office to fax a document. During this time adults and elderly realize activities in a similar way. Indeed, they fax the document, feed the fishes, deposit the shirt if it is dry and continue or begin to store the grocery. It should be noted that half of the young adult group does not answer to the third phone call because these participants have already finish all activities in the apartment and so, stop the VMT-2. Others answer to the third phone call and go the entrance to check the tickets. At this point, if they finish all tasks they stop the VMT-2 or go back in the bedroom for finishing the shirt task and then go the kitchen for storing the last items before stopping the test. Focusing on the elderly group only 2 participants go to check the tickets. Others finish the tasks in the office or in the bedroom and then in the kitchen.

Fig. 2.
figure 2

Diagramm of the action realisation in the VMT-2 by adults. N = mean of participant. T = mean time.

Full size image
Fig. 3.
figure 3

Diagramm of the action realisation in the VMT-2 by elderly. N = mean of participant. T = mean time.

Full size image

5 Discussion

This study investigated navigation patterns of elderly and young adults while performing multitasking in a virtual apartment, by analyzing data such as completion time. In this context, navigation through each room has been analyzed using a diagram. When participants begin task they know they have to go in the kitchen to store the groceries but phone calls interrupted them during their first action. The call ask to realize others tasks in several rooms of the virtual apartment. The addition of successive tasks will distract the subject but also ask him to locate himself in space to be as efficient as possible in carrying out the tasks. To do it and complete the VMT-2, young adults took significantly less time to complete the tasks than elderly. It is now to understand why and where elderly need more time and what difficulties did they encounter.

Elderly took more time to realize the tasks in the VMT-2 which is congruous with previous studies where older participants had higher time completion to manage navigation tasks [10]. In their study, Sayers et al. [10] proposed some navigation aids to increase participant’s spatial awarness. However, differences between young adults and elderly were maintain. Elderly may have issues with the use of the interaction technique itself and not only with space awareness. Indeed, participants begin VMT-2 with a familiarization stage where they can visit the VE once with HMD and once with mouse and keyboard. Before being assessed in VE, participants generally begin with a familiarization step where they can learn how to interact with the VE. Being familiar with the VE is also often required for a successful navigation [10]. The recorded data during these early stages could be a reference for the participant’s skill using the interaction technique to classify them according the user’s abilities such as expert or novice. Indeed, during the familiarization step, paths with mouse are less saccadic than paths with HMD for both adults and elderly and participants made more collisions with HDM during the experimentation stage. When using the HMD, participants have non-continuous movements. They stop travelling, then look in the wanted direction (i.e. steering) and move on again. This lake of spontaneity is shown by right angles marked on the virtual apartments map (Fig. 1). Moreover, most of participants decided to travel with mouse and keyboard during the experimentation because they felt uncomfortable using the HMD. Interaction technique can modulated the participant’s performance especially in the case where steering and travel are two different processus. Indeed, if travel is the same in both condition, steering is doing by mouse or by movements of the head, with changing orientation point of view. Steering with head movements is not intuitive for participant without experience in immersive VE and could lead to higher cognitive load for HMD users.

Aging is associate with cerebral change that may cause mild decline in attention, executive functions and working memory [35]. These declines may make elderly slower at performing tasks. As expected, elderly are less efficient than adults in the realization of tasks in VE. Indeed, they took more time to initiate tasks in the VMT-2 and half of elderly begin activities only after the second phone call. During the first and the second phone call, they stay in the same zone and seem to use the time to think. As they need more time to initiate activities in the end of the task the had several tasks to finish during the same time, so they are doing multitasking to manage all tasks. The multitasking context is present in IADL and is linked to independent living at home [36]. It requires several cognitive functions as executive functions, prospective and retrospective memory [37,38,39] and can lead to a higher cognitive load and difficulties to remember all the things to do. Indeed, most of elderly forget to realize the last task which is to check the tickets. Moreover, the multitasking context with unexpected events could lead to errancies for adults and elderly. During this time, they may think about the way to managed the next tasks. Indeed, when checking logs, we recorded any activities of the participant during errancy. However, we also observe another type of long time passed in a zone especially in the corridor or in the kitchen. According logs, in these room, participant had issue with item selection and orientation in space in these restricted places.

Taking together these results indicate that the elaboration of a EV for assessing cognitive functions is not an easy thing. Indeed, results showed that elerdly realized tasks in an inefficient way because phone calls were unexpected and disturbe participants but also because they need more time to be ready to act in the VE. In the VMT-2 the first phone call seems to occurs too early because the participants do not have the time to go in the kitchen to store the groceries. Moreover, adult participants finish the task quickly and the third phone call did not ring. Start actions in VE as function of time could lead to difficulties for participants to managed all the task. Actions could be launch not by time but by others actions. For instance, the first phone call ring after participant stored some grocery item instead of 1 min after the beginning of the VMT-2. A longer completion time is associated with cognition but also with the interaction technique. Analyze several variables instead of having a performances rate helps to make a distinction between cognition and abilities with the interaction technique. Understand how which part impacts more completion time could be make with analysis of collisions, time passed in each zone and logs. Collisions could be a quantitative indicator of usability of interaction technique and logs visualized with a diagramm could be a better indicator of participant’s cognition. Analyze logs to construct diagram aims to be a useful method which allow to see where and what kind of difficulties the participant met.

As limitation, the study has not enough participant using the HMD. Complementary analyses should be conduct with more participants wearing the HMD. To avoid cybersickness with elderly the comparison could be done only with adults.