Keywords

1 Introduction

The analysis of the movements of human users interacting with touchscreen devices have been used to study the ergonomics of devices and graphical interfaces. By recording the user’s body postures and positions, it is possible to compare different situation of use of touchscreen devices [1] or interaction techniques [2], in order to identify postures that could cause lack of comfort or even musculoskeletal disorders [3]. Besides, the posture of the users’ body in relation to the position of the devices can affect the performances of the users during the execution of interaction tasks [4]. For this reason, designers should also consider the analysis of the interaction with technologies from an ergonomics approach.

In view of the adoption of touchscreen devices by older aged users [5], it is important to consider the comfort of use of these technologies. The age-related changings in muscle, bones and articular cartilage may lead to a functional loss, affecting the movements the users need to perform during tactile interaction [6]. The analysis of the movements of older-aged users during interaction with touchscreen devices should provide some evidence of the difficulties this group of users may find when using these devices [7].

We implemented an experimental study associating the analysis of the movements and performances of fifteen older aged adults (65–84). The task consisted on positioning twenty-five targets with drag-and-drop gestures using their index finger, on a tablet. The movements of the participants were recorded through a motion capture system, providing data for the analysis of the postures of their bodies and articulatory angles.

In this paper, we present a preliminary analysis of the movements of the users’ wrist. Previous studies demonstrate a great mobilization of the users’ wrist during tactile interaction [8, 9]. In this study, the movements of the wrist represent the arrangement of the articulations of the users’ upper limbs as the users move to execute the interaction task.

Results present the analysis of the articulatory angles we registered and a discussion about their consequences on the comfort of use of touchscreen devices. Additionally, the analysis of the performances of the participants for the interaction task discusses the possibility of the effects of ergonomics on the usability and accessibility of touchscreen interaction.

Finally, this paper discusses the use of movement analysis (A) for understanding the differences in performances for users with different functional capacities and the impact on accessibility of interactive technologies and (B) for identifying postures and positions of the users’ bodies that could present a risk for developing musculoskeletal injuries for older and younger users.

2 State of the Art

In this session, we review some previous studies using motion capture systems to analyze the movements of the users during the use of touchscreen devices, most of them to evaluate the positions and postures of the body from an ergonomic point of view. Then, we discuss the mobilization of the user’s wrist during interaction with touchscreen, in order to justify our choice for this articulation on the preliminary analysis we present in this study.

2.1 Movement Analysis During Use of Touchscreen Devices

The analysis of the movements of human users interacting with technologies consists on recording the user’s body postures and positions during the execution of an interaction task. Mobile devices present the advantage of being adapted to different situations of use, at home, office or mobility. Consequently, we consider that mobile devices are particularly suitable for older aged users in regard of their possibilities of use at home, in institutions and in mobility.

However, it is difficult to provide ergonomic recommendations that could take into account the context of use of mobile devices. Besides, mobile devices are often equipped with touchscreen, which has being shown to be easier to learn and use for older aged users [7]. In order to evaluate the comfort of use of mobile devices and touchscreen interaction, the analysis of the users’ movements seems fundamental.

Previous studies have used motion capture systems to study the ergonomics of devices and graphical interfaces for younger and adult users. Pereira et al. [2] compared pen and finger interaction for participants holding mobile devices at their hand. Bachynski et al. [1] evaluated and compared different screen sizes and situations of use of touch sensitive surfaces. In their study, the use of motion capture system allowed the definition of the groups of muscles and articulations participants employed during interaction. Additionally, data collected through the interactive systems can provide information about the users’ performances, helping designers to compare different devices [10]. However, the studies we reviewed included only young participants. The ergonomics of touchscreen interaction for older aged users is yet to be investigated.

More than identifying the postures that could cause lack of comfort, the posture of the users’ body in relation to the position of the devices can affect the performances of the users during the execution of interaction tasks. Pereira et al. [2] showed that users holding tablets with one hand should prefer small or median devices, so to improve usability and ergonomics. The association of the analysis of the users’ movements and the usability of devices is necessary to provide recommendations for users and designers.

2.2 Mobilization of the User’s Wrist During Interaction with Technologies

Studies that evaluated the movements of adult users during the use of touchscreen devices demonstrated a great mobilization of the users’ wrist during tactile interaction. Young et al. [9] showed that wrist postures are affected by the position of the devices. In their study, they discuss that using cases for holding tablets tilted requires increased extension of the wrist. Jacquier-Bret et al. [8] demonstrated that adults adopt different strategies for accomplishing an interaction task, adapting the movements of their upper-limbs when using touchscreen devices horizontally placed on a desk. According to their analysis, some users present small movements of the wrist but increased mobilization of the elbow and arms, while others present little movements of elbow and arms but compensatory mobilization of the wrist, for allowing the fingers to reach to the targets presented on the touchscreen. Both studies alarmed us about the risks of musculoskeletal injuries for intensive or prolonged use of touchscreen interaction.

During touch interaction, users would try to optimize their performances, increasing velocities for example. An intensive use of touchscreen would be related to repetitiveness, which could also present a risk for musculoskeletal disorders [11]. An advantage of touchscreen devices is the sensitivity of the screen, that would reduce force of touch compared to traditional physical input devices [12]. However, wrists disorders are also related to the angle deviations and amplitude of movements of this articulation [11, 13]. It is though important to consider that the movements of the wrist articulation are affected by the natural aging process [14]. For this reason, evaluating postures and movements of the wrist can provide relevant information about the comfort of use of touchscreen.

3 Experiment and Equipment

Fifteen older adults, aged 65 to 84, were recruited for this study. Their mean hands’ length was 18 cm (SD = 1.3) and mean index finger length was 8 cm (SD = 0.6). According to a pre-evaluation form and practice trials for the task, participants do not presented any motor, visual or cognitive loss that could hinder the interaction with touchscreen. They were right-handed. They were familiar to the use of computers and touchscreen devices.

The task consisted on positioning twenty-five targets with drag-and-drop gestures on a tablet (Samsung Galaxy Note 10.1 inches screen). The interactive system was set to display nine large and sixteen small targets and their sizes were: 46 × 35 mm for large and 35 × 27 mm for small targets on the tablet.

Participants were seated and the tablet was horizontally placed on the desk in front of them. The top of the device was 30 cm from the border of the desk. Participants executed three iterations of the task. They used their index finger to interact with the system. Figure 1 illustrates the set of the experiment.

Fig. 1.
figure 1

Experimental set

Full size image

The postures of the users have been captured through an optoelectronic motion capture system (Qualisys AB, Gothenburg, Sweden). This motion capture system was chosen because it is not invasive and do not disturb the movements of the users during the execution of the task. Four reflective markers were placed at the users’ right hand and forearm: metacarpi 2 (MCP2) and 5 (MCP5), radial (RSP) and ulnar styloid process (UCP). The markers’ positions were registered in three dimensions (X, Y, Z) at sampling rate of 200 Hz, tracking of the movements of the users’ wrists.

4 Data Analysis

At total, 1125 gestures were registered (15 participants × 3 iteration series × 25 targets). Data from motion capture was synchronized with touch information registered from the interactive system (timestamp and coordinates).

In this paper, we present a preliminary analysis of the movements of the users’ wrist. The articular angles were calculated through the coordinates of the anatomical markers MCP2, MCP5, RSP and USP in relation to the coordinates of the device, as illustrated in Figs. 2 and 3. Articular angles vary from neutral position (0 degrees) to increased deviations (negative or positive angles, until 45 degrees or over). Radial deviation is positive and ulnar deviation is negative as illustrated on Fig. 2. Extension angles are positive and flexion angles are negative, as illustrated on Fig. 3.

Fig. 2.
figure 2

Dorsal view of the hand

Full size image
Fig. 3.
figure 3

Medio-lateral view of the hand

Full size image

Median deviation angles (minimal, mean and maximal deviations) and median amplitudes of movements have also been calculated for each subject at each situation of the task. Then, for the analysis of the users’ movements, we calculated the mean percent of time participants’ wrist spent on different postures during the execution of the task.

For the analysis of performances, we calculated the median time for positioning a target and the median number of errors from data recorded by the interactive system.

5 Results

5.1 Characterization of the Movements of the Users’ Wrist

We identified a predominant radial deviation (median 28°, one inter-quartile interval 16°) and extension of the wrist (median 7°, inter-quartile 12°). There was a great amplitude of movements on radial-ulnar deviation (median range 56°, inter-quartile 19) and on flexion-extension angles (37°, inter-quartile 19°). Details are described in Table 1.

Table 1. Minimal, mean and maximal radial deviation and extension angles (median values)
Full size table

Figure 4 describes the percent of time participants spent on radial or ulnar deviated postures of the wrist during the task. On average, radial deviation was most of the time characterized by an articular angle between 20° and 40° (52% of the time of interaction). Only 8% of the time the wrist was close to a neutral position (near 0°, between -5 and 5°). 28% of the time the radial deviation was inferior to 20° while 12% of the time there was a great radial deviation (superior to 40°). The users’ wrist assumed an ulnar deviated posture 7% of the time during interaction.

Fig. 4.
figure 4

Percent of time on ulnar (negative) or radial (positive) deviated postures of the wrist during interaction

Full size image

Figure 5 describes the percent of time participants spent on flexed or extended postures of the wrist during the task. On average, the users’ wrist were extended most of the time (61% of the time of interaction). Extension was characterized by small angles, between 5° and 15° during 45% of the time. The mobilization can be described by the time wrists spent close to a neutral position (32% of the time of interaction near 0°, between -5° and 5°) and then flexed (27% of the time). Great deviations were registered during 12% of the time, when flexion or extension angles were greater than 15°.

Fig. 5.
figure 5

Percent of time on flexed (negative) or extended (positive) postures of the wrist during interaction

Full size image

5.2 Performances

The median time spent for positioning one target was 4.4 s (inter-quartile 1.5 s). The median number of errors for positioning one target was 0.4 (inter-quartile 0.9).

For better understanding this results, we searched for possible relationships between movements of the wrist and performances. We found a small negative correlation between mean radial deviation and time (Spearman’s coefficient of correlation -0.25). The relationship between radial-ulnar deviation and number of errors (0.03) was not significant. We did not found any significant correlation between flexion-extension angles and time (-0.07) nor number of errors (-0.01).

6 Discussion

In the present study, we used a motion capture system to assess the movements of older adults during interaction with touchscreen in order to find the relationship between their movements and performances for an interaction task. In this preliminary analysis, we evaluated the time users’ wrist assumed deviations from neutral positions during interaction with finger on a tablet.

The time spent on different postures describes the movements of this articulation during the task. Median angles and amplitudes we observed demonstrate a predominant radial deviation and extension of the wrist with a great amplitude of movements (superior to 45° on radial-ulnar). These characteristics are related to the configuration of the experiment, showing that the use of touchscreen devices placed on a desk could be considered more comfortable for the wrist than using cases or holding the device on tilted angles as described in the literature [9].

Our analysis shows that participants spent more than a half of the time of interaction (52%) on fixed radial deviated postures of the wrist, oscillating between 20° and 40°. Concerning the flexion-extension angles, they variate between -10° and 10° about 64% of the time, which explains the small median extension angle. Despite the great amplitudes of movements, participants spent little time on increased deviation of the wrist. This is important to note because increased angle deviations could cause injuries or disorders of this articulation [11, 13]. Wrist disorders could hinder the abilities of older adults to execute the movements employed on touchscreen interaction.

The predominant radial deviation and extension with great amplitudes of movements could imply a lack of comfort for older adults after intensive or prolonged time of use of touchscreen interaction. Indeed, Dennerlein et al. [15] recommend users to do not hold the same configuration of use for too long in order to prevent discomfort.

The great amplitudes of movements of the wrist could also be related to the group of participants. Aging effects on psychomotor system can result on difficulties for users to execute or perceive the movements of the wrist [14], which could imply on bigger movements of this articulation for accomplishing the gestures of interaction. This hypothesis should be further investigated with another study, comparing older aged users with a younger group of people.

The values reported in this paper describes a general characterization for a group of older-aged adults. When evaluating users with different functional capacities, we would recommend to take into account the inter-individual differences such as minimal and maximal angular deviations of radial-ulnar or flexion-extension of the wrist. Further evaluations are needed for identifying postures and positions of the users’ bodies that could present a risk for developing musculoskeletal injuries for older and younger users.

Our study demonstrates that assessing the users’ movements during the evaluation of interaction with technologies should also be considered in order to provide recommendations for users with different functional capacities or users with special needs. Indeed, the analysis of the movements of the users could be used in order to better understand the causes of their difficulties and provide appropriate assistance.

7 Conclusion

In the present study we presented a preliminary analysis of the movements of the users’ wrist during interaction with touchscreen. We demonstrated a predominant radial deviation and extension of this articulation and great amplitude of movements for older-aged adults. Participants executed drag-and-drop interaction with their index finger on a tablet, horizontally placed on a desk. The results we presented encourage the use of movement analysis for understanding the differences in performances for users with different skills and functional capacities.

Designers should consider the impact of ergonomics on the usability and accessibility of interactive technologies. In view of the adoption of touchscreen devices by older aged users, improve comfort of use could also determine a better user experience.