Keywords

1 Introduction

Multitasking is required in many jobs and the goal of multitasking is to perform multiple tasks simultaneously by switching between individual tasks [1, 2]. Previous research demonstrated different approaches for assessing user performance in a multitasking environment [3–5], but only few studies quantitatively analyzed multitasking performance. This study proposed a quantitative model to estimate user performance in a multitasking environment.

The modeling of human information processing has been contributing to human-computer interaction since the early 1950s [6–8]. The Hick-Hyman Law [6, 7] was built upon a systematic relationship between the number of alternate stimuli and choice-reaction times. The law predicts a linear relationship between reaction time and transmitted information. Fitts’ Law [8, 9] states a linear relationship between task difficulty and movement time. These human information processing models were widely applied in this area and originally applied to single-task scenarios. In 1948, Shannon [10] published a mathematical theory of communication, which laid the foundation of information theory and was employed in many areas including human-computer interaction. In this study, a quantitative model was proposed based on these human information processing models to estimate user performance quantitatively and improve their multitasking performance. The Multi-Attribute Task Battery II (MATB-II) was applied as a multitasking environment in this study.

The remainder of this paper is organized as follows: Sect. 2 introduces the proposed quantitative model and discusses a two-phase study demonstrating the application of the proposed model. Section 3 presents the results from the experiments while Sect. 4 explains and discusses the details of the findings. The last section discusses the implications of the proposed model and future research.

2 Methods

2.1 Quantitative Model and MATB-II Tasks

In this study, Multi-Attribute Task Battery II (MATB-II) was employed as a platform for multitasking [11]. Four subtasks were applied during the experiment: Light, Scale, Tracking, and Communication. Figure 1 shows the interface of MATB-II tasks.

Fig. 1.
figure 1

Multi-Attribute Task Battery II (MATB-II) tasks

Full size image

The Light task requires the user to respond to the absence of the green light and the presence of the red light (top left of Fig. 1). The Scale task requires the user to observe and detect a deviation of the moving scale from the midpoint (left, middle of Fig. 1). The Communication task plays audio messages with particular “callsigns” and requires the user to choose the announced channel and frequency (bottom left of Fig. 1). The Tracking task has two modes: manual and Automatic. When the task is under manual mode, the user needs to control a joystick with their left hand and keep the cursor in the center area (top, middle of Fig. 1). A Workload Rating Scale (WRS) is built into the MATB-II program and is based on the NASA Task Load Index [12]. The subjective workload assessed by WRS include six subscales: mental demand, physical demand, temporal demand, performance, effort, and frustration. The rating score ranges from 0 to 100 for each subscale. The questionnaire is presented after each trail.

To estimate the information produced by each subtask in MATB-II, baud rate (bit per second or bps) of each subtask is employed and defined based on Shannon’s information theory.

$$ {\text{B(i)}}\, = \,\frac{{{\text{H}}({\text{i}})}}{{\Delta {\text{T}}({\text{i}})}} $$
(1)

where H(i) is the total information produced by subtask i and \( \Delta {\text{T}}({\text{i}}) \) is the time interval of two events of subtask i. When the subtask i is Light, Scale, or Communication,

$$ {\text{H}}\left( {\text{i}} \right) \, = { \log }_{ 2} \left[ {{\text{p}}\left( {\text{i}} \right)} \right] $$
(2)

where p(i) is all possible events of subtask i. For MATB-II tasks of Light, Scale, and Communication, the events contained in each task have equal probability. For example, there are two events contained in the Light task (green and red lights) and each of them have a probability of occurrence of 0.5. Hence, all possible events of Light is p(Light) = 2 and the information produced in this task is H(Light) = log2[p(i)] = 1 bit.

When the subtask i is Tracking, which involves a circular cursor with diameter D and a target with width W, based on Fitts’ Law [8, 9] and Shannon’s information theory [10], the information produced from this subtask is

$$ {\text{H}}\left( {\text{i}} \right) \, = { \log }_{ 2} \left( {\frac{{2{\text{D}}}}{\text{W}}} \right) $$
(3)

To investigate user performance among different levels of task difficulty, different overall baud rate (BTOT) was assigned by manipulating the time interval for each subtask to represent different difficulty levels during the experiment. To assess user performance among individual subtasks, a response ratio of each subtask [13] was applied to the model to determine the weight of each subtask. The response ratio (RR(i)) is defined as correct response per trial for subtask i. The weight of subtask i is denoted as w(i) and

$$ \mathop \sum \nolimits {\text{w}}({\text{i}}){\text{RR}}({\text{i}})\, = \,1 $$
(4)

Overall baud rate of MATB-II tasks is defined as BTOT.

$$ {\text{B}}_{\text{TOT}} \, = \,\sum {\text{w(i)B(i)}} $$
(5)

To estimate the weight of each subtask, this approach includes two phases. During the first phase, all subtasks are assigned with equal weight and all response ratios are assumed to equal to 1. After a user completes a set of tasks in first phase, the response ratio of each subtask is calculated and the weight of each subtask is estimated according to Eq. (4). To maintain the same level of task difficulty and to balance among each individual tasks, a new set of tasks with different weights is assigned according to Eq. (5).

2.2 Participants

Ten users were recruited to participate in this study. Participants consisted of 6 males and 4 females, with an average age of 25 years old. All participants reported normal or corrected-to-normal vision and were right handed. All participants were native English speakers. All participants completed two phases of this study.

2.3 Experiment Design

Three levels of task difficulty were manipulated by setting up different overall baud rate (BTOT = 0.5, 1.0, and 1.6 bps, respectively). There were two phases in this study with the three levels of overall baud rate kept the same for both phases. All participants were required to complete both phases.

Before the first phase, a training session of MATB-II tasks was provided to each participant. During the first phase, each participant was required to complete 6 trials (3 levels × 2 replicates) of MATB-II tasks. All participants were required to complete the tasks as quickly and accurately as possible. Each trial was 5 min. At the end of each trial, the NASA-TLX questionnaire was presented. After the participant completed the questionnaire, s/he was allowed a 3 min rest before the next trial. Phase I took 60 min for each participant.

After Phase I, user performance was analyzed and a new set of MATB-II tasks was assigned to each participant. All participant needed to complete the second phase of the experiment at least one day after first phase. Phase II included 6 5-minute trials (3 levels × 2 replicates) of MATB-II tasks. There was no training session during Phase II. It took approximately 50 min for each participant.

3 Results

All participants completed two phases of MATB-II tasks. For each phase, four subtasks were performed by each participant. The response time and response ratio of the Light (RT_Light and RR_Light), Scale (RT_Scale and RR_Scale), and Communication (RT_COMM and RR_COMM) tasks were recorded during experiments. Mean root mean square (MRMS) of the distance between the target and the center point for the Tracking task was also recorded. At the end of each trial, a Workload Rating Scale (WRS) was presented and completed by the participant. The mean rating scores were recorded for all trials in order to estimate the workload of each trial. Table 1 shows mean values of user performance parameters.

Table 1. Means of user performance parameters
Full size table

Most parameters of user performance showed the appearance of non-normal data in normality test, except for response time in the Scale task (RT_Scale) and WRS. A nonparametric method, Kruskal-Wallis one-way ANOVA, was applied to investigate the differences in user performance among different difficulty levels. Table 2 shows the p-value from ANOVA for each parameter.

Table 2. ANOVA of difficulty level for user performance
Full size table

All parameters of the changes between two phases showed the appearance of non-normal data in normality test. To compare user performance between two phases, a Wilcoxon Signed Rank test was applied for each parameter and each difficulty level between two phases. Table 3 shows the mean changes in each parameter and implies all significant results from the Wilcoxon Signed Rank test.

Table 3. Parameter comparion between two phases
Full size table

4 Discussion

The results of all user performance parameters showed an overall improvement during Phase II compared to Phase I. After reassigning task weights for each individuals, users spent less time to complete Light, Scale, and Communication tasks at all difficulty levels, meanwhile, they correctly responded more to signals from Light, Scale, and Communication tasks than during Phase I. Users also had less MRMS for Tracking task at all difficulty levels during Phase II. The subjective rating scores of workload (WRS) implied a decrease at all levels during second phase.

Findings indicated the effect of task difficulty level on user performance in multitasking environment. During both phases, user response time in the Scale task, MRMS of the Tracking task, and the overall workload (WRS) showed significant differences among task difficulty level. In addition, during Phase II, response ratio implied significant results among task difficulty levels.

Comparison of parameters between two phases demonstrated significant improvement in user performance after reassigned subtasks to users based on their performance during Phase I. The findings of response time of the Light, Scale, and Communication tasks indicated decreases at all three difficulty levels and more than half of them showed significant changes in Phase II. The findings of response ratio implied an overall increase in Phase II. Only two treatments of this parameter showed significant increases after reassigning the subtasks in MATB-II. A potential reason is that users already had high response ratios among these subtasks (RR(Light) > 0.97, RR(Scale) > 0.96, and RR(COMM) > 0.91) during Phase I. After reassigning the subtasks, users were still able to maintain a higher level of correct responses among different difficulty levels and showed an overall increase in their performance. The findings of the Tracking task revealed significant decreases of MRMS among all difficulty levels. Results of workload ratings (WRS) showed an overall decrease among all levels.

5 Conclusion

This study proposed a quantitative model to assess user performance in multitasking environment and demonstrated the approach of task weight assignment based on individual’s performance to improve overall multitasking performance. Results implied an overall improvement in user performance after reassigning subtasks to each participant. Findings in this study also indicated there was an effect of task difficulty levels on user performance and several parameters showed significant difference among different levels of task difficulty. This approach is able to provide a guideline for multitasking users to train themselves and improve their performance based on different multitasking system and individual performance. Principles of multitasking operation can be established on the proposed model to achieve different task goals (e.g., satisfaction, response ratio, task efficiency, etc.).

There should be other potential factors than task difficulty influence on user performance in multitasking. Future research could investigate the effect of other factors on multitasking performance (e.g., training effect, individual differences).