Keywords

1 Introduction

When users interact with technical systems for the first time, they start to develop a mental representation of the functioning of the technical system that becomes more detailed by and by. In dealing with the system, users have to learn how it works and to deal with uncertainties about its exact functioning due to inexperience. If users interact, for example, with a ticket machine the first time, they might need some time to search and find the appropriate items when using it, press more often the back button or do not understand that they have to touch the screen in order to activate the menu. If they are trained to use this ticket machine and know the possible processes, they are quite fast and could nearly automatically navigate to the desired ticket. Thus, they develop a mental representation of the ticket machine which is a reduced but for the user coherent copy of the reality. The performance improves with more accurate mental representation [1]. It would be helpful if the technical system could take the state of the learner into account and adapt accordingly to support the learning process. In line with this, the question arises which indicators could be used for this adaptation. In human-computer interaction literature, eye movement parameters are often described to be useful for gaining insights into cognitive processes and thus to detect usability problems. The most known parameters are already listed by e.g. Jacob and Karn [2] and Ehmke and Wilson [3].

One parameter that might prove highly relevant but has not yet gained lots of attention in the context of adaptive systems is the blink rate. It has already been reported that eye blinks are related to information processing and attentional processes e.g. during choice response tasks [4]. There is evidence that an increasing blink rate indicates the completion of a cognitive task when attention decreases and the likelihood of missing relevant information is low. Further, the blink rate decreases when the task is cognitively more demanding, indicating a higher attentional load [5, 6]. Also Brookings et al. [5] concluded in their applied study with air traffic controllers and different challenging scenarios that eye blinks are more sensitive to workload caused by task difficulty than other eye movement parameters. These findings emphasize the potentials of the blink rate as an interesting indicator for the state of the learner. Due to the close link to information processing and attentional processes, blinks in the current study might also reflect learning processes and the level of uncertainty. In the following, it is assumed that the blink rate increases parallel to the learning process.

The blink rate was also already studied in the context of the IQ by Paprocki and Lenskiy [7]. Participants had to rest and to perform an IQ test. The authors grouped their participants into a higher IQ group and a lower IQ group. Interestingly, they found that participants in the higher IQ group showed generally a lower blink rate and a higher eye-blink rate variability while resting than participants in the lower IQ group. The authors stated that the eye-blink rate variability indicated the cognitive state of the user and assume a strong relation to cognitive performance. Thus, we also assume that users with accurate mental representations of the technical system show less blinks in comparison to users with flawed mental representations due to cognitive abilities and the increased attentional effort to process information.

Importantly, depending on the situation the blink rate is also associated with other effects that have to be kept constant like lightning and climate or have to be controlled like fatigue [8]. It is quite common to use blinks as an indicator for fatigue in sleepiness alert systems, for example in cars. In literature, blink duration seems to be one of the most noticeable oculomotoric parameter to reflect fatigue [e.g. 9]. However, in the current study we assume that fatigue is not the driving force as participants constantly have to concentrate on the task for a limited period of time.

2 Method

2.1 Participants

A total of 31 (20 female) with mean age 23 (SD = 4) participated in the experiment at the Leibniz Research Centre for Working Environments and Human Factors. All but four were not students. All participants were right-handed and had normal vision without visual aid. They obtained informed consent and were naïve about the study’s purpose before participating in the experiment. At the end of the experiment, they received either course credit or were paid for their participation.

2.2 Experimental Settings

Gabor figures with different patterns (vertical, horizontal or diagonal lines) with a diameter of 2 cm were presented on a 23.6-in. monitor (1080 × 1920) with a distance of approximately 75 cm. The target object moved into a dark room, which means a black square of 20 × 20 cm size, that was presented in the middle of the monitor. An entrance at the bottom and three exits left, top and right were drawn in the black square (Fig. 1). While performing the experimental task eyes were tracked with the SMI RED500 (SensoMotoric Instruments, Teltow, Germany) every 2 ms (500 Hz screen refresh frequency). Blinks were detected with the Event Detector for High Speed Event Detection provided by the iView software. In order to avoid head movements a skin rest was used. Further, indirect light was set up to prevent irritations of the Eyetracker due to light reflection.

Fig. 1.
figure 1

Starting position of the experiment: Participants were asked to predict the exit of the presented target object by pressing the appropriate arrow key (left, top or right).

Full size image

2.3 Procedure

In order to investigate if the blink rate is a useful indicator for the learning process under uncertainty, the occluded visual spatial search task (OVSST) was set up. Participants had to observe one of three target objects disappearing in a quadratic room and reappearing at one of three exits (left, top, right). Participants were asked to predict the reappearance of the target objects at the exit by pressing the appropriate arrow key of the keyboard directly after the target object disappeared into the room. Participants had to perform the prediction task within 2 000 ms and auditive error feedback was provided if the arrow key was pressed too late or too early (see Fig. 2 for a schematic description). In order to increase task performance, participants had to learn the underlying probability concept. Each object was associated to one of the exits with a higher probability (74%) and to the other exits with a lower probability (11%). In the remaining 4% of the cases objects reappear at the bottom entrance. This rare occurrence is used to analyze the behavior of the participants with regard to special unexpected and uncertain situations.

Fig. 2.
figure 2

Schematic description of the occluded visual spatial search task (OVSST) performed in the current experiment. First, participants fixate a fixation cross in the middle of the room (750 ms). Then, one of the three target objects appears at the bottom entrance (1 000 ms) and moves into the room within 1 000 ms. After the target object has disappeared into the room participants are instructed to predict at which exit the target object will reappear with the arrow keys of the keyboard within 2 000 ms. Finally, the target object fades in at one of the exit positions within 3 000 ms.

Full size image

Participants were not informed about any probability concept, they were only instructed to improve the prediction. In the experimental condition, participants had to perform four blocks of the prediction task each with 81 trials. They started with two training blocks with 21 trails each in order to become familiar with the task. At the end of the experiment, participants were asked to estimate the probability relations via questionnaire and to notify if they understand any probability concept (“Do you think the target object (diagonal lines, horizontal lines, vertical lines) influences the exit where the symbol reappears?”).

Participants performed a computer version of the D2 Test of attention by Brickenkamp [10] before and after the completion of the experimental task in order to check for systematic changes in attention capacities, e.g. due to fatigue as a confounding variable. The D2 Test of attention measures accuracy and speed during the discrimination of similar visual stimuli.

2.4 Data Analysis

Data of the experimental condition were adjusted by excluding 3.5% of the trials from data analysis due to missing responses. Another 1.7% of the trials were excluded, because less than 65% of the eye tracking data within those trials were not available due to tracking errors. A two-way repeated measures ANOVA with the within-subjects variable block (1–4) and judgment (correct, incorrect) was employed to statistically analyze the development over time for the number of correct predictions and the number of blinks. For the within-trial analysis, blinks of the valley and peak values before and after the object appearance, prediction task and object reappearance (+/−50 ms) were averaged and planned t-test performed to check if the number of blinks indicates information processing and the completion of the task (Fig. 5).

A hierarchical cluster analysis was performed to group participants into two performance cluster (high and low performer). High performers (N = 16) predicted 62% (SD = 7%) of the cases correctly and low performer (N = 15) made only in 33% (SD = 5%) of the cases a correct prediction. Planned t-tests were used to identify meaningful differences of the number of correct predictions and the number of blinks between these groups.

3 Results

Overall, 20 of the 31 participants confirmed that the target object was related to the exits. In 74% of the cases this self-report coincided with the grouping of the cluster analysis into low and high performer. Further, results of the questionnaire showed that participants who confirmed to understand a probability concept actually learned the tendencies of the probability concept in a correct way as shown in Table 1 (mean maximum deviation from the given probabilities: 12%). In total, they predicted 62% of all trials correctly and chose in 62% of the cases the likely object-exit association of the probability concept. If the Gabor figure with diagonal lines appeared, participants predicted the likely left exit in 65.5% of the cases. If the Gabor figure with horizontal lines was presented, then the likely left exit was chosen in 54.9% of the cases and if the Gabor figure with vertical lines was displayed, the likely top exit was chosen in 65.6% of the cases. The results showed that in 64% of the cases the response strategies did not change after the rare occurrence, the object reappearance at the bottom entrance. Thus, this special situation was not examined in more detail.

Table 1. Subjective probability concept of all participants who confirmed to understand any probability concept (N = 20).
Full size table

The number of correct predictions for all participants (N = 31) increased significantly across blocks, F(3;90) = 13.65, p < .001, \( \eta_{p}^{2} = .313 \), indicating learning effects particularly from Block1 to Block2 (Fig. 3). Planned t-test for high performer, t(15) = 4.57, p < .001, as well as for low performer t(14) = 6.29, p = .060, showed a significant or almost significant increase from Block1 to Block2. Parallelly to the increasing number of correct predictions, the main effect of block for the number of blinks showed a similar trend, F(3;90) = 2.25, p = .104, \( \eta_{p}^{2} = \) .070, suggesting an increase of the number of blinks across blocks. However, we observed no main effect of judgment for the number of blinks, F(1;30) = 0.80, p = .779, \( \eta_{p}^{2} = \) .003 (Fig. 4).

Fig. 3.
figure 3

Mean number of correct predictions for high and low performer across blocks (1–4). Maximum are 81 correct prediction per block. Error bars depict standard errors.

Full size image
Fig. 4.
figure 4

Results of the number of blink rate as a function of block (1–4) and judgment (correct, incorrect). Error bars depict standard errors.

Full size image

The within-trial analysis of the number of blinks showed an increase after the appearance and reappearance of the target object at the exit as well as after the completion of the prediction task (see Fig. 5) indicating a strong relation to the presentation of information and thus attentional processes.

Fig. 5.
figure 5

The mean number of blinks during the time course of the trial for correctly and incorrectly predicted trials. The first vertical grey line marks the beginning of the object appearance (1), the second line marks the beginning of the prediction task (2) and the third line marks the beginning of the reappearance of the object (3) from left to right.

Full size image

Most interestingly, results of the cluster analysis showed that high performer blinked significantly less often compared to low performer, t(29) = 3.24, p = .003 (Fig. 6).

Fig. 6.
figure 6

The number of blinks for low and high performer. Error bars depict standard errors.

Full size image

The analysis of the D2 test showed that participants selected significantly more targets after the experiment than before, t(30) = 7.04, p < .001. In the pre-test, they detected 64% (SD = 16%) of the cases on average and in the post-test 71% (SD = 17%) of the cases on average, presumably due to learning effects. Additionally, the error rate changed significantly from the pre-test (M = 98%, SD = 2%) to the post-test (M = 99%, SD = 1%), t(30) = 2.67, p = .013. The results did not differ between low and high performer (p > .05). Thus, attention generally did not seem to decrease from the beginning to the end of the experiment.

4 Discussion

Besides learning effects across blocks, it was expected that the blink rate reflects the development of the mental representation. The blink rate indeed increased while participants learned the probability concept supposedly indicating that relevant information was processed more easily and the attentional effort accordingly decreased. However, we could only detect a trend probably due to the rapid learning in Block1 due to the easiness of the task. Participants had only to learn three target objects associated to three exits and tend to focus mainly on the likely object-exit association. This result is generally in common with the inverse relation between task difficulty and eye blink frequency reported earlier in the review by Martins and Carvalho [11]. In the current experiment, task difficulty also decreases as participants learn to handle the situation. If the participants advised to understand the probability concept, they developed a quite accurate mental representation of the relations between the target objects and mainly chose the likely object-exit association. Interestingly, they tend to underestimate higher probabilities and overestimate lower probabilities, probably because of a regression to the mean [12, 13]. Participants who did not understand any probability concept seem to learn also a strategy to deal with the situation, but they develop an inaccurate mental representation of the task so that task performance does not increase to a greater extent.

The within trial analysis showed that blinks were suppressed during the object appearance and object reappearance in order to process the presented information of the target object. Participants need to identify the target object to activate the association to the appropriate exit and they have to learn this association by combining the target object with the reappearance at the distinct exits. One step further, they have to estimate the probability of occurrence to improve their predictions and to develop an accurate mental representation of the OVSST. Blinks were also suppressed before the completion of the prediction task suggesting the acquisition of adequate attentional resources to perform the task. Thus, results of earlier studies could be replicated [e.g. 4, 14, 15]

Finally, high performer showed a lower blink rate than low performer which could be attributed to the focus on the task and cognitive abilities. As already reported in the Introduction, Paprocki and Lenskiy [7] studied eye blinks during the performance of a IQ tests and they also found that subjects in the higher IQ group blinked less often than in the lower IQ group. They explain this finding by the connection of eye-blinks to the quality of higher cognitive processes. The findings for high and low performers seem to reflect these findings, since the high performers obviously have developed a better mental representation of the task than the low performers.

It might seem contradicting that on the one hand the overall number of blinks increase with the number of correct predictions but on the other hand high performer blink less than low performer. However, in general all participants develop a mental representation and learn a strategy indicated by the increasing number of blinks. Thereby good learners develop an accurate mental representation of the task and seem to be more focused on the task reflected by less blinks than poor learners. The low task performance of poor learners caused by an inaccurate mental representation of the task might have different reasons: motivational causes, concentration deficits or cognitive inabilities. As the D2 concentration test indicated no systematic changes in attention capacities, a general lack of attention resources and fatigue can be excluded.

Overall, there seemed to be high interindividual variability with regard the number of correct predictions indicated by the high range of the standard error bars and to the number of blinks. The uniqueness of eye blink patterns of the individual participants was already reported in literature [11] and has to be taken into account if blinks are used as an indicator for the state of learner, e.g. by first defining a baseline for each individual user.

In sum, the blink rate might be a relevant indicator for the state of the learner. In future, technical systems with eye-tracking software might detect the state of the user and adapt in the way, for example, that information is kept simple and even repeated and additional information is presented for low performers. Thus, users will be supported on an individual level possibly before they experience difficulties during the interaction with the technical system. Such an approach could provide a way to maintain or optimize the cognitive load during learning [e.g. 16]. However, more research is needed to foster the results. The next step towards using the blink rate as an indicator for the learner state diagnosis in adaptive instructional systems would be to implement and test the blink rate in more applied research studies, for example in a VR learning setting [e.g. 17] as blinks could be easily detected and information accordingly adapted.