Abstract
Technologies to measure gaze direction and pupil reactivity have become efficient, cheap, and compact and are finding increasing use in many fields, including gaming, marketing, driver safety, military, and healthcare. Besides offering numerous useful applications, the rapidly expanding technology raises serious privacy concerns. Through the lens of advanced data analytics, gaze patterns can reveal much more information than a user wishes and expects to give away. Drawing from a broad range of scientific disciplines, this paper provides a structured overview of personal data that can be inferred from recorded eye activities. Our analysis of the literature shows that eye tracking data may implicitly contain information about a user’s biometric identity, gender, age, ethnicity, body weight, personality traits, drug consumption habits, emotional state, skills and abilities, fears, interests, and sexual preferences. Certain eye tracking measures may even reveal specific cognitive processes and can be used to diagnose various physical and mental health conditions. By portraying the richness and sensitivity of gaze data, this paper provides an important basis for consumer education, privacy impact assessments, and further research into the societal implications of eye tracking.
Keywords
- Eye tracking
- Gaze
- Pupil
- Iris
- Vision
- Privacy
- Data mining
- Inference
Download chapter PDF
1 Introduction
Being an important part of visual perception and human behavior, eye movements have long been a subject of research interest. The first approaches to measure a person’s gaze direction date back to the early 1900s [74]. Until recently, these technologies were severely limited by the cost of the equipment required, a lack of precision, and poor usability and were only used in very specific niches of research. Over the last few years, however, with rapid advances in sensor technology and data processing software, eye tracking solutions have become easy to use, lightweight, efficient, and affordable and found increasing adoption in many fields, including gaming, marketing, automotive technology, military, and healthcare [26].
While alternativesFootnote 1 exist, the most popular method today is video-based eye tracking, where mathematical models are used to calculate a person’s gaze direction from video recordings, for example based on the shape and position of pupil and iris, or based on light reflection patterns in the eyes [2]. This method can not only be used in head-mounted devices, such as smart glasses and virtual reality headsets, but also through built-in cameras in laptops, tablets, and smartphones without requiring any additional hardware [45, 56]. With further improvements in cost and performance, eye tracking may soon be included as a standard feature in various consumer electronics, moving us towards a “pervasive eye tracking world” [58].
The many beneficial uses and enormous potentials of the rising technology have to be acknowledged and should be embraced. However, a more ubiquitous use of eye tracking will also raise serious privacy concerns – not only because gaze data may be collected and shared in non-transparent ways, but also because such data can unexpectedly contain a wealth of sensitive information about a user.
Drawing from a broad range of scientific disciplines, including neuroscience, human-computer interaction, medical informatics, affective computing, experimental economics, psychology, and cognitive science, this paper provides a structured overview and classification of sensitive pieces of information that can be disclosed by analyzing a person’s eye activities. According to the reviewed literature, eye tracking data may reveal information about a user’s biometric identity (Sect. 2.1), mental activities (Sect. 2.2), personality traits (Sect. 2.3), ethnic background (Sect. 2.4), skills and abilities (Sect. 2.5), age and gender (Sect. 2.6), personal preferences (Sect. 2.7), emotional state (Sect. 2.8), degree of sleepiness and intoxication (Sect. 2.8), and physical and mental health condition (Sect. 2.9). In order to take rapidly evolving technology trends and newly emerging privacy threats into account, we will consider not only proven and established approaches but also inference methods that are subject to ongoing research. Limitations of the presented methods and their practical applicability will be reflected upon in Sect. 3, followed by a conclusion in Sect. 4.
2 Inference of Personal Information from Eye Tracking Data
With reference to published research, filed patents, and existing commercial products, this section presents and categorizes personal information that can be inferred from eye tracking data. As a basis for potential inferences, eye tracking devices can record a large variety of gaze parameters.
Some of the most commonly measured eye movements are fixations, saccades, and smooth pursuit eye movements [85]. During a fixation, the eyes are relatively stable and focused on a specific position, allowing for information to be acquired and processed. Saccades are rapid eye movements from one fixation point to another, lasting 30 to 80 ms [87]. Smooth pursuit movements are performed when eyes are closely following a moving visual target. In addition to the spatial dispersion, duration, amplitude, acceleration, velocity, and chronological sequence of such eye movements, many eye trackers capture various other eye activities, including eye opening and closure (e.g., average distance between the eyelids, blink duration, blink frequency), ocular microtremors, pupil size, and pupil reactivity [19, 58]. Furthermore, most eye trackers videotape parts of the user’s face and may thereby capture additional information, such as the number and depth of wrinkles, and a user’s eye shape and iris texture [40]. Therefore, these parameters were also considered in our investigation into the richness and sensitivity of eye tracking data. Fig. 1 provides an introductory overview of common eye tracking measures and the categories of inferences discussed in this paper.
2.1 Biometric Identification
Due to differences in physical oculomotor structure and brain functioning, certain gaze characteristics are unique for every individual, similar to fingerprints, and can thus be exploited for biometric identification [9, 74, 76]. Among other methods, people can be told apart based on distinct patterns of pupil reactivity and gaze velocity [9], or by comparing their eye movement trajectories when they focus on a moving target – even if the eye activity is only recorded through an ordinary smartphone camera [55].
Aside from such gaze-based measures, the complex textures and color patterns in a person’s iris are also suitable for biometric identification. This approach, called iris recognition, is being used in a variety of real-world security and surveillance applications and has been recognized as “one of the most powerful techniques for biometric identification ever developed” [64]. Even though their iris scanning capability is usually not advertised, it should be understood that commodity eye trackers often record and process high-resolution images of the user’s iris, which can not only be used to uniquely identify the user but also to deceive iris-based authentication mechanisms and thereby steal the user’s identity [40].
In cases where a unique identification of an individual is not possible (e.g., because the person is not registered in the recognition system database), other attributes inferred from eye tracking data, such as age and gender (see Sect. 2.6), health condition (see Sect. 2.9), or ethnicity (see Sect. 2.4), can still help to classify the target person into a specific demographic group and thereby approximate the identity [74].
2.2 Monitoring of Mental Workload and Cognitive Processes
Certain patterns in eye movement, pupil dilation, and eye blinking have been recognized as reliable indicators of mental workload in people of any age [19, 63], sometimes offering higher accuracy than conventional methods like Electroencephalography [8]. Through eye tracking, it is also possible to distinguish a user’s moments of awareness from moments of distraction and mind wandering [31, 84].
Apart from detecting a user’s mental presence and measuring the mere intensity of cognitive processing, eye tracking can also provide insights into specific conscious and unconscious thought processes in a large variety of contexts. Among other mental tasks and activities, ocular measures have been used to study memory retrieval [19, 31], problem solving [31, 75], learning processes [44, 69], the formation of expectations [19, 27], internal reasoning [19], and mental computations [19, 31].
Eye tracking data can not only – to a certain extent – reveal what we remember, imagine, expect, and think about, but also our specific decision-making strategies [19, 28] and cognitive styles, i.e., individual differences in the way we acquire, process, and interpret information [72]. For example, people can be classified as field-dependent vs. field-independent (people of the latter type pay more attention to detail and exhibit a more analytical approach to processing visual information) [72], or as verbalizers vs. visualizers (people of the latter type can process visual information, such as images and diagrams, better than textual information) [44]. The gaze-based inference of such cognitive styles is feasible and can achieve high accuracies, as has been confirmed in a recent study by Raptis et al. [72].
Researchers from the field of cognitive science and experimental psychology have suggested that eye tracking data will not only be used for the real-time analysis but also for the prediction of human decisions and behavior [28].
2.3 Inference of Personality Traits
Experimental research has shown that it is possible to automatically infer personality traits from eye tracking data [34, 35, 42]. For example, gaze patterns captured during everyday tasks can be used to evaluate users along the so-called Big Five traits, namely openness to experience, conscientiousness, extroversion, agreeableness, and neuroticism [34, 42]. The gaze-based assessment of personality traits is possible not only in binary form (high vs. low) but also in the form of ranges. In [35], for instance, eye movement analysis was used for the automatic recognition of different levels of curiosity.
Besides the Big Five traits and curiosity, gaze metrics were found to be associated with various other personality traits, including emotional intelligence [54], indecisiveness [36], the tendency to ruminate [21], trait anxiety [42], sexual compulsivity [87], boredom susceptibility [70], and general aggressiveness [6]. Eye tracking has even been used to investigate people’s attachment styles in interpersonal relationships (e.g., secure, withdrawn, fearful, enmeshed) [81].
Based on data from 428 study participants, Larsson et al. [53] also suggest that some personality traits, including tendermindedness, warmth, trust, and impulsiveness, are genetically linked to certain iris characteristics, offering – besides gaze behavior – another potential ocular biomarker to analyze people’s personalities.
2.4 Inference of Cultural Affiliation and Ethnicity
It is widely agreed that culture fundamentally shapes human cognitive processing and behavior [11]. Studies have shown that intercultural differences are reflected in certain gaze characteristics [12, 24, 41, 61]. For example, people of different cultural background were found to exhibit discriminative eye-movement patterns when seeking information on search engine results pages [61], when exploring complex visual scenes [12, 24], and when viewing videos of actors performing cultural activities [41]. Some cultural biases in visual processing are so pronounced that they can still be measured when external stimuli draw attention in an opposite manner to the respective bias [24].
Additionally, eye movements can reveal a person’s knowledge of certain cultural practices. For instance, in an eye tracking study by Green et al. [27], Chinese infants exclusively predicted the goal of eating actions performed by an actor with chopsticks, whereas European infants only anticipated that food would be brought to the mouth when eating actions were performed with Western cutlery, as indicated by their predictive gaze shifts towards the actor’s mouth.
Some studies have also investigated how people of different “race”Footnote 2 differ in their viewing behavior [25, 33, 88]. Apart from the fact that video-based eye trackers can directly record the eye color, eye shape, and skin color of a user, it has been observed in eye tracking studies that test subjects view “other-race faces” differently than faces of their “own race” in terms of the facial features scanned (e.g., initial focus and greater proportion of fixation time on the eyes vs. nose and mouth) [25, 88]. Furthermore, researchers have observed characteristic changes in pupil size, which are attributed to elevated cognitive effort during face recognition, when people look at “other-race faces” [88]. Such differences have been reported, for example, between “Black and White observers” [33] and between “Western Caucasian and East Asian observers” [25] and could potentially allow inferences about the genetic and ethnic background of eye tracking users.
Eye tracking data may also allow inferences about a user’s native language. For instance, considerable differences in eye movement patterns during reading can be observed between native and non-native speakers of English [39]. Eye tracking can even be used to determine which specific words are difficult to understand for a person [51]. Among other things, such information could help in estimating a subject’s nationality or geographical origin.
2.5 Skill Assessment
Eye tracking has been used extensively in the study of human expertise and to discriminate between performance levels in a variety of areas [30, 31, 69, 75]. For example, gaze behavior can be analyzed to assess reading and listening comprehension skills [10, 92]. During a corresponding task or scenario, eye tracking can also be used to distinguish between experts and novices in chess [75], several sports [46], chemistry [69], mathematics [31], school teaching [14], and various medical skills, including surgery, nursing, anesthesia, and radiology [30].
Among other gaze characteristics, expertise is often associated with systematic eye movement patterns reflecting a specific task strategy [31], with the targeted inspection of important regions and task-relevant information [30, 75], and with more consistent gaze patterns over consecutive trials of a task [46].
In some fields, eye tracking has not only been used as a tool to discriminate between people of different skill levels, but also to predict people’s task performance and learning curves [52, 69] and to examine specific learning disabilities, such as mathematical difficulties and dyslexia [31, 85].
2.6 Age and Gender Recognition
Just like physical shape, skin texture, and cognitive abilities, human eyes and visual behavior are fundamentally affected by the aging process [20, 36]. For example, eye tracking studies found age-related differences in people’s visual explorativeness, pupil reactions to certain visual stimuli, and error rates in eye movement tasks [36, 42].
Furthermore, detailed frontal face images, which are typically required for video-based eye tracking, have already been used for automated age estimation, for instance based on wrinkles in the eye area [15]. Dynamic facial expressions, such as smiles, may also be analyzed to infer the age of test subjects [17]. Other parameters utilized for computerized age-group recognition include iris size and iris texture [20].
As with age, a person’s gender can be reflected in certain eye tracking measures. For instance, studies found systematic gender differences in people’s fixation distribution while viewing natural images (e.g., stills from romance films or wildlife documentaries) [68], during online shopping [38], when playing video games [42], and when viewing sexual stimuli [87]. Researchers have already used such differences in visual behavior to automatically classify the sex of test subjects [68].
2.7 Inference of Preferences and Aversions
Eye tracking is widely employed to investigate people’s interests, likes, and dislikes. Spontaneous attention to specific objects in a visual scene (e.g., in terms of frequency, duration, and sequence of gaze fixations) is regarded as a natural indicator of interest [19, 74, 87]. For data presentation and analysis, gaze fixations are commonly aggregated into heat maps to quickly identify potential regions and objects of interest [74].
Besides the focus of visual attention, other eye parameters, such as pupil dilation and blink properties, can also be used to analyze a person’s degree of interest and to distinguish between positive, neutral, and negative responses to visual stimuli [55]. Emotion detection from gaze data, which can assist in analyzing a user’s interests and preferences [55, 83], will be discussed in Sect. 2.8.
Among other things, eye tracking has been used to examine preferences for certain types of gambling [65], mobile apps [56], activities of daily living [86], types of food [32], colors, geometric shapes, and product designs [3], pieces of clothing, animals, video game characters, and items of furniture [83]. Beyond mere interest, existing research even suggests that people’s patterns of visual attention reflect their consumption and purchasing behavior [91].
Eye tracking has also been used extensively in the study of love and sexual desire. For example, researchers have analyzed pupillary responses and the allocation of visual attention to measure levels of sexual arousal and to investigate mating preferences towards specific facial characteristics, age groups, body shapes, body parts, and signs of social dominance [3, 87].
Apart from positive interests, visual attentional biases captured by eye trackers can also reflect a person’s phobias and aversions (e.g., fear of spiders) [3, 37]. Some interests and preferences can already be inferred from eye tracking data with high accuracy [56, 73, 87] and several patents have been filed in this field [3, 83].
2.8 Detection of Short- and Medium-Term User States
Moods and Emotions.
Eye tracking is increasingly used in the interdisciplinary field of affective computing, where systems are developed to automatically recognize human emotions based on physiological signals and behavioral cues [73, 83]. It has been shown that various ocular measures, including pupil size, blink properties, saccadic eye movements, and specific biases of visual attention, can contain information about a person’s emotional state [4, 23, 55, 59].
Gaze data can reflect emotional arousal and the valence of emotions (positive, negative, neutral) [19, 55] as well as more specific affective states, such as happiness and enthusiasm [83], acute stress and worry [59], humorous moods and disgust [73], curiosity [4], distress, nervousness, and hostility [23], fear, anger, sadness, and surprise [55].
Eye tracking can not only be used to detect emotions with high accuracy [73] but also to estimate the intensity of emotions [55, 83]. Based on gaze parameters, existing methods can even distinguish whether a user’s emotional response to a given stimulus is rational or purely instinctive [55].
Fatigue and Sleepiness.
For over two decades, there have been approaches to automatically derive a person’s level of sleepiness from certain ocular measures, such as blink rate, blink duration, average distance between the eyelids, fixation durations, and velocity of eye movements [57]. Recent studies have confirmed the suitability of eye tracking measures as indicators for sleepiness and fatigue [63, 89]. Corresponding methods have already been patented and achieve high accuracies – not only while the user is working on specific cognitive tasks, but also during everyday natural-viewing situations [57, 89].
Intoxication.
The consumption of alcohol and other recreational drugs can have measurable effects on various eye and gaze properties, such as decreased accuracy and speed of saccades, changes in pupil size and reactivity, and an impaired ability to fixate on moving objects [29, 67, 85].
Apart from alcohol, significant abnormalities in oculomotor functioning were found in people under the influence of nicotine, 3,4-methylenedioxymethamphetamine (“MDMA”), and tetrahydrocannabinol (“THC”) [29, 70].
Researchers have demonstrated the ability to differentiate between drug-impaired and sober subjects with high accuracy based on eye tracking data [29]. The magnitude of some ocular effects is closely associated with the amount of drugs consumed [85] and certain effects can even be detected at non-intoxicating doses [77]. In addition to pupillary changes and eye movement impairments, an attentional bias towards drug-related visual stimuli has been observed among intoxicated test subjects [67].
Not only a state of intoxication, but also an acute state of drug deprivation and craving can have a distinct effect on certain eye tracking parameters [29, 70].
2.9 Health Assessment
Physical Health.
Many diseases and medical conditions directly affect the eyes, or parts of the brain that are responsible for oculomotor function, and thereby cause gaze impairments [3, 19, 30]. Characteristic eye movement patterns were found, for example, in people suffering from concussion [43], fetal alcohol syndrome [3], irregular growth [3], chronic pain [22], neurocognitive impairment due to preterm birth [82], multiple sclerosis [3], Alzheimer’s disease [30, 43], Tourette syndrome [19], Parkinson’s disease [30], and various vision disorders (e.g., myopia, farsightedness, and blind spots) [3, 43].
As filed patents and published experimental studies show, eye movement analysis can be used to diagnose, monitor, prognose, and sometimes even predict various health disorders [30, 43] which can be subsumed under the umbrella term ETDCC (“Eye Tracking-Relevant Diseases, Conditions, and Characteristics”) [3].
Research has further demonstrated that certain patterns in gaze orientation and pupil reactivity to food-related stimuli (e.g., high vs. low calorie food images) can be indicative of overweight and obesity [32].
Mental Health.
Abnormal eye movements can be used as behavioral biomarkers for the diagnosis of various mental health problems [1, 5, 29]. Oculomotor dysfunctions and gaze peculiarities are found, for example, in sufferers of anxiety disorder [29], depression [1], bipolar disorder [30], borderline personality disorder [6], schizophrenia [5], obsessive–compulsive disorder [13], binge-eating disorder [79], ADHD [7], mild cognitive impairment [30], autism [43], and posttraumatic stress disorder [66].
Some common symptoms of mental disorders are irregularities in blink rate and blink duration [19], abnormal stability and dispersal of gaze fixations during free viewing [5], unusual biases of visual attention [66], impaired smooth pursuit eye-movement performance [85], eye contact avoidance, and abnormal distance between the eyelids [1].
Certain mental illnesses, including depression and schizophrenia, can already be detected automatically via eye tracking [1, 5, 30] and corresponding methods have been filed as patents [43]. Besides the possibility of binary classification (suffering vs. not suffering), some ocular measures are associated with the severity of mental disorders [19]. Not only acute disorders can be reflected in gaze data, but also past mental health issues and even the personal risk of future outbreaks [71, 78]. For example, researchers have observed characteristic gaze patterns in previously depressed individuals [78] and found biases in visual attention that were predictive of future depression scores at a delay of more than two years [71].
Substance Use Disorders.
Apart from acute states of intoxication (which we have discussed in Sect. 2.8), eye tracking data may contain information about a user’s longer-term drug consumption habits and addictions. Numerous eye tracking studies have reported a strong attentional bias towards drug-related visual cues in addicts of cocaine [16], alcohol [67], cannabis [90], and tobacco [18, 70].
Among other possible methods, such attentional biases can be detected by measuring how quickly, how often, and for how long a person’s eyes fixate on corresponding stimuli in comparison to neutral stimuli, or by testing the person’s ability to look away from drug-related stimuli on command [16, 18]. Significant biases have not only been observed in long-term addicts but also in habitual drug users without clinical symptoms of dependency [18, 67]. The strength of attentional biases towards drug-related visual cues was found to be correlated with scores on drug use scales, such as the Obsessive Compulsive Cocaine Scale [16] and with self-reported lifetime drug consumption [62]. Research has also shown that certain biases in visual attention can be predictive of craving and even relapse in drug addiction [16].
3 Discussion and Implications
As shown in the previous section, various kinds of sensitive inferences can be drawn from eye tracking data. Among other categories of personal data, recorded visual behavior can implicitly contain information about a person’s biometric identity, personality traits, ethnic background, age, gender, emotions, fears, preferences, skills and abilities, drug habits, levels of sleepiness and intoxication, and physical and mental health condition. To some extent, even distinct stages of cognitive information processing are discernable from gaze data. Thus, devices with eye tracking capability have the potential to implicitly capture much more information than a user wishes and expects to reveal. Some of the categories of personal information listed above constitute special category data, for which particular protection is prescribed by the EU’s General Data Protection Regulation (Art. 9 GDPR).
Of course, drawing reliable inferences from eye tracking data is not a trivial task. Many situational factors can influence eye properties and gaze behavior in complex ways, making it difficult to measure the effect of a particular action, internal process, or personal characteristic of the user in isolation [55]. Seemingly identical ocular reactions can result from completely different causes. For example, an intensive gaze fixation on another person’s face may indicate liking, aversion, confusion, recognition, and much more. Similarly, a sudden change in pupil size can be indicative of many different feelings or internal states, including physical pain, sexual arousal, interest, happiness, anger, or simply be a reaction to ambient events and conditions, such as noise or varying lighting [19, 55].
In spite of existing challenges and limitations, the reviewed literature demonstrates that there is considerable potential for inferences in many areas and that numerous research projects, patented systems, and even commercial products have already taken advantage of the richness of eye tracking data to draw inferences about individuals with high accuracy.
It should be acknowledged that many of the cited inference methods were only tested under controlled laboratory conditions and lack evaluation in real-world scenarios [4, 18, 27, 52, 65, 67, 69, 86, 88]. On the other hand, it may reasonably be assumed that some of the companies with access to eye tracking data from consumer devices (e.g., device manufacturers, ecosystem providers) possess larger sets of training data, more technical expertise, and more financial resources than the researchers cited in this paper. Facebook, for example, a pioneer in virtual reality and eye tracking technology, is also one of the wealthiest and most profitable companies in the world with a multi-billion dollar budget for research and development and a user base of over 2.3 billion people [93]. It seems probable that the threat of unintended information disclosure from gaze data will continue to grow with further improvements of eye tracking technology in terms of cost, size, and accuracy, further advances in analytical approaches, and the increasing use of eye tracking in various aspects of daily life.
In assessing the privacy implications of eye tracking, it is important to understand that, while consciously directed eye movements are possible, many aspects of ocular behavior are not under volitional control – especially not at the micro level [19, 55]. For instance, stimulus-driven glances, pupil dilation, ocular tremor, and spontaneous blinks mostly occur without conscious effort, similar to digestion and breathing. And even for those eye activities where volitional control is possible, maintaining it can quickly become physically and cognitively tiring [58] – and may also produce certain visible patterns by which such efforts can be detected. Hence, it can be very difficult or even impossible for eye tracking users to consciously prevent the leakage of personal information.
Though this paper focuses on privacy risks, we do not dispute the wide-ranging benefits of eye tracking. Quite the opposite: we believe that it is precisely the richness of gaze data and the possibility to draw insightful inferences from it that make the rising technology so valuable and useful. But to exploit this potential in a sustainable and socially acceptable manner, adequate privacy protection measures are needed.
Technical safeguards have been proposed to prevent the unintended disclosure of personal information in data mining, including specialized solutions for eye tracking data [58, 80]. These comprise the fuzzing of gaze data (i.e., inserting random noise into the signal before passing it down the application chain) and the utilization of derived parameters (e.g., aggregated values instead of detailed eye fixation sequences) [58]. Experiments have already shown that approaches based on differential privacy can prevent certain inferences, such as user re-identification and gender recognition, while maintaining high performance in gaze-based applications [80]. In addition to approaches at the technical level, it should also be examined whether existing laws provide for sufficient transparency in the processing of gaze data and for proper protection against inference-based privacy breaches. The promises and limitations of existing technical and legal remedies are beyond the scope of this paper but deserve careful scrutiny and will be considered for future work.
Even though eye tracking is a demonstrative example, the threat of undesired inferences is of course much broader, encompassing countless other sensors and data sources in modern life [47]. In other recent work, we have examined sensitive inferences that can be drawn from voice recordings [49] and accelerometer data [48, 50], for instance. In our view, the vast possibilities of continuously advancing inference methods are clearly beyond the understanding of the ordinary consumer. Therefore, we consider it to be primarily the responsibility of technical experts, technology companies, and governmental agencies to inform consumers about potential consequences and protect them against such covert invasions of privacy. Also, since it is unlikely that companies will voluntarily refrain from using or selling personal information that can be extracted from already collected data, there should be strong regulatory incentives and controls.
4 Conclusion
While the widespread adoption of eye tracking holds the potential to improve our lives in many ways, the rising technology also poses a substantial threat to privacy. The overview provided in this paper illustrates that, through the lens of advanced data analytics, eye tracking data can contain a rich array of sensitive information, including cues to a user’s biometric identity, gender, age, ethnicity, personality traits, drug consumption habits, moods and emotions, skills, preferences, cognitive processes, and physical and mental health condition. Since inference methods are often based on hidden patterns and correlations that are incomprehensible to ordinary consumers, it can be impossible for them to understand and control what information is revealed.
Although there is extensive literature on the analysis of eye tracking data, we believe that many possible inferences have not yet been investigated. Keeping track of the evolving possibilities of data mining methods in this field is therefore an important avenue for future research. This paper represents a crucial first step towards understanding the sensitivity of eye tracking data from a holistic perspective. The findings compiled herein are significant enough to warrant a warning to users whose privacy could be affected, as well as a call for action to the public and private actors entrusted with protecting user privacy in consumer electronics. Considering the rapid proliferation of eye tracking technology, existing technical and legal safeguards urgently need to be assessed regarding their ability to avert undesired inferences from gaze data, or to at least prevent the misuse of sensitive inferred information.
Notes
- 1.
For an overview of existing types of eye tracking, refer to [2].
- 2.
The authors share the UNESCO’s position [60] that the classification of human populations into “races” is inadequate and obsolete. Nevertheless, it is important to monitor the state of research in this field, especially because any information indicative of a person’s ethnic background can serve as a basis for racist discrimination. All terms related to the concept of “race” in this paper are cited from external sources and do not reflect the authors’ views.
References
Alghowinem, S., et al.: Eye movement analysis for depression detection. In: IEEE International Conference on Image Processing, pp. 4220–4224 (2013)
Al-Rahayfeh, A., Faezipour, M.: Eye tracking and head movement detection: a state-of-art survey. IEEE J. Transl. Eng. Health Med. 1, 2100212 (2013)
Avital, O.: Method and System of Using Eye Tracking to Evaluate Subjects (Patent No.: US20150282705A1) (2015)
Baranes, A., et al.: Eye movements reveal epistemic curiosity in human observers. Vis. Res. 117, 81–90 (2015). https://doi.org/10.1016/j.visres.2015.10.009
Benson, P.J., et al.: Simple viewing tests can detect eye movement abnormalities that distinguish schizophrenia cases from controls with exceptional accuracy. Biol. Psychiatry 72(9), 716–724 (2012). https://doi.org/10.1016/j.biopsych.2012.04.019
Bertsch, K., et al.: Interpersonal threat sensitivity in borderline personality disorder: an eye-tracking study. J. Pers. Disord. 31(5), 647–670 (2017)
Blazey, R.N., et al.: ADHD Detection by Eye Saccades (Patent No.: US6652458B2) (2003)
Borys, M., et al.: An analysis of eye-tracking and electroencephalography data for cognitive load measurement during arithmetic tasks. In: 10th International Symposium on Advanced Topics in Electrical Engineering (ATEE), pp. 287–292 (2017)
Cantoni, V., et al.: Gaze-based biometrics: an introduction to forensic applications. Pattern Recogn. Lett. 113, 54–57 (2018). https://doi.org/10.1016/j.patrec.2016.12.006
Chita-Tegmark, M., et al.: Eye-tracking measurements of language processing: developmental differences in children at high risk for ASD. J. Autism Dev. Disord. 45(10), 3327–3338 (2015). https://doi.org/10.1007/s10803-015-2495-5
Chizari, S.: Exploring the role of culture in online searching behavior from cultural cognitive perspective: case study of American, Chinese and Iranian Graduate Students. In: iConference Proceedings. iSchools, Philadelphia (2016)
Chua, H.F., et al.: Cultural variation in eye movements during scene perception. Proc. Natl. Acad. Sci. 102(35), 12629–12633 (2005). https://doi.org/10.1073/pnas.0506162102
Cludius, B., et al.: Attentional biases of vigilance and maintenance in obsessive-compulsive disorder: an eye-tracking study. J. Obsessive Compuls. Relat. Disord. 20, 30–38 (2019). https://doi.org/10.1016/j.jocrd.2017.12.007
Cortina, K.S., et al.: Where low and high inference data converge: validation of CLASS assessment of mathematics instruction using mobile eye tracking with expert and novice teachers. Int. J. Sci. Math. Educ. 13(2), 389–403 (2015)
Dehshibi, M.M., Bastanfard, A.: A new algorithm for age recognition from facial images. Signal Process. 90(8), 2431–2444 (2010)
Dias, N.R., et al.: Anti-saccade error rates as a measure of attentional bias in cocaine dependent subjects. Behav. Brain Res. 292, 493–499 (2015)
Dibeklioğlu, H., et al.: A smile can reveal your age: enabling facial dynamics in age estimation. In: Proceedings of the 20th ACM International Conference on Multimedia, pp. 209–218. ACM Press, Nara (2012). https://doi.org/10.1145/2393347.2393382
DiGirolamo, G.J., et al.: Breakdowns of eye movement control toward smoking cues in young adult light smokers. Addict. Behav. 52, 98–102 (2016)
Eckstein, M.K., et al.: Beyond eye gaze: what else can eyetracking reveal about cognition and cognitive development? Dev. Cogn. Neurosci. 25, 69–91 (2017)
Erbilek, M., et al.: Age prediction from iris biometrics. In: 5th International Conference on Imaging for Crime Detection and Prevention (ICDP), pp. 1–5 (2013)
Fang, L., et al.: Attentional scope, rumination, and processing of emotional information: an eye-tracking study. Emotion 19(7), 1259–1267 (2018)
Fashler, S.R., Katz, J.: Keeping an eye on pain: investigating visual attention biases in individuals with chronic pain using eye-tracking methodology. J. Pain Res. 9, 551–561 (2016). https://doi.org/10.2147/JPR.S104268
Gere, A., et al.: Influence of mood on gazing behavior: preliminary evidences from an eye-tracking study. Food Qual. Prefer. 61, 1–5 (2017)
Goh, J.O., et al.: Culture modulates eye-movements to visual novelty. PLoS ONE 4(12), e8238 (2009). https://doi.org/10.1371/journal.pone.0008238
Goldinger, S.D., et al.: Deficits in cross-race face learning: insights from eye movements and pupillometry. J. Exp. Psychol. Learn. Mem. Cogn. 35(5), 1105–1122 (2009)
Grand View Research: Global Eye Tracking Market Size By Type, Industry report. https://www.grandviewresearch.com/industry-analysis/eye-tracking-market. Accessed 25 Oct 2019
Green, D., et al.: Culture influences action understanding in infancy: prediction of actions performed with chopsticks and spoons in Chinese and Swedish infants. Child Dev. 87(3), 736–746 (2016)
Guazzini, A., et al.: Cognitive dissonance and social influence effects on preference judgments: an eye tracking based system for their automatic assessment. Int. J. Hum Comput Stud. 73, 12–18 (2015). https://doi.org/10.1016/j.ijhcs.2014.08.003
Hall, C.A., Chilcott, R.P.: Eyeing up the Future of the Pupillary Light Reflex in Neurodiagnostics. Diagnostics 8(1), 1–20 (2018). https://doi.org/10.3390/diagnostics8010019
Harezlak, K., Kasprowski, P.: Application of eye tracking in medicine: a survey, research issues and challenges. Comput. Med. Imag. Graph. 65, 176–190 (2018)
Hartmann, M., Fischer, M.H.: Exploring the numerical mind by eye-tracking: a special issue. Psychol. Res. 80(3), 325–333 (2016). https://doi.org/10.1007/s00426-016-0759-0
Hendrikse, J.J., et al.: Attentional biases for food cues in overweight and individuals with obesity: a systematic review of the literature. Obes. Rev. 16(5), 424–432 (2015)
Hills, P.J., Pake, J.M.: Eye-tracking the own-race bias in face recognition: revealing the perceptual and socio-cognitive mechanisms. Cognition 129(3), 586–597 (2013)
Hoppe, S., et al.: Eye movements during everyday behavior predict personality traits. Front. Hum. Neurosci. 12, 1–8 (2018). https://doi.org/10.3389/fnhum.2018.00105
Hoppe, S., et al.: Recognition of curiosity using eye movement analysis. In: International Conference on Pervasive and Ubiquitous Computing, pp. 185–188 (2015)
Horsley, M. (ed.): Current Trends in Eye Tracking Research. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-02868-2
Huijding, J., et al.: To look or not to look: an eye movement study of hypervigilance during change detection in high and low spider fearful students. Emotion 11(3), 666–674 (2011). https://doi.org/10.1037/a0022996
Hwang, Y.M., Lee, K.C.: Using an eye-tracking approach to explore gender differences in visual attention and shopping attitudes in an online shopping environment. Int. J. Hum. Comput. Interact. 34(1), 15–24 (2018)
Ito, A., et al.: Investigating the time-course of phonological prediction in native and non-native speakers of English: a visual world eye-tracking study. J. Mem. Lang. 98, 1–11 (2018). https://doi.org/10.1016/j.jml.2017.09.002
John, B., et al.: EyeVEIL: degrading iris authentication in eye tracking headsets. In: ACM Symposium on Eye Tracking Research & Applications (ETRA), pp. 1–5. ACM Press, Denver (2019). https://doi.org/10.1145/3314111.3319816
Kardan, O., et al.: Cultural and developmental influences on overt visual attention to videos. Sci. Rep. 7(1), 11264 (2017). https://doi.org/10.1038/s41598-017-11570-w
Kaspar, K., König, P.: Emotions and personality traits as high-level factors in visual attention: a review. Front. Hum. Neurosci. 6, 321 (2012)
Kempinski, Y.: System and Method of Diagnosis Using Gaze and Eye Tracking (Patent No.: US20160106315A1) (2016)
Koć-Januchta, M., et al.: Visualizers versus verbalizers: effects of cognitive style on learning with texts and pictures – an eye-tracking study. Comput. Hum. Behav. 68, 170–179 (2017). https://doi.org/10.1016/j.chb.2016.11.028
Krafka, K., et al.: Eye tracking for everyone. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2176–2184. IEEE, Las Vegas (2016)
Kredel, R., et al.: Eye-tracking technology and the dynamics of natural gaze behavior in sports: a systematic review of 40 years of research. Front. Psychol. 8, 1–15 (2017)
Kröger, J.: Unexpected inferences from sensor data: a hidden privacy threat in the Internet of Things. In: Strous, L., Cerf, V.G. (eds.) IFIPIoT 2018. IAICT, vol. 548, pp. 147–159. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-15651-0_13
Kröger, J.L., et al.: Privacy implications of accelerometer data: a review of possible inferences. In: Proceedings of the 3rd International Conference on Cryptography, Security and Privacy (ICCSP). ACM, New York (2019). https://doi.org/10.1145/3309074.3309076
Kröger, J.L., et al.: Privacy implications of voice and speech analysis - information disclosure by inference. In: Fricker, S., et al. (eds.) Privacy and Identity 2019. IFIP AICT, vol. 576, pp. 242–258. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-42504-3_16
Kröger, J.L., Raschke, P.: Is my phone listening in? On the feasibility and detectability of mobile eavesdropping. In: Foley, S.N. (ed.) DBSec 2019. LNCS, vol. 11559, pp. 102–120. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-22479-0_6
Kunze, K., et al.: Towards inferring language expertise using eye tracking. In: CHI 2013 Extended Abstracts on Human Factors in Computing Systems, pp. 217–222. ACM Press, Paris (2013). https://doi.org/10.1145/2468356.2468396
Lallé, S., et al.: Prediction of users’ learning curves for adaptation while using an information visualization. In: International Conference on Intelligent User Interfaces, pp. 357–368. ACM Press, Atlanta (2015)
Larsson, M., et al.: Associations between iris characteristics and personality in adulthood. Biol. Psychol. 75(2), 165–175 (2007). https://doi.org/10.1016/j.biopsycho.2007.01.007
Lea, R.G., et al.: Trait emotional intelligence and attentional bias for positive emotion: an eye tracking study. Pers. Individ. Differ. 128, 88–93 (2018)
Lemos, J.: System and Method for Determining Human Emotion by Analyzing Eye Properties (Patent No.: US20070066916A1) (2007)
Li, Y., et al.: Towards measuring and inferring user interest from gaze. In: International Conference on World Wide Web Companion, pp. 525–533. ACM Press, Perth (2017). https://doi.org/10.1145/3041021.3054182
Liang, C.-C., et al.: System for Monitoring Eyes for Detecting Sleep Behavior (Patent No.: US5570698A) (1996)
Liebling, D.J., Preibusch, S.: Privacy considerations for a pervasive eye tracking world. In: International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication, pp. 1169–1177 ACM Press, New York (2014)
Macatee, R.J., et al.: Attention bias towards negative emotional information and its relationship with daily worry in the context of acute stress: an eye-tracking study. Behav. Res. Ther. 90, 96–110 (2017). https://doi.org/10.1016/j.brat.2016.12.013
Mader, G.: Declaration of Schlaining Against Racism, Violence and Discrimination. Austrian Commission for UNESCO, Vienna (1995)
Marcos, M.-C., et al.: Cultural differences on seeking information: an eye tracking study. In: CHI 2013: Workshop Many People, Many Eyes. ACM, Paris (2013)
Marks, K.R., et al.: Fixation time is a sensitive measure of cocaine cue attentional bias. Addict. Abingdon Engl. 109(9), 1501–1508 (2014). https://doi.org/10.1111/add.12635
Martins, R., Carvalho, J.: Eye blinking as an indicator of fatigue and mental load—a systematic review. In: Arezes, P., et al. (eds.) Occupational Safety and Hygiene III, pp. 231–235. CRC Press (2015). https://doi.org/10.1201/b18042-48
Matey, J.R., et al.: Iris on the move: acquisition of images for iris recognition in less constrained environments. Proc. IEEE 94(11), 1936–1947 (2006)
McGrath, D.S., et al.: The specificity of attentional biases by type of gambling: an eye-tracking study. PLoS ONE 13(1), e0190614 (2018)
Milanak, M.E., et al.: PTSD symptoms and overt attention to contextualized emotional faces: evidence from eye tracking. Psychiatry Res. 269, 408–413 (2018)
Miller, M.A., Fillmore, M.T.: Persistence of attentional bias toward alcohol-related stimuli in intoxicated social drinkers. Drug Alcohol Depend. 117(2), 184–189 (2011)
Moss, F.J.M., et al.: Eye movements to natural images as a function of sex and personality. PLoS ONE 7(11), e47870 (2012). https://doi.org/10.1371/journal.pone.0047870
Peterson, J., Pardos, Z., Rau, M., Swigart, A., Gerber, Colin, McKinsey, J.: Understanding student success in chemistry using gaze tracking and pupillometry. In: Conati, C., Heffernan, N., Mitrovic, A., Verdejo, M.F. (eds.) AIED 2015. LNCS (LNAI), vol. 9112, pp. 358–366. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-19773-9_36
Pettiford, J., et al.: Increases in impulsivity following smoking abstinence are related to baseline nicotine intake and boredom susceptibility. Addict. Behav. 32(10), 2351–2357 (2007). https://doi.org/10.1016/j.addbeh.2007.02.004
Price, R.B., et al.: From anxious youth to depressed adolescents: prospective prediction of 2-year depression symptoms via attentional bias measures. J. Abnorm. Psychol. 125(2), 267–278 (2016). https://doi.org/10.1037/abn0000127
Raptis, G.E., et al.: Using eye gaze data and visual activities to infer human cognitive styles: method and feasibility studies. In: Conference on User Modeling, Adaptation and Personalization (UMAP), pp. 164–173. ACM Press, Bratislava (2017)
Raudonis, V., et al.: Evaluation of human emotion from eye motions. Int. J. Adv. Comput. Sci. Appl. 4(8), 79–84 (2013). https://doi.org/10.14569/IJACSA.2013.040812
Ravi, B.: Privacy Issues in Virtual Reality: Eye Tracking Technology. Bloomberg Law, Arlington County (2017)
Reingold, E., Sheridan, H.: Eye movements and visual expertise in chess and medicine. In: Liversedge, S.P., Gilchrist, I.D., Everling, S. (eds.) The Oxford Handbook of Eye Movements, pp. 528–550. Oxford University, Oxford (2011)
Rigas, I., et al.: Biometric recognition via eye movements: saccadic vigor and acceleration cues. ACM Trans. Appl. Percept. 13(2), 1–21 (2016)
Roche, D.J.O., King, A.C.: Alcohol impairment of saccadic and smooth pursuit eye movements: impact of risk factors for alcohol dependence. Psychopharmacology 212(1), 33–44 (2010). https://doi.org/10.1007/s00213-010-1906-8
Sears, C.R., et al.: Attention to emotional images in previously depressed individuals: an eye-tracking study. Cogn. Ther. Res. 35(6), 517–528 (2011)
Sperling, I., et al.: Cognitive food processing in binge-eating disorder: an eye-tracking study. Nutrients 9(8), 903 (2017). https://doi.org/10.3390/nu9080903
Steil, J., et al.: Privacy-aware eye tracking using differential privacy. In: ACM Symposium on Eye Tracking Research & Applications, pp. 1–9 (2019). https://doi.org/10.1145/3314111.3319915
Szymanska, M., et al.: How do adolescents regulate distress according to attachment style? A combined eye-tracking and neurophysiological approach. Prog. Neuropsychopharmacol. Biol. Psychiatry 89, 39–47 (2019). https://doi.org/10.1016/j.pnpbp.2018.08.019
Telford, E.J., et al.: Preterm birth is associated with atypical social orienting in infancy detected using eye tracking. J. Child Psychol. Psychiatry 57(7), 861–868 (2016)
Thieberger, G., et al.: Utilizing Eye-tracking to Estimate Affective Response to a Token Instance of Interest (Patent No.: US9569734B2) (2017)
Tobii: Tobii Pro wearable eye tracking for driver safety. https://www.tobiipro.com/fields-of-use/psychology-and-neuroscience/customer-cases/audi-attitudes/. Accessed 13 Sept 2019
Vidal, M., et al.: Wearable eye tracking for mental health monitoring. Comput. Commun. 35(11), 1306–1311 (2012). https://doi.org/10.1016/j.comcom.2011.11.002
Wang, C.-Y., et al.: Multimedia recipe reading: predicting learning outcomes and diagnosing cooking interest using eye-tracking measures. Comput. Hum. Behav. 62, 9–18 (2016)
Wenzlaff, F., et al.: Video-based eye tracking in sex research: a systematic literature review. J. Sex Res. 53(8), 1008–1019 (2016)
Wu, E.X.W., et al.: Through the eyes of the own-race bias: eye-tracking and pupillometry during face recognition. Soc. Neurosci. 7(2), 202–216 (2012)
Yamada, Y., Kobayashi, M.: Fatigue detection model for older adults using eye-tracking data gathered while watching video: evaluation against diverse fatiguing tasks. In: 2017 IEEE International Conference on Healthcare Informatics (ICHI), pp. 275–284 (2017). https://doi.org/10.1109/ICHI.2017.74
Yoon, J.H., et al.: Assessing attentional bias and inhibitory control in cannabis use disorder using an eye-tracking paradigm with personalized stimuli. Exp. Clin. Psychopharmacol. (2019). https://doi.org/10.1037/pha0000274
Zamani, H., et al.: Eye tracking application on emotion analysis for marketing strategy. J. Telecommun. Electron. Comput. Eng. 8(11), 87–91 (2016)
Zhan, Z., et al.: Online Learners’ reading ability detection based on eye-tracking sensors. Sensors 16(9), 1457 (2016). https://doi.org/10.3390/s16091457
Fourth Quarter and Full Year 2018 Results. Facebook, Inc., Menlo Park, USA (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2020 The Author(s)
About this chapter
Cite this chapter
Kröger, J.L., Lutz, O.HM., Müller, F. (2020). What Does Your Gaze Reveal About You? On the Privacy Implications of Eye Tracking. In: Friedewald, M., Önen, M., Lievens, E., Krenn, S., Fricker, S. (eds) Privacy and Identity Management. Data for Better Living: AI and Privacy. Privacy and Identity 2019. IFIP Advances in Information and Communication Technology(), vol 576. Springer, Cham. https://doi.org/10.1007/978-3-030-42504-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-030-42504-3_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-42503-6
Online ISBN: 978-3-030-42504-3
eBook Packages: Computer ScienceComputer Science (R0)
-
Published in cooperation with
http://www.ifip.org/