Eye Tracking: Eye-Gaze Technology

Abstract

With ongoing advancements in eye-tracking technology, more individuals with complex communication needs (CCN) and who have limited arm and hand movements are exploring the use of eye-gaze technology for accessing their augmentative and alternative communication (AAC) devices. This chapter outlines the various factors that need to be taken into account when evaluating an individual’s ability to use this technology; many of these relate to the roles and functions of an occupational therapist as a member of a multidisciplinary team. Clinical research examining the suitability of eye-gaze technology for various populations is limited; documentation of its effectiveness is based on individual’s testimonials or reports. A case study highlights the role of the occupational therapist in conducting eye-gaze trials with a girl with Rett syndrome.

I like the Tobii eye-gaze system, because it’s fast and accurate. When I used a switch access device, it would take me at least twice as long to find my comments. With Tobii, my comments are more accessible, so it’s easier to say what I mean and be myself. Gabriela Cellini.

Also see Steve Gleason—former National Football League (NFL) Player who lives with Amyotrophic Lateral Sclerosis (ALS) and is using a Tobii EyeMobile with a Microsoft Surface. He was featured in a Microsoft advertisement during the Super-bowl in Feb 2014. Here is a link to his video “No White Flags” https://www.youtube.com/watch?v=JObFlEvc-Eg

Appendices

A Case Study of Shri Living with Rett Syndrome

Keywords

Access, augmentative and alternative communication (AAC), communication, eye-gaze technology, Rett syndrome

Introduction

The theme of this case study concerns the occupational therapist’s (OT) role in evaluating the features of an eye-gaze system, including its setup and configuration, for supporting the access and communication needs of children or young adults with Rett syndrome (RTT). Rett syndrome is a rare genetic postnatal neurological disorder that occurs almost exclusively in girls, but can be rarely seen in boys. Apraxia and severe physical challenges affect their ability to speak, walk, eat, and in many cases, use of their hands. Cognitive assessment in children with Rett syndrome is extremely difficult; recent eye-tracking research and detailed accounts of what girls and women are expressing spontaneously using an augmentative and alternative communication (AAC) device with eye-gaze technology are providing insight into their true communication and learning potential.

Moreover, this case study demonstrates the role of OTs within a multidisciplinary team in terms of identifying those tasks or activities in which a child with RTT may use an AAC device with eye-gaze technology to actively engage throughout the day, both at home and at school. In these situations, the eye-gaze system represents a means for a girl to use communication as a means for participating rather than relying on more passive hand-over-hand physical assistance during these activities.

The occupational therapy interventions (OTIs) addressed in this case study also have applications to other children and adults with CCN and severe motor disabilities.

The students’ tasks include:

• 1. To recognize the unique role of an OT as a member of a multidisciplinary team in analyzing an individual’s natural range of eye movements, field of vision, length of visual fixation, head and body movements, and how this information can be used to identify the most suitable eye-gaze system.

• 2. To identify the importance of seating and positioning in relation to eye-gaze technology , the positioning of a person’s eyes relative to the “track box” of an eye-gaze accessory, the length of dwell and visual feedback settings for effective use of an eye-gaze system, as well as strategies for successful calibration of the system.

• 3. To identify how a child or young adult with RTT can use an eye-gaze system to socially interact with communication partners and direct their actions in completing various classroom and leisure activities rather than always relying on participation through hand-over-hand physical assistance.

As a starting point, students should use the following references to gather background information:

• Brooks M (2012) Eye-gaze technology helping to reach Rett syndrome patients. Medscape News: Neurology. http://www.medscape.com/viewarticle/763464. Accessed 26 Feb 2014

• Lariviere J (2007) Exploring options for access: enhancing communication and learning for girls with Rett syndrome. TSIS Quart 17(4):1–4. http://store.ablenetinc.com/press/news/sp_int_sec_quarterly_12_07 s.pdf. Accessed 26 Feb 2014

• Liu H, Chuang H (2011) An examination of cognitive processing of multimedia information based on viewers’ eye movements. Interact Learn Envir 19(5):503–517. doi:10.1080/10494820903520123

• International Rett Syndrome Foundation (2014) What is Rett syndrome? Understanding Rett syndrome. http://www.rettsyndrome.org/understanding-rett-syndrome/about-rett-syndrome/what-is-rett-syndrome. Accessed 26 Feb 2014

• Miesenberger KK, Adriaens LL, Gelderblom G (2011). Everyday technology for independence and care: AAATE 2011. IOS Press, Amsterdam

• National Institute of Neurological Disorders (2014) Rett syndrome fact sheet. http://www.ninds.nih.gov/about_ninds/message/message-BRAIN-funding.htm. Accessed 26 Feb 2014

• Shane HC, Blackstone S, Vanderheiden G, Williams M, DeRuyter F (2012) Using AAC technology to access the world. Assist Technol 24(1):3–13. doi:10.1080/10400435.2011.648716

• Tobii Technology AB (2010) Tobii eye tracking: an introduction to eye tracking technology and Tobii eye trackers. Tobii Technology 
Whitepaper January:3e11 http://www.tobii.com/Global/Analysis/Training/WhitePapers/Tobii_EyeTracking_Introduction_WhitePaper.pdf. Accessed 26 Feb 2014

• King-DeBaun P, Skinner R (2012) Jumpstart AAC for students with severe and multiple disabilities. Closing the Gap Solutions October/November:10–14

Overview of the Content

One of the major goals of the OT’s intervention was to identify an eye-gaze system that was easy for Shri to access using her natural range of eye movements, quick to set up and use in her classroom environment, and provided her with efficient ways to communicate her physical and emotional needs related to her sensory regulation. In conjunction with Shri’s AAC specialist and parents, the OT also played a role in setting up and configuring the settings and design of the vocabulary pages on the eye-gaze systems being trialed so that these would support Shri in her communication. The OT played a role in supporting how Shri used her eye-gaze system in different settings to communicate and actively learn and participate in classroom activities, and have independent access to some leisure activities at home based on her interests.

Learning Objectives

• 1. To identify those factors to consider in identifying the best positioning and configuration of the eye-gaze technology that is most effective for Shri, and others with Rett syndrome, during trials with different systems and when using her own AAC device with eye-gaze technology.

• 2. To recognize the role that Shri’s natural range of eye movements play in customizing vocabulary pages in terms of size of buttons and page layout to support ease of access.

• 3. To identify those activities in which Shri and other children with RTT can actively participate through social interaction or communication using an AAC device with eye-gaze technology throughout her day.

The Background History of the Clinical Case Study

Personal Data

Shri was a 9-year-old girl with Rett syndrome. She lived at home with her mom, dad, and older brother. She attended an elementary school where she was enrolled in a third-grade classroom with other students with disabilities.

Shri was nonverbal and her eyes represented her best means of access to a communication device. She used a Prentke Romich Company (PRC) ECO2 with ECOPoint with Unity 45 Sequenced (Fig. 23.8). Shri received funding for this speech-generating device with eye-gaze accessory through insurance based on the recommendation from her AAC specialist. An OT was not involved in the trials and selection of this device.

Fig. 23.8

Prentke Romich Company (PRC) ECO2 with ECOPoint with Unity 45 Sequenced vocabulary/language system. (Photo taken by Judy Lariviere)

Prior to this time, Shri used a communication device, Tango, which had a dynamic touch-screen display consisting of six vertical buttons arranged horizontally across the middle of the device (Fig. 23.9). Shri had limited hand function , which is why she did not consistently have sufficient control and strength to press the larger arrow buttons on the Tango for navigating between topics. Although she could use a single switch with her left hand in combination with automatic auditory scanning set to only scan the arrows and outer buttons, this method of access was found to significantly slow down her interactions with her family members and her classmates. As a result, she often directed her communication partner to change between topics or to different areas of the Tango vocabulary using partner assisted scanning. Her communication partner would say out loud the different choices to which Shri could navigate, such as “more choices; new topic; home; pop talk,” etc. and when Shri heard the choice she wanted, she would give one of her “yes” responses to her communication partner.

Fig. 23.9

Tango AAC device. The horizontally arranged six buttons in the middle of the device were activated through a touch display or screen. The arrows surrounding the screen supported navigation between the dynamic display screens for accessing additional vocabulary on the same topic and for changing topics. The outer buttons provided access to the home screen, alphabet, volume control, and PopTalk, which included high-frequency, quick phrases. (Photo: Judy Lariviere)

Disability

Shri’s motor and speech disability. She was not able to walk independently, but she walked with support of an adult from behind her for short distances and was pushed in a manual wheelchair for traveling longer distances at school or in the community. She could sit unsupported on the floor in ring sitting or using a supportive seating insert secured to a wooden chair. At school, she sat in a wooden chair with adaptations and standby supervision. Shri was unable to speak, thus she was introduced at a young age to AAC devices, so that she had access to a voice to express herself. Shri could use her full left hand to activate raised and large buttons on her musical toys when highly motivated as this emotional engagement helped her to overcome her apraxia or challenges with motor planning. She did not have the ability to grasp and hold objects or isolate her index finger to touch or press keys on a keyboard or iPad. Shri relied on using her eyes with an intense stare to connect socially with people in her life. She had various ways to communicate a “yes” response or to indicate her agreement with a statement including looking at her communication partner, raising her eyebrows, vocalizing, smiling, and increasing her rate of breathing. In addition, Shri used various means for giving a “no” response by looking away or keeping her body very still.

Reason for Seeking Occupational Therapy

Given Shri’s motor challenges in accessing all areas of the screen using her eyes with her ECO2 with ECOPoint in combination with the limited gains she demonstrated in her communication abilities, Shri’s mom requested the assistance of an OT with an expertise knowledge in assistive technology to explore Shri’s access to an AAC device incorporating eye-gaze technology . In collaboration with Shri’s new (AAC) specialist who was also a speech language pathologist, trials were conducted with various AAC devices incorporating eye-gaze technology to determine the communication device that would support Shri’s ease of access using her eyes.

When working with Shri using her ECOPoint, the OT identified that she experienced difficulty in accessing buttons on the right side of the screen with the most difficulty in selecting buttons in the lower right corner. These observations were consistent with her parents’ report. Shri’s parents had taken her to an ophthalmologist who determined that she did not have any visual field cuts or other undiagnosed problems with her vision. It was identified that Shri had many sight words and the 2-hit Unity did not build on her ability in this area. As text was not displayed on the buttons with this language system, Shri had to make her selections based on the meaning of the icons and their location. It was also noted that combining words together in the correct sequence to access single words was very physically demanding for her, based on her apraxia. As an interim measure to increase Shri’s success in her interactions with her communication partners at home and at school, the ECOPoint version of The Dynamic Communication Book for Girls was programmed onto her eye-gaze device (Fig. 23.10). This comprehensive communication system was developed by Pati King-DeBaun based on her integrated model of communication instruction.

Fig. 23.10

Screenshot of start page of Pati King-DeBaun’s Dynamic Communication Book © 2010 Creative Communicating, set up in Prentke Romich Company’s ECO2 PASS Demo software. © 2014 Prentke Romich Company. (Screenshot taken by Judy Lariviere)

Shri already had previous experience in using an eye-gaze system, so she was given access to the Dynamic Communication Book for Girls. The layout was customized in the same way that her OT had already developed for girls with RTT. This layout was based on the girls’ natural eye movements from left to right across a page and their ease in looking either up or down (Fig. 23.11). Shri was first introduced to Tobii’s EyeMobile because of its smaller screen size and the ability of this eye-gaze system to keep tracking one’s eyes with head and upper body movement.

Fig. 23.11

Tobii EyeMobile with main page from Pati King DeBaun’s Dynamic Communication Book for Girls © 2010 Creative Communicating in © 2009–2014 Lariviere Eye Gaze Layout in Tobii Communicator. © 2013 Tobii Technology. The EyeMobile was supported on a Daessy C-Shaped Desktop Mount. © 2012 Daedalus Technologies, Inc. (Photo: Judy Lariviere)

Given Shri’s apraxia, it was difficult for her get a good calibration with the EyeMobile.

Calibration is the process during which an individual needs to focus their visual attention for a few seconds on visual stimuli, displayed in the form of a circle, image, or small video, at different locations on the screen in a predetermined sequence for a 5- or 9-point calibration of the eye-gaze accessory. Therefore, her mom calibrated the eye-gaze systems during the trials, because her mom’s eyes represented a close genetic match to Shri’s eyes and it was important for Shri to use a good calibration for improved accuracy in her selections in all areas of the screen. Each time Shri used the EyeMobile, Shri’s eyes were first located in the Track Status window for optimal positioning of the eye-gaze system in relation to her eyes. The dwell, the setting that determines how long Shri needed to look at a button in order to select it, was set to approximately 397 ms given she experienced difficulty in sustaining her gaze longer than this to select the button with her desired message. The visual feedback was also set to a red clock so that her communication partners could see when she was looking at a specific button but not long enough to select it using the dwell setting so that her communication intent and motor approximations could be acknowledged (Fig. 23.12).

Fig. 23.12

Screenshot of visual feedback settings for Gaze Interaction used with Shri on Tobii eye-gaze systems. © 2013 Tobii Technology. (Photo of Screenshot: Judy Lariviere)

Shri had the opportunity to use the EyeMobile for a 2-week period at home. She also had a few opportunities to try out using the EyeMobile at school. Shri’s access to and use of the Tobii I-12 with a dwell of 600 ms was also evaluated over a 2-week trial period at both home and school, as these represented the primary locations where she would be using these AAC devices to interact with her family members, teachers, therapists, and friends (Fig. 23.13).

Fig. 23.13

Tobii I-12 with main page from Pati King DeBaun’s Dynamic Communication Book for Girls in © 2009–2014 Lariviere Eye Gaze Layout in Tobii Communicator © 2013 Tobii Technology. The Tobii I-12 was supported on a Daessy C-shaped desktop mount. © 2012 Daedalus Technologies, Inc. (Photo taken by Judy Lariviere)

The screen Shri used was slightly larger on the Tobii I-12; Shri was observed to experience more difficulty in accessing the buttons on the far right of the screen. The option of reducing the size of the displayed pages within Tobii Communicator was used to decrease the height and width of the page set to 80 % and position it in the top left corner of the screen (Fig. 23.14).

Fig. 23.14

Screenshot of page set properties feature in Tobii Communicator showing settings for reduced Window view with the docked position of 80 % width and height in the top left corner of the screen. © 2013 Tobii Technology. (Screenshot taken by Judy Lariviere)

Using this feature, the page set was shifted to the top left of the screen to accommodate Shri’s reduced visual field when using the page set so that she could easily access the buttons positioned on the right side of the screen (Fig. 23.15). In addition, the Tobii I-12 was positioned to the left of midline to build upon Shri’s ease in looking to the left and her increased difficulty in accessing buttons on the right side of the screen. These adjustments assisted her in accessing all of the buttons on the page so that her access supported her communication.

Fig. 23.15

Screenshot of Window view of Tobii Communicator with the height and width of the page-set set to 80 % relative to the screens size and positioned in the top left corner of the screen. Page set properties feature in Tobii Communicator showing settings for reduced Window view with the docked position of 80 % width and height in the top left corner of the screen. © 2013 Tobii Technology. (Screenshot taken by Judy Lariviere)

Current Circumstances

Shri received Tobii I-12 after funding was approved for this device. Shri was given access to Pati King-DeBaun’s Dynamic Communication Book that was programmed in Lariviere’s eye-gaze layout,¹ which supported ease of access for girls with Rett syndrome. Through her use of the Dynamic Communication Book, Shri had access to vocabulary for greetings, expressing feelings, opinions, indicating when she needed or wanted something or someone, asking questions, and participating in activities for home such as directing another person in reading a book, cooking, playing with a doll, watching television, going shopping, etc.

Occupational Performance Issues

Because Shri had limited hand use, she required assistance with her work related to learning and participation in various classroom activities (i.e., reading, writing, art, and music), leisure in independently accessing books she could read silently or have read to her, access to music, and music videos. More specifically, she required hand-over-hand assistance to pick up and hold a crayon or pencil and make marks on the page but did not have a way to actively participate in art class or other classroom-based activities involving arts and crafts. She also experienced difficulty in turning pages of a book because of her hand function but showed an interest in independently accessing books she could read on her own. Shri was developing her independent writing skills using technology; words that she spelled phonetically with a communication partner using partner-assisted scanning with an alphabet flipbook, represented close approximations to conventional spelling. She also loved music, but did not have a way to independently access and play a selection of songs, like other children using an MP3 music player.

The Student’s Report

The following guiding questions have been identified in developing possible solutions:

Questions

• 1. Based on the background information about Shri, what are some important considerations for her from a vision and positioning perspective that will support her ease of access to her Tobii eye-gaze system?

• 2. Taking into account Shri’s occupations at home and school, what areas in the near future would you recommend be included or programmed into her Tobii eye-gaze system to help her fulfill these roles?