1 Introduction

Children with hearing impairment are at high risk for language, academic and social difficulties [1]. Fortunately, the advancement of medical technology (e.g., hearing aid, cochlear implant) has significantly changed the prospects of children with hearing impairment. As Levy (2012) explained, with reference to the invention of the cochlear implant, there is now a device that allows a large proportion of deaf people to hear; this has dramatically changed the deaf community and deaf education [2]. Because of the advancement of medical technology, the education of children with hearing impairment has evolved into a change in speech therapy, educational audiology, and special education [3]. Educational programs that serve hearing impaired children have changed and keep changing, especially since most of these children are now placed in mainstream schools rather than schools for the deaf in the developed countries [4]. Regional school administration and councils have stopped training programs for children with hearing impairment, on the idea that these students, supported by medical technology, should not be educated separately; instead, they are included to general/mainstream classroom whilst they receive additional therapy and support from speech therapists and educational audiologists [3]. Despite the medical technology advancement, amplification itself does not provide optimal development of hearing and speaking; it must be combined with a specialized program for early intervention [5].

The focus of this study is on the integration of an intelligent robot, NAO, as a supportive tool for learning through play in this early intervention, particularly in auditory-verbal therapy for hearing impaired children, an approach to the development of auditory and verbal skills that does not allow lipreading or other non-verbal cues to facilitate communication. In general, play is crucial for the communication, cognitive, physical, social, and emotional development of young children [6]. While initiating and taking part in playful activities children are motivated to communicate and practice their language [7]. For this reason, play is extensively used in therapy and education. There are many studies exploring play and language learning in groups of children with hearing disorders. Some studies are related to play and deaf children in special schools [8]; other studies compare the ability of children with hearing impairment to play with their hearing peers [9,10,11,12,13]. NAO is well documented as a tool that can be used successfully in learning and therapy through play (e.g., [14,15,16,17,18], although not with hearing impaired children.

Furthermore, NAO has a unique characteristic and a great affordance in the context of the present study. The robot does not have a human mouth and therefore children with hearing impairment cannot do lipreading. The unique feature of the technology has been identified by the authors of this work (an educational technologist and a speech therapist - researcher and practitioner) and has been used in this study to create a unique environment for auditory-verbal therapy sessions for children with hearing impairment. This study was inspired by a true need for engaging hearing-impaired children in auditory-verbal therapy sessions, as identified during previous work of the authors [19] as well as practical experience of the therapists involved. The present manuscript focuses on:

  1. 1.

    What are the children’s gains in auditory skills, name the ability of the hearing-impaired child to detect sounds and follow instructions?

  2. 2.

    What is the perceived value of the robot-enhanced environment for auditory-verbal therapy?

The rest of this manuscript begins with an overview of deaf studies and technology. Then, we elaborate on the methods and procedures of the study, including details on the robot-enhanced activities. We finally present our findings followed by a discussion of implications for future research and practice.

2 State of the Art

2.1 Therapy of Children with Hearing Disorder

Humans use speech to share information, to predict and explain behavior such as desires, beliefs, and feelings [20]. Young children learn to participate in communication situations through listening to the speech of their parents and caregivers. The first years of life are crucial for language development, while about 85% of neural development happens till the third year of life. Early access to the auditory brain is of importance for developing spoken language adequate to the norm for certain age [21]. If mild or partial hearing loss occur in this sensitive period, it could result in delay of language development.

Today, there are about 34 million children with hearing loss worldwide [22]. Various factors affect the communication skills of hearing impaired children, ranging from the degree of hearing loss, age of onset of hearing loss, age of identifying the child’s hearing disorder, etiology of the disorder, adequacy of intervention, the presence of other disabilities, age of receiving intervention, type of intervention program, family and environmental influences, consistency of use of medical technology (e.g., hearing aid, cochlear implant), and the attitudes of the child or parents [7, 9,10,11,12,13].

One of the main issues the family of a deaf child has is whether to use manual or verbal communication with their child [23]. Initially the parents of children with hearing impairment and later the children themselves can choose a communication approach to learn speech and language. Parents seek information from the professionals (medical doctors, audiologist, speech and language pathologist, and teacher) about the communication method. Those who choose the oral communication method expect from their child an ability to fit in with typically developing children [24]. According to Gallaudet Research Institute report (2008), around 96% of hearing impaired children have hearing parents [25]. Consequently, hearing parents want to give the opportunity to their child with hearing impairment to communicate verbally. Unfortunately, there is no data that clearly illustrates the best communication approach [20, 24].

According to Harris (2014), the degree of a child’s hearing loss has been a primary factor used by parents to decide which communication modality is applicable for their child. Usually the parents of children with mild or moderate hearing loss choose oral mode (i.e., listening and spoken language) whereas with more severe losses, parents opt for a manual form of communication such as Sign Language and finger spelling. Socialization, academic achievement and self-esteem may also influence the decision of the parents regarding the communication modality [11]. Researchers in Australia [26] have investigated the way of decision making about the communication mode in families of deaf children. They found out that most of the children with hearing impairment use oral communication; only one third have experience using a type of non-verbal communication mode. The caregivers choose the communication mode depending on source of information – professionals, family and friends, own research by parents; child individual needs and preferences; accessibility of communication – languages and communication modality used in family and community; access to intervention; audiological characteristics; child’s future life [26, 27]. The aim of all communication approaches is to give the opportunity to the child with hearing impairment to develop sufficient skills to be able to communicate.

2.2 Medical Technology and Therapy of Hearing Impaired

The advancement of medical technology has significantly improved the prospects of children with hearing impairment. Amplification via electronic devices such as hearing aids or cochlear implants can provide a sense of sound to hearing impaired people, although they do not restore hearing loss. In any case, speech therapy and early intervention upon use of an appropriate aid (hearing aid or cochlear implant) is a major factor for development of the child’s communication skills.

Hearing aid amplifies sounds and makes them louder. The microphones of the device pick up the sounds and after the amplification, they are sent into the ear canal, through the middle ear to the cochlear where the hair cells are activated and provide sound signals to the brain. Hearing aid is suitable for people with mild to severe hearing loss. Cochlear implant is surgically implanted electronic medical device. Cochlear implants are suitable for people with severe-to-profound sensorineural hearing loss, who cannot benefit from hearing aid. It has two main parts – internal (receiver/simulator) and external (speech processor). The internal part is implanted by surgery under the skin behind the ear. Electrodes are placed by medical professionals into the cochlear to stimulate the auditory nerve. Cochlear implant substitutes the function of the damaged hair cells in the inner ear and directly stimulates the auditory nerve that provides the signals to the brain. The external part is behind the ear. It is attached via transmitter cord to the transmitter coil, which magnetically attaches itself to the internal receiver [20, 28]. Children with successful surgery could hear the whole range of speech sounds and they have the possibility to learn and use the verbal speech; however, it is difficult to predict whether the cochlear implant(s) surgery will be successful or not [29]. Then, there are situations when the surgery is successful but the child is not using the implant due to other communication preferences, lack of family support, psychological weakness to accept the implant, the school that the child is attending, the difficulty to get used to the signal of the implant [11]. Resent research has shown that cochlear implant children trained without the use of sign language have shown significant advantage in their narratives, range of vocabulary and use of expressions, and in the complexity of the syntax used in their language [30].

Assuming an appropriate aid (hearing aid, cochlear implants) is used, support from speech therapists and educational audiologists in the early years of age is critical. This puts strong emphasis on the options of auditory-oral and auditory-verbal therapy approaches to development of verbal communication skills for the child who uses amplification [31]. In the auditory-oral approach, the child listens to others while s/he pays attention to lipreading (i.e. receiving visual information from the lips) and body language. Instead, the auditory-verbal method is a very strict approach that does not allow the use of lipreading or other non-verbal cues (e.g. gestures) to facilitate communication [20]. Auditory-verbal approach is child-directed, based on the child’s interests and using play. It includes strategies that encourage joint attention, turn-taking, thus facilitates the development of listening skills, speech and language [32]. In this case, during therapy sessions, speech therapists stay in position which does not allow lipreading to force the child use his/her amplification device (hearing aid or cochlear implant) to hear the speech sounds. This enables children to make auditory images of what they hear in their brains and to create neural pathways for speech and language development [30, 33].

Overall, research of Cole and Flexer (2015) has shown that learning through listening is the most effective way of developing spoken language and cognition [30]. Thus, the auditory-verbal method is a very attractive in early intervention and therapy of children with all kind of amplification (e.g., hearing aid, cochlear implant). A technique that has been associated with auditory-verbal therapy is covering the therapist’s mouth with his/her hand to eliminate lipreading. In their recent work, Estabrooks et al. (2016) encourage therapists to act more naturally during auditory-verbal therapy. They argue that covering the mouth might disrupt sensorimotor input during infancy and may have negative implications for the development of speech motor control; it can also provoke stress in young children which, in turn, negatively affects speech perception. The listening skill should be stimulated, but therapists should find ways to eliminate lipreading without having to cover their mouth, a rather atypical act in social interaction [32]. The humanoid robot NAO has a unique characteristic in this case - it does not have a human mouth and therefore children with hearing impairment cannot do lipreading. This unique feature of the technology has been identified by the researchers and has been used in this work to create a unique environment for auditory-verbal therapy sessions that can be playful and effective.

2.3 Play and Learning with Robots

Play has an adaptive function when children explore new situation, new environment, like games with a humanoid robot. It could support the development of sensorimotor skills and cognitive development – in our case stimulating the listening skills and the language [44].

The published studies about use of intelligent robots in interaction with children with and without disabilities are rapidly increasing during the past decade. Some of them explore the human-robot interaction in play scenarios with typically developing children (e.g., [16, 34]). Others investigate play and learning with intelligent robots in the therapy of children with disabilities, e.g., diagnosed with cerebral palsy [35] or Autism Spectrum Disorder (ASD) [14, 17, 18, 36,37,38]. One example is the program known as ASKNAO (Autism Solution for Kids) by Aldebaran Robotics, which uses NAO as a tool to help ASD children; the program has supported ASD children in learning about the social and communicative skills they lacked [39]. Intelligent humanoid robots show effectiveness with ASD children because they are simple and predictable with basic conversational function; they can be used as mediators to interact with the professional in therapy sessions, with the parents at home or with their peers at school [18]. Overall, research on the use of robots for play and learning shows that children (with disabilities or not) tend to interact with the intelligent robot smoothly, accept it as a peer in play, and are willing to initiate communication with the robot.

Studies on using intelligent robots with deaf children are scarce. Sawada et al. (2008) used a robotic voice simulator for speech therapy with hearing impaired children; the invented robot consisted of motor-controlled vocal organs (vocal cords), a vocal tract and a nasal cavity to generate a natural voice imitating a human vocalization. The robot reproduced the vocalization of the deaf children and taught them how to generate clearer speech through repeating the correct vocalization [40]. Moreover, NAO has been used in play activities for the assessment of listening and speaking skills of seven hearing-impaired students who use cochlear implant(s) deaf children [19]. A couple of other studies used intelligent robots with deaf children for manual communication. One such study involved NAO to assist a story telling by showing signs from the Turkish sign language, therefore facilitating non-verbal communication [41]. Then, unlike NAO that has only three fingers and thus, limited ability to present all signs used in manual communication, [42] used a robotic platform with five fingers (Robovie R3) for teaching sign language to deaf participants. There are virtually no studies focused on the use of intelligent robots in therapy of children with hearing impairment aiming specifically at the development of auditory and verbal communication skills.

3 Methodology

This was a mixed-method study which made use of quantitative and qualitative data to realize gains in the ability of the hearing-impaired child to detect sounds and follow instructions as well as the value of the robot-enhanced experience based on the observations of the participating speech therapist and special teacher.

3.1 Participants and Setting

The study took place in a mainstream public kindergarten-preschool in an Eastern Mediterranean country. This kindergarten-preschool (2–5 years old children) has the largest percentage of children with hearing impairment of the country. It is located next to and shares facilities and staff with the single school for the deaf in the country. Upon completion of the kindergarten, the parents of hearing impaired children have to decide to mainstream their children (i.e., put them in a general school) or keep them at the school for the deaf, with only 3% choosing the latter, as of today’s data (personal communication with school principal). Children with hearing impairment in the kindergarten are supported by special teachers and speech therapy experts who focus on the child’s development of auditory and communication skills.

Six kindergarten children, 3–4 years old, participated in the study. Detailed characteristics of the children (e.g., current speech and language skills, degree of hearing loss, age at identification of hearing loss, age at intervention for hearing loss) were unknown to the researchers, as requested by the school principal for confidentiality purposes. Despite the unfortunate fact, the researchers decided that the study was still worth pursuing in order to detect and document, even preliminary, the added value of an intelligent robot in the particular context. All children were supported by either a cochlear implant or a hearing aid or both, as seen in Table 1. These children attended regular kindergarten lessons together with eight hearing children in the same class. They also attended personal speech therapy sessions at the school’s special unit.

Table 1. Participants profile
Full size table

Participants were also the school therapist (audiology and speech therapy expert) and the special teacher of the school (also speech therapy expert). The school therapist worked with the researchers in the co-design of the activities for the study; she was also active during the implementation of the study and data collection as described below. The special teacher of the school was an observer in most (80%) of the therapy sessions. During the implementation and data collection the researchers (educational technologist and a speech therapist) were observers of the experience.

3.2 Activities and Co-design

The activities were co-designed by the researchers (an educational technologist and a speech therapist - researcher and practitioner) and the school therapist, based on previous experiences in research and practice, although such activities are considered general practice in speech therapy. Based on the school therapist, she had used similar activities with her students in the past, but there was no structured way of doing these activities (how often, what duration, what kinds of animals, what kinds of sounds). Also, in applying the auditory-verbal therapy method she was concerned about covering her mouth and acting unnaturally in any way that could disturb the child.

A total of four activities were designed and deployed on NAO. In accordance to the auditory-verbal method, all activities involved listening, followed by spoken instructions by NAO in order to assess children’s ability to correctly respond to the spoken sounds using their cochlear implant and/or hearing aid, rather than other cues in the environment. For every correct answer, NAO praised the child and NAO’s eyes turned green; for every mistake, NAO said “try again” and his eyes turned red. The visual clue of “green” and “red” eyes was the only visual clue in the environment. These four activities are briefly described below (see also Fig. 1).

Fig. 1.
figure 1

Implementations of farm animals (top), Ling sounds story (bottom)

Full size image
  1. 1.

    Ling Sounds Story. An object for each of the six Ling Sounds is placed on the desk (i.e., /m/: ice cream, /i:/: mouse, h/a/: Plane, sh/∫/: baby who sleeps, /s/: Snake, /u/: ghost). These six sounds represent speech in different frequencies (low, mid and high). They are used to check how well children are hearing with their cochlear implant and /or hearing aid. This check is used in order to detect changes in child’s quality of hearing. NAO tells a story and when the above sounds are spoken, the child picks the appropriate object to give to NAO e.g., “Maria spent a wonderful weekend riding with her family in the mountain. As they walked she heard a sound /iii /, what can it be? [child picks up the mouse]. However, her dad saw something terrible /sss /[child picks up the snake]. “Dad, this snake is very big !!”…”. The story changes from session to session so that the Ling Sounds are spoken in different order.

  2. 2.

    Music Density. The child holds a musical toy-instrument of his/her selection e.g., a drum (child selects from a box of a few toy-instruments). As NAO signs a rhyme in gentle intensity, the child beats the drum gently. When the intensity goes stronger at unexpended time, the child needs to immediately respond by beating the drum louder, and vice versa. In a round of play, after a medium starting density of the song, there are three unexpended changes in sound intensity (louder or lower). The game is used in order to develop child’s ability of discrimination and identification of sound intensity.

  3. 3.

    Farm animals. Various animals (farm and jungle animals) and a farm built of blocks are placed on the desk. NAO asks the child to place in and out of the farm one animal at a time, based on the animal’s sound e.g., “Take [cow audio sound] and place it in the farm”, “Take [dog audio sound] and place it out of the farm”. The seven farm animals are cow, sheep, dog, cat, rooster, pig, horse and duck; instructions play in random order for animals in and out of the farm. In a round of play, there are six statements to be executed by the child. With this game children develop their ability to discriminate and identify animal sounds which are with different frequency (e.g. low frequency – cow sound, high frequency – cat sound).

  4. 4.

    Vegetables. Vegetable-toys and a basket are placed on the desk. The child listens to NAO’s simple statements, such as “Put x in the basket”, “Take x and y from the basket”. The five vegetables are: potato, tomato, lettuce, cucumber and carrot; instructions play in random order. In a round of play, there are five statements to be executed by the child. The purpose of the game is discrimination and identification of vegetables.

3.3 Procedures

The study took place during six consecutive weeks of speech therapy sessions using strictly the auditory-verbal therapy method, which does not allow lipreading or other non-verbal cues to facilitate communication. Therapies were done daily, during the morning, in the special unit of the school. Some sessions were lost due to schools events, other obligations of the school therapist, or absence of the child from the school. In six weeks, children completed between 16 and 22 sessions with NAO. Children arrived at the special unity in groups of three (i.e., 2 groups of 3 children) and the session lasted 45-min. Within the session, the children performed in the activities one by one, while the other two children waited for their turn. That is, each child typically had a 15-min slot to participate in the four activities (Ling sounds story, Music density, Farm Animals, Vegetables). Some children managed to complete all four activities in their 15-min slot. Some others completed fewer activities. By week 3 (around 10 sessions) all children were fast enough to get through all four activities within their 15 min slot in each session.

The week before the investigation begun, all spoken sounds in the activities were pilot-tested with all participating children. For each child, NAO’s pitch tone and speed of spoken words was adjusted and noted for subsequent session, based on the school therapist’s input from observing the child’s respond to the sounds. NAO’s speak volume was set nearly to maximum for all children, which according to the therapist approximated how loud a teacher spoke in a typical lesson in the classroom. NAO’s language was set to the national language of the country. We used NAO’s robotic voice only for the instructions e.g., “take [audio sound for cow] and place it in the farm.” However for the sounds in the activities, such as the “cow “sound in the farm game or “/i:/” for mouse or “h/a/” for the plane in the Ling sounds game, the sounds were downloaded as audio files in high quality and per screening of the school therapist for quality and appropriateness for the activity.

During the study, NAO was sitting without nay interaction with the child other than verbal. We used strictly the auditory-verbal therapy method, which does not allow lipreading or other non-verbal cues to facilitate communication. Any cues from NAO or in the environments could have jeopardize the philosophy of the auditory-verbal therapy method. Moreover, NAO was semi-autonomous. NAO could play the activities and randomize the order of sounds within each activity. The therapist could touch NAO’s head (head-sensor on NAO) for NAO to act a positive response whilst a touch on his foot (foot-sensor on NAO) triggered a negative response (e.g., switching between activities, activating positive feedback or “try again” via touching as relevant). Although the experience could be manageable by the school therapist only, due to the exploratory nature of the present study, a student programmer was also in close proximity to the robot to ensure everything would work as planned. The special teacher of the school as well as the two researchers (authors of this work) observed the study from a seated position and without interfering.

In terms of data collection, the school therapist recorded each child’s performance in every activity completed (Ling sounds story, Music density, Farm Animals, Vegetables) in every session. Although NAO gave a “try again” option for the sake of play and learn, for data collection purposes the therapist marked a correct answer only on child’s first trial and only for correctly detected sounds. Table 2 presents an example of data recording for the “Ling Sounds Story.” Similar tables were used for data recording of other activities in every session and for each participating child. At the end of the activity the therapist together with the special teacher who observed the study participated in a group-interview with the researchers (authors of this work). The interview aimed to elicit details about the overall experience and perceived added value of the technology.

Table 2. Sample data recording and interpretation table.
Full size table

4 Findings

4.1 Gains in Detecting Sounds and Following Instructions

Using the detailed data recording as of Table 2, a percent success score was computed for each child per activity per week; see Table 3 for the average percent success scores across children. Plotting of participates’ success scores across weeks allowed us to realize the progress of each participating child’s gains in the ability to understand tasks presented through NAO over time. Some children mastered the detection of sounds earlier than others (e.g., by week 3; around 10 daily sessions), while all of them presented a consistent record of detecting sounds correctly by the last week of the study (i.e., week 6; 16–22 daily sessions). Figure 2 illustrates the progress charts of two of a participating child, as example. All participating children had steadily progress on all activities until a consistent correct pattern of detecting sounds occurred towards the last two weeks of the study.

Table 3. Average percent (%) success score across participants, on weekly activities
Full size table
Fig. 2.
figure 2

An example of child’s progress chart in terms of % correct answers across weeks (percent of correct answers on y-axes)

Full size image

With respect to observed gains in detecting sounds and following instructions, the perspectives of the school therapist and special teacher were consistent with the quantitative outcomes. Both argued that the auditory-verbal therapy with NAO seemed to improve the children’s auditory skills, including their ability to pay attention to NAO and to use their amplifier technology rather than relying on external cues. They also noted their observation of increased self-confidence for the participating children, as they performed better and better from session to session and as NAO positively reinforced their responses (“good job” and green eyes). As the therapist explained:

“NAO kept children’s attention, which is very important in developing verbal communication skills. NAO is interesting and cute enough to keep the child playing and completing the tasks while s/he is forced to listen. In early sessions, I noticed that all of them looked at me again and again seeking for clues. Because I directed their attention to NAO, they learnt to rely less on other cues, and use their amplifiers more, to play with NAO. I think the use of the robot in this sense is brilliant! […]. I also noticed gains in self-confidence from session to session and as children started learning to listen and to recognize the sounds that NAO spoke, for example [child’s name] was very shy at the beginning, but gradually she became confident and enthusiastic in playing with NAO.”

4.2 Playful and Engaging Therapy Sessions

The therapist elaborated on how the children responded positively and enthusiastically to the activities during the auditory-verbal therapy. As she explained,

“…the auditory-verbal therapy is not always pleasant for the child with hearing impairment. The child seeks for cues and s/he is confused and annoyed when I cover my mouth. I often hold a picture in front of my month, to distract him/her from the fact that I am covering my mouth on purpose.”

Yet, both therapist and special teacher thought that NAO robot made auditory-verbal therapy fun for the child with hearing impairment, engaging them in a playful learning experience. In the teacher’s own words:

“Children were happy playing with NAO and I saw their enthusiasm, in completing the tasks and working with NAO. Their joy lasted for the duration of the activity, and even after therapy, for the duration of the school day.”

In addition to NAO’s great prospects for speech therapy, a rather unforeseen idea in teachers’ feedback was the potential of play and learn with NAO in inclusive education settings. In their view, the experience could be beneficial for both hearing impaired students and their hearing peers (i.e., the activities could serve all students in class). For example, as the special teacher explained, the Music Density game could involve more density variations, from sharp & steep to smooth & gradual changes; this could make it challenging for hearing children of this age, in a fun, class-wide, music activity. Also, the special teacher thought that 3–4-year old hearing children do not necessarily know the sounds of farm animals and with some more customization (e.g., to include less familiar animal sounds) the activity could easily challenge the hearing child. This way all children could enjoy sessions of play and learn with NAO. Furthermore, in the special teachers’ view, the use of NAO in typical lessons both learning and children with hearing impairment could expand the auditory-verbal therapy time for these children who unavoidably do much lipreading and use other non-verbal cues as the teachers interacts with the whole class. As stated by the special teacher:

“I think it is a given that NAO can be used successfully in auditory-verbal therapy sessions. But he could work equally well in the inclusive classroom. Some of our hearing 3-year old children do not necessarily know the farm animals yet, neither the vegetables, so there could be value for them in playing with NAO […]. I just think the possibilities of this technology are beyond individual speech therapy sessions. Also, consider that in class, my students with hearing impairment continue to do lipreading, which is unavoidable. NAO could help me expand the auditory-verbal therapy time for them without taking anything away from my hearing children […].”

5 Discussion

This study aimed to examine an innovative way of using an intelligent robot to support speech therapy for hearing impaired children through play. The study presented a unique environment for auditory-verbal therapy for hearing impaired children. NAO does not have a human mouth and therefore it is not supporting lipreading during auditory-verbal therapy; this unique characteristic of the technology was successfully used in this study to create playful therapy sessions for six kindergarten children with hearing impairment over the course of six weeks. The co-designed activities and implementation procedures were detailed in this work. The authors believe that the innovative idea can be successfully transferred and replicated with success in similar circumstances and learning/therapy contexts.

Are there gains in the ability of the hearing-impaired child to detect sounds and follow instructions? The study demonstrated quantitative gains in hearing impaired children’s ability to respond correctly to sounds and to follow directions. All participating children had steadily progress on all activities until a consistent correct pattern of detecting sounds occurred towards the last two weeks of the study. We need to acknowledge however, that because no baseline measurements were provided as to how well the children could complete the tasks in non-NAO contexts, or data about their level of development in speech, language, or listening (please see discussion of confidentiality earlier), the results presented can only describe children’s gains in the ability to understand tasks presented through NAO over time, which may or may not, reflect a change in their level of skill in listening and language. Future work should aim to address these important limitations of the present work.

What is the perceived value of the robot-enhanced environment for auditory-verbal therapy? The added value of the intelligent robot during the auditory-verbal therapy was documented by the speech therapist and special teacher-observer in a couple of ways. The views of the researchers are also consistent with these reported observations. First, they noted observed gains in children’s ability to pay attention to NAO and to use their amplifier technology rather than relying on external cues. In other words, the hypothesized affordance of the intelligent robot was well materialized in the implementation. Second, they noted the playfulness of the learning experience and the children’s overall positive attitude towards the lack of non-verbal cues. This finding replicates previous work on children learning through play with intelligent robots, especially involving ASD children (e.g., [14, 18, 36]). Overall, the study might be suggesting that NAO can make auditory-verbal therapy fun and engaging for children with hearing impairment, compared to an experience in which the therapist must hide all non-verbal cues (i.e., covering mouth). Of course, we need to acknowledge that the input for the therapist may be biased because of her involvement in the design of the activities and implementation with NAO. We were bounded by this limitation in the context of this study; despite the speech therapy (research and practice) and educational technology expertise of the researchers, the direct involvement of the school therapist was instrumental for the investigation in an authentic school-therapy setting and practice. Overall however, the findings suggest an innovative way of using an intelligent robot in this context and justify further investigation.

Furthermore, the technology and experience were perceived as a promising pathway to inclusive education, an unforeseen finding, which however, the authors, consider a significant reflection. A complete suite of NAO applications with customizable content and difficulty levels can support the inclusive (kindergarten) classroom and benefit both hearing and students with hearing impairment. How intelligent robots can support the educator in adopting an inclusive education approach is a very promising direction for future research, considering the rise of children with hearing impairment in mainstream schools vis-à-vis the underutilization of novel technologies [43]. Yet, while the above-mentioned idea presents an exciting opportunity for future research in inclusive education settings, the present study does not yet provide evidence that NAO would be appropriate for or beneficial for use with children without hearing loss. Also, in a mainstream classroom (i.e., an inclusive setting) background noise in the classroom could make the task impossible for the children with hearing loss. These ideas coupled with plausible difficulties merit investigation in future research.

All in all, this study aimed to explore the humanoid robot’s potential for assisting therapists auditory-verbal exercises. Our conclusions are provisional and at this stage, we can only claim that NAO has potential in this context. A potential is only a possibility; we cannot offer any certainties, based on this initial work, but we do share our positive initial impression. The study remains a subject of future work and replication. We have offered our lessons learned as well as initial evidence that NAO could be an important means of assisting therapy of children with hearing impairment. We further demonstrated that NAO is an attractive tool in this therapy context because of the lack of visual clues (has no mouth) while at the same time it is fun and friendly [16]. We believe that a laptop or tablet could not have had the same effect, although future work could address this hypothesis via an experiment design with a control/comparison group.

6 Conclusion

The presented study focused on the use of the humanoid NAO robot in auditory-verbal therapy, an approach to the development of auditory and verbal skills, which does not allow lipreading or other non-verbal cues to facilitate communication. This is, in our view, innovative and is worthy of further exploration while aiming to address the limitations of the present study.

Play with intelligent robots is now a reality. The question of interest is how to best utilize such technology-mediated experiences for the sake of learning, including therapy. The study contributes to the technology-enhanced learning, speech therapy and special education communities by presenting a case of practical utility of humanoid robots with real world impact. The findings of this study are encouraging and warrant further investigation to fully exploit the possibilities of intelligent robots in the context of education and therapy for children with hearing impairment.