1 Introduction

Hearing impairment is the third largest burden of disease for the state, society and individuals. Especially, it brings more heavy and long-term burden for hearing-impaired children. It makes the children low self-esteem and closed. As the decline of cognitive ability, they will face a lot of challenges in the learning process. It always has a deep influence for hearing-impaired adult in the workplace. The loneliness in life is harmful for development of the whole society. A research suggests that the prevalence of child and adult hearing impairment is substantially higher in middle- and low-income countries than in high-income countries, demonstrating the global need for attention to hearing impairment [2].

The developing countries has no suitable treatment as the economically backward, which makes a huge burden for economic development. So it is a vicious circle. Besides, a research supports the conclusion that, in children with permanent childhood hearing impairment (PCHI), newborn hearing screening and early identification lead to beneficial effects on language development, with the most consistent evidence provided for links between early identification of PCHI and positive language outcomes [3].

A number of methods and products have been developed. However, these products are just like laboratory instruments and lack of humane and fun. As a result, it has no appeal to children. In the age of advocating the user experience, it is essential to understand the products from the perspective of hearing-impaired children to increase interest of the product [4, 5].

Thorny problems in Speech recovery for hearing-impaired children:

  • Hearing impairments are mainly in low-income families and developing countries with backward medical technology;

  • It’s lack of experts in speech training area and lively and interesting training methods. Ignoring the role of parents

  • Existing products ignore the children’s active, noisy nature.

It’s essential to make a contribution for hearing-impaired children by design. We aim to design a speech training system of self-service, which is cheap and easy to popularize. Gamification design is central theory of our design. If we blend the fun of game in the boring speech training process, it will improve the hearing-impaired children’s enthusiasm.

2 Gamification

2.1 The Difference Between Gamification and Serious Game

Gamification has been an important theory in innovative product design and it is also an outstanding method for improving user experience. The term “gamification” was proposed in the DICE 2010. The Professor from Carnegie Mellon, a game designer,Jesse Schell proposed that the game elements will invade every aspect of our daily lives [6]. A prediction from Gartner was that more than 50 percent of organizations that manage innovation processes would adopt the gamification design for improving the user experience in 2015 [7].

Gamification, big data and internet of things was included into the 2011 Hype Cycle for Emerging Technologies. The report said gamification would peak in 2011 and it could be accepted in 5−10 years. Gamification has been making a transition to steady development after the peak in 2014 (Figs. 1 and 2).

Fig. 1.
figure 1

The 2011 Hype Cycle for emerging technologies

Full size image
Fig. 2.
figure 2

The 2014 Hype Cycle for emerging technologies

Full size image

Gamification is not perfect and the existence of some shortcomings result in the transition. Most criticism originated in the mechanical applying of points and game toll-gate, which ignored the heart for improving the user experience [8].

Some researchers take the view that all elements of the game in the application can be defined as gamification, such as serious game, interesting interaction and games technology. Sebastian Deterding propose a rigorous definition of “gamification” as the use of game design elements in non-game contexts [9,10]. The following sections unpack this definition in detail (Fig. 3).

Fig. 3.
figure 3

The difference between gamification, serious game and, interesting interaction.

Full size image
Game::

In terms of defining “gamification”, this means that it has to be analytically distinguished from playfulness or playful design.

Elements::

Whereas “serious game” describes the design of full-fledged games for non-entertainment purposes, “gamified” applications merely incorporate elements of games.

Non-game contexts::

Similar to serious games, “gamification” uses elements of games for purposes other than their normal expected use as part of an entertainment game.

Design::

For the purposes of terminological and conceptual clarity, it is more helpful to reserve the term “gamification” for the use of game design, not game-based technologies or practices of the wider game ecology.

We can get the following theory combined with the previous research theories. The game is certain product or service with pure purpose for entertainment; serious game is a special form of game with the function and entertainment. Gamification is a great method that can improve the user experience.

Therefore, the design of speech training system can be defined as “improvement based on the gamification”.

2.2 Gamification and Flow

Psychology professor Mihaly Czikszentmihalyi describes a special happiness that was named as the “flow” [9,10]. Flow is a state that is absorbed in something with the characteristic “Concentration of strong, self-forgetful state and modest challenges”. When you are in the flow, each little action sucks you in more and before you know it, you have spent many hours.

The flow can be divided into two parts: conditions and characteristics. Condition refers to the factors you need to flow and the characteristic refers to your state of mind when you are in flow (Table 1).

Table 1. A core component of the heart flows
Full size table

Having triggers is the premise to get you in this state of flow. Most importantly, every task should be challenge enough and not in excess of your ability. Task difficulty and user capacity should be improved at the same time (Fig. 4).

Fig. 4.
figure 4

The flow state falls in the middle of anxiety and boredom

Full size image

2.3 Gamification Design for Hearing-Impaired Children

Gamification has been used in the health care, physical excise, education and so on. Some companies have made the game as an incentive system for service, which is composed of rank, toll-gate, reward. It is a common practice to make the incentive mechanism into the service and the event planning for improving the participant. The real meaning of gamification design is not the quality of finishing the task. What the most important is the experience in the task.

Gamification design is very common in children’s products at present. It’s the key point on whether or not the amount of entertainment is enough for children in the interaction with the product. The most products focus on guidance, remind, record, data accumulation. So most products have not enough entertainment components.

Gamification design for children should fully pay more attention to the two points: Nature of children’s hyperactivity and immature understanding. In a word, attraction and readability are the two most important things. It shows the different types of player in Richard Bartle‘s research, which are Achievers, Explorers, Socialisers, Killers [11].

  • Achievers are interested in doing things to the game, i.e. in acting on the world. Achievers are proud of their formal status in the game’s built-in toll-gate hierarchy, and of how short a time they took to reach it.

  • Explorers are interested in having the game surprise them, i.e. in interacting with the world. Explorers are proud of their knowledge of the game’s finer points, especially if new players treat them as founts of all knowledge.

  • Socialisers are interested in interacting with other players. Socialisers are proud of their friendships, their contacts and their influence.

  • Killers are interested in doing things to people, i.e. in acting on other players. Killers are proud of their reputation and of their oft-practiced fighting skills.

Based on the research, more children belong to the role of achievers and explorers. The reward for achievers can protect children’s self-confidence. When children advance in the game, modest task can stimulate children’s desire to explore.

Prime time of hearing-impaired children to recover the speaking ability is 3−6 years old. It is contrary to the children active and lively nature to make children in this age group accept boring training for long time. So it usually lasts just 30-min in each training to ensure the children’s attention. Due to children’s physical defects and the special growing up, they are often psychologically fragile. So the design principles are as follows.

  • The appraisal encourages primarily;

  • To encourage the bold participation;

  • The difficulty of toll-gate should be moderate;

  • The game mechanism is easy to understand;

  • The input and output system is in a simple way;

  • It should not make the task completion as the only index;

3 A Case of Gamification Design for Hearing-Impaired Children

A case is developed for hearing-impaired children to do speech training based on these theories. This speech training system is made up of the software and the hardware. The software side is training mechanics that are made up of courses, gamification, social contact, data processing. The hardware consists of cartoon modeling, natural interaction. Software and hardware need cooperation each other in this training and work as a whole by the smooth interactive experience. It creates an interactive, funny and freely training scenario in this speech training system (Fig. 5, Table 2).

Fig. 5.
figure 5

Logical frame of the speech training system

Full size image
Table 2. The description of user role
Full size table

The description of user role:

The software is an application named “NewVoice”. Parents record the courses that the therapists upload and the parents’ faces can reduce the sense of oppression in the training by the affinity of products (Fig. 6).

Fig. 6.
figure 6

The application interface

Full size image

The gamification system includes the GUI(graphic user interface of game), toll-gate, achievement, reward and the doll. The cartoonish interfaces show more affinity to the children and the game “whack-a mole” has a perfect fusion with the training (Figs. 7 and 8).

Fig. 7.
figure 7

The training interface of the application

Full size image
Fig. 8.
figure 8

The showing of the using scene

Full size image

Children’s pronunciation can be captured by the speech recognition system. If the accuracy reaches the preset, the mole will be beat down. Each toll-gate is made up of word, words and sentences, whose the amount and difficulty are set by the therapist based on the statistical table.

Beyond that, it is very important for speech training to distinguish the morphological changes of the vocal organs. The mechanism in the doll’s mouth will show the morphological changes of tongue and lips in real-time. The doll consists of cartoon modeling, natural interaction and voice feedback. The doll has two major roles:

  1. 1.

    To show the morphological changes;

  2. 2.

    To extend the game mechanics from online to offline;

The cartoon modeling is to show the design more visually appealing and interesting The doll was designed into four interaction motions “swing, spin, hug, pat” with different voice feedback based on the characteristic of the children’s body movements in nature. What’s more, the doll adopts tumbler movement mechanism with Chinese characteristics, which is beneficial to the realization of interactive action. It aims to create an interactive, funny and freely training scenario in this speech training system.

4 The Experiment for Usability

4.1 Measurement of Attention and Emotions

Each training lasts 30 min and it’s made up of 4 parts: Review, Study, Practice, Reinforce. The experimenters Recorded the change of the emotions by observation and graded the emotion by -2–2, while recording the reason for distraction over time (Figs. 9 and 10).

Fig. 9.
figure 9

The recording process of attention and emotions

Full size image
Fig. 10.
figure 10

Grade the emotions

Full size image

The speech training system based on gamification design has a clear comparative advantage. Children’s emotional average value is negative, which shows that children have more negative than positive emotions in traditional process. No matter the traditional training or gamification-based training, the sense of achievement can produce more negative emotions. When the children answer the therapist’s questions correctly or succeed passing the game toll-gate, they always have an emotional value recovery. So the difficulty of the task should be set with strict, scientific and comprehensive consideration.

4.2 The Experiment of Eye Tracking

“God closed a leafed door for you to open a leaf of window inevitably for you.” Though hearing-impaired children suffer hearing loss, they always have a better eyesight. Eyes are the main information transmission channel for hearing-impaired children. We can explore the world in the eye of hearing-impaired children by the experiment of eye tracking. So we can find the shortcomings in user interface design.

Study the difference when adults and hearing-impaired children observe the same interface by the heatmap produced by eye tracking (Fig. 11).

Fig. 11.
figure 11

Adults (left) and hearing-impaired children (right)

Full size image

The interface of pronunciation correction that is the core of application consists of 4 visual elements: mother, the game “Whack-A-Mole”, word and form, images of other objects. The parts with the deeper color means longer observation time and larger number of observation. It’s obvious that the images of participants’ mother have the strongest attraction for themselves. However, children are more interested in the game “Whack-A-Mole”, the image of other objects while the adults have more desire for reading the word and form.

All visual elements of the training process can be divided into 5 parts: mother, the game “Whack-A-Mole”, word and form, images of other objects and the doll. Recording the observation time on static visual elements by the eye tracking (Table 3).

Table 3. Recording the observation time of static images
Full size table

The mother’s image is more attractive in all static visual elements. When all the visual elements have a dynamic display based on the game mechanism, great changes have taken place in the children’s visual world unexpectedly (Table 4).

Table 4. The difference of dynamic images and static images in observation time
Full size table

Children are not interested in the word and form, even disgusting them. If the training operates based on the game mechanism, children are more excited. Because of the dynamic game and doll, children reduce dependency on parents.

5 Conclusion

Parents rarely participate in training in the traditional process. Actually, parents’ affinity play a pivotal role in training. However, parents forced to cope with stress in daily life, and there is little time to participate in training. This self-service training system can reduce the cost of training and take full use of the parents’ free time. Besides, training time should be one of the index to pass the toll-gates in the game mechanism design. If so, it can make the hearing-impaired children have enough time to be in the sound environments, especially, when they are alone. Most products just have cartoon static interfaces and text messages, which has little appeal for the hearing-impaired children. The experiment of eye track shows that the hearing-impaired children are more sensitive to the dynamic elements. The training process can be designed to be more interactive by the gamification design. Children usually play a role of “achiever” in game. As they are often psychologically fragile, a modest difficulty of doll-gate in the game mechanism should be designed to protect the self-confidence for the hearing-impaired children. At the same time, children will have the desire to keep on exploring.

It’s very effective to use the gamification theories to reduce the monotony in the training. The game can produce an addictive magic power that promote the productivity for working and training. When the gamification is used in the daily life, it means the improvement in the quality of your life. However, the gamification is not perfect now. It’s just the collection of lever, point, achievement and so on. It’s the key point to integrate the gamification to the daily life deeply. There is still far to go.