1 Introduction

Learning and developmental disorders in children are intrinsic to the individual, presumed to be due to central nervous system dysfunction, and may occur across a life span [1]. These brain’s disabilities will cause not only learning difficulties, but also communication difficulties, and physical difficulties in various situations. To moderate these difficulties, it requires daily training of social skills in terms of interaction and communication with others. To improve social skills, play therapy has been acknowledged as a major method by which to help children communicate and express their emotion with counseling [2]. Play therapy is a “here and now” approach to play therapy that is considered similar to a behavioral approach in which children practice new behaviors to prepare for real-life settings [3]. However, there are problems: these settings are expensive, and need trained therapists for supporting enrolment and immersing the settings interactively. Sometimes, it requires coming to a special care-center, therefore it is difficult to train their skills by using this method with settings in daily-life. In the therapy method that could improve social skills, it is important to listen to others, and express their emotion about their imminent environment with their language. Since daily training is required, the device should not be large and hopefully not be installed any specific place, and it is desirable to have a theme that children want to get involved with spontaneously.

Based on the above requirements, we present a digital dollhouse that enhances traditional psychological play therapy with sensors and computer graphics. For the children, it is similar to a house environment like their home, so it is possible for them to immerse themselves into doll play space in their pretend role play as a family member. We consider that this environment is suitable to train the children to improve the required social skills. To accelerate the basic effect of the dollhouse and compensate to meet the requirements, various sensors are installed to sense their physical world, to share the feeling and emotion with one another; the result would be visualized in the screen of the computer to share information more easily (Fig. 1).

Fig. 1.
figure 1

Orange roof’s dollhouse and CG

Full size image

A psychologist who uses play methods for children told us that there are important social skills which should be trained as follows: Firstly, we consider (1) basic training for hearing and speaking. Next, (2) training for choosing the appropriate word for explaining a particular situation.

To realize functionality with our digital dollhouse, we provide the following functions. (1) In the dollhouse, we installed temperature sensors, and a light sensor (photo detector), and these sensed the changing of physical situations that reflects to the virtual room illustrated on the screen. (2) We installed a switch that could change the situation of the room. We assume that the reactive system could change to help children to express the situation by their operation. Further, in the room that is illustrated in a computer graphic, we prepare a window that shows the changing of seasonal views. We assume that that encourages training into words’ subtle changes in seasons. Though they share the sense of the physical world in the dollhouse and the augmented reality in the computer with visual appeal, we assumed that children easily immerse the play and express their word by sharing the situations visually and sensuously with the trainee.

As the first stage of the discussion, we discuss the idea with an expert, who has experience in play therapy, and organize the requirements for the device, and we have developed a prototype. In this paper, we will describe the design and prototype implementation, and describe how to apply in the discussion, and also about the challenges of the future.

The sections are organized as follows. In Sect. 2, we describe the related work for digital therapy and clears required issues. In Sect. 3, we describe the prototype implementation. In Sect. 4, we show the result of the discussion and factors for enhancing empathy resulting from the evaluation of a prototype. We conclude this paper in Sect. 5.

2 Requirement for Digital Play Therapy

2.1 Related Work of Play Therapy

In the case of children with disabilities of hearing in the lower grades of elementary school, it is recognized as the pre-operational stage of cognitive development, which is suggested by the Piaget’s theory of cognitive development [4]. Since it is difficult for the children who were in the state of pre-operation to treat something logically just in their brain, it is important to give some specific things to treat at training. For these children to train hearing ability, a kind of games that are required to hear carefully from others are generally used (such as street hint game, flag up game etc.), and also widely used are social skill training games that involve the element of hearing such as sugoroku (Japanese backgammon) talk, and storytelling [5, 6, 7]. These games are used easily, since these are generally used in children’s play, and are not only used for this specific purpose. On the other hand, these games are too general to focus on the training of specific abilities consciously. To satisfy the requirement especially for the improvement of hearing ability, the training by the expert and the teachers that have knowledge and experience of hearing and cognitive linguistics, and need to understand the elements of hearing that is included in the training and gaming materials.

Play therapy and Sand-play therapy are used for children who have communication disabilities [8, 9]. Play therapy has an advantage in that it provides education for the children to have experience of touching some play objects within the therapy. Particularly, the therapy is considered as an effective method for the children that could not see the environment objectively, since it is required to reproduce an objective environment in their play, by using an agent and surrounding devices. On the other hand, these therapies require experts to support, and sometimes require the use of large equipment such as therapy-balls and trampolines, and expensive dollhouses.

In these years, teaching devices have emerged that are supported by various computers such as the personal computer, PDA, smart phone or something. However, these may then require responsive reaction with keyboard and touching the screen by the children that provides some stimulus materials [10, 11]. These ways of training would be effective for typing and seeing the screen repeatedly. However, since this information is limited in the screen, it is difficult to increase the motivation to communicate with others for the children who are in the pre-operational stage, where it is difficult to have a concrete image without experience of treating actual object. In the HCI research area, like Tangible Computing, that are proposed by Prof. Ishii, novel interaction is provided through physical environment with digital information. However these are not taking advantage of the immersion agent [12].

2.2 Issues

Our proposed interactive devices have been aimed at training for children who suffer learning and developmental disorders, for the purpose of improving specific functions in their brain. Here we mean that “training” is brain function training through the growth of language and communication skills. In this training, we consider the ages to be about three years to eight years old who are undergraduate and lower graduate in primary school, since it is expected to obtain the effect easier than with elders. In the instruction of language and communication, there are mainly two target social skills that should be trained for improvement, being speaking and listening. We firstly consider the requirement for the training tools that improve these two social skills. The most important requirement for the device is that it gives a motivation to the children for having fun and being evolved in spontaneously and continuously without being bored, since the practices are necessary to be repeated in the same situation.

3 Design and Implementation

3.1 Proposal

We propose a digital dollhouse that enhances traditional psychological play therapy with digital sensors and computer graphics. For the children, it is familiar with the house environment; it is possible to immerse them in doll play space in their pretended roles. We consider this environment is suitable to train the children to improve required social skills that we discussed in the previous section. We prepare a dollhouse device that is possible to treat in small space and their roof will be opened with their small hand, since it is important for these children, who are in the stage of pre-operational age, that tend to think intuitively through concrete devices and to have a interest for the “touch” itself. Based on this idea, we installed the following features of the device that have the purpose to improve the three social skills that we discussed in Sect. 2.2.

3.2 Support for the Therapy

In the Fig. 2, we illustrate the abstract image of training in the therapy. We assume that a trainer trains the child. Between the two, our proposed device will be set (Fig. 2). Here we treat the therapist and trainer as having the same meaning. Through the training in conversation and interaction with the proposed device’s functionalities, the abilities of the child are promoted so that they can choose appropriate words for explaining the feelings that were aroused in the house by different situations (Fig. 3).

Fig. 2.
figure 2

Sharing the visual and tactile through the screen of a personal computer

Full size image
Fig. 3.
figure 3

Inside the room on the screen, Right: Without Light/Roof, Left: Natural Light

Full size image

3.3 System Design

Our prototype is controlled with Arduino, [13] which is an open-source electronics prototyping platform. On the board, we installed a temperature sensor, light sensor, and switches. Through the sensors, the microcomputer read the changing analog and digital values, and it was possible to operate them to show the result on the screen. We developed a program that reads the analog values from the board, and analyzes them to select appropriate computer graphics interfaces that were illustrated as virtual rooms.

We assume that there are four states that may change in the room according to the combinations of parameters from the sensors as follows. (1) Roof (on/off) from the light sensor (2) Light (on/off) from switch (3) Air conditioner (on/off) from switch (4) Temperature in the room (analog value) from temperature sensor. There are two switches on the board, and each of them reacts according to the user input that changes the status of light, and the air conditioner. These four states indicate the status of the room to reflect to the physical environment and interaction from the children (Fig. 4).

Fig. 4.
figure 4

Connections of sensors on the board

Full size image

We use LM35 temperature sensors to output a voltage (the analog of which is proportional to the Celsius temperature conversion formula that is output in accordance with (Vout_LM35 (T) = 10 mV /Celsius × T Celsius). The optical sensor (Photo resistor) is a sensor for detecting the intensity of light. With the electric resistance decreasing, the intensity of the incident light is increased, it is assumed that this quantifies the amount of light, corresponding to the brightness of the current room, it is possible to adjust the brightness of the room on the CG.

3.4 Software Design

Based on the idea of the changing status of the room being read and changing the CG on the screen, we designed the software program that decides which picture on the virtual room combined with the parameters from the microcomputer-board. Like other microcomputer programs, this program’s flow chart in the figure consists of two phases. The first one is initialization phase (setup() in Arduino syntax), and loop phase (loop() in Arduino syntax). In the loop, the microcomputer reads sensors’ values through their I/O connected sensors such as switches and light and temperature sensors, then these parameters firstly show on the screen (with appropriate values convered), since sometimes microcomputers only provide the limited range of parameter such as 0-255, therefore programmers need to map the value to the familiar parameter that should be understandable for the average person that ranges from 15-32c in temperature. In the case of deciding the brightness with light, we previously set two ranges of parameters that show “bright” and “dark” in our evaluation, then, the program could decide which picture should show on the screen in terms of reading the analog values from the sensor. Switches are simpler than the light and temperature sensors, because they just read on/off status from the sensor. To work correctly, we just set pull-up registers on the board.

3.5 Training Program

According to the functionalities we proposed, we consider the new training program using the proposed device.

To consider the training program to promote the ability of child, we use the classification metrics of LC-Scale (Language Communication Developmental Scale) [16]. It provides observable assessment of language communication behaviors of the typically developing children who are from 0 to 6 years old, and classifies these behaviors into different areas. According to the LC-Scale areas, we picked up appropriate functionalities of our device and programs, and classified each of them into an area as the training program. In the Table 1, we show the training program. In the first column we show a function that the device proposed, the second is the area of the LC-Scale, and the third is the obtained social skills that are assumed, and the last is the provided training program that contains detailed instructions from the trainer to the child.

Table 1. Device functions and training programs
Full size table

The optical sensors were considered to perform training of promoting an understanding of the operation instruction of the position of the object, since it requires the ability to detect the presence or absence of the roof of the house on the top. This is the area of vocabulary of the LC scale.

Next, temperature sensor has the function of detecting temperature of the surrounding environment. Therefore, associating training of promoting an understanding of the vocabulary that represents the quantity by asking the temperature degree. The remote control using infrared light has the ability to switch the electricity and air conditioner ON /OFF in the room as well as in the room interior of the CG image being displayed on the PC screen. It is considered to perform training to understand the complex instructions, since the controller gives various operations. We assign the function to understand the word chain and syntactic area of the LC scale.

In the entire training materials, there is a function to change the scenery outside the window of the room inside of the CG image in the spring /summer /autumn /winter. This is because it can be used to train the mental function that classifies the understanding of temperature change was assigned a discourse-word operation area of the LC scale.

Training for choosing appropriate words to explain the changing situation: we installed a switch that could change the situation of the room. The change will be the result of the operation of the child pushing the button. We assume that the reactive system could help children to express the changing situation through performing this operation. Training for choosing an appropriate word for the more complex instructions is thus given. In the room that is illustrated in the computer graphics, we prepare a window that shows changing seasonal views. Though they share the sense of the physical world in the dollhouse and the augmented reality in the computer with visual appeal, we assumed that children easily immerse in the play and express their emotions by sharing the situations visually and sensuously with the trainee.

4 Discussion

We developed a prototype of our proposed system, and demonstrated it in front of an experienced expert. We discussed the issues and detected the problems with digital play therapy that are given from psychological viewpoints. We categorized the four issues about the device into four areas.

1. Using the Method of Play Therapy.

We prepared a dollhouse device that is possible to treat the parts of device, since it is important for these children, who are in the stage of pre-operational age, who tend to think intuitively through physical devices and to have an interest for the “touch” itself. From the viewpoint of the expert, it is also preferable. Further, the “house” is a good device as also used in sand-play therapy; there is an advantage that is associated with the space in which they are living on a daily basis, in that it is easy to image the specific results from their operations. It is evaluated that it is easy to train and enjoy the play on words of everyday life by using it, to reproduce the behavior of them. It may also be effective even for children with no developmental disabilities.

2. Providing Functions According to the Development of Skills.

In this research, we propose a novel device that has a purpose for development of three social skills. According to the recognition, we developed the device to have functionalities with sensors and CGs. These facilities are interesting; it should be needed to consider the evaluation of the effectiveness of the functionalities. Moreover, it may give a good motivation to speak the words “good morning” and “good night” naturally with easily understandable pictures. We also discuss the idea that, if there is a character that is preferable for the children, its use would be more effective to promote the motivation to speak.

3. CG Supports the Immersive Augmented Reality.

The expert evaluates positively the CG support for visualization of the reactive operation and changing situation on the screen, since it can share the feeling easily through the screen between them. In the training, it is important to share the feeling with the therapist and children. The effectiveness of the CG support can be evaluated such as for the following example. When the room temperature is 20 degrees, we compare the two cases. The first one is where experts tell the children “It feels warm, do you?” and the second case is where the CG shows the cherry blossoms fall in the window of the room in accordance with the parameters of the temperature sensor on the CG. We consider that it is possible to imagine the specific meaning of that season as being more realistic. In other cases, children enjoy the changing result of their operation that causes the change of the situation in the room, to see the temperature will go down after their push of the switch.

CG representation is also the embodiment of the real world, thus, the extension is possible. It is believed to be able to stimulate the empowerment of operational feeling of operation, and give a pleasure to share and expand the world. In particular, it is considered to be very important that they can check the result of their operation in visual world (CG) that is an extension of the real world. These studies have been proposed as Blended Reality, and other names, [14] however, it has not been sufficiently discussed yet.

4. Augmented Reality and Development Empathy with an Agent.

During the discussion between engineers and a psychologist, we found the process of our system enhances the ability of empathy for the child. Rogers noted in his work that the key factor of empathy is to feel the other’s viewpoint “as if it were your own” [15]. Our proposed three devices support the enhancement of the ability to shift one’s viewpoint to another location.

Based on the above discussions, we hypothesized that a character in a digital dollhouse enhances feeling of empathy. Children tend to talk to a character that seems to be familiar with them. These activities will be expected to encourage development of their ability to speak to the others. Moreover, these activities would develop the ability to select the most empathetic method of speech to others.

For example, if there is a character that seems to be almost cold, children talk to the character, such as “Are you cold? I will push the button to warm up the room”. It is good training for developing the abilities of empathy and selects the appropriate words to express, and selects the appropriate actions to improve the situation of others. Thus, it is important to evolve the functions that can encourage sharing feelings with others and to have empathy with others, and develop their abilities to talk to others naturally through the changing of seasons and temperature. Moreover, in the future, we need to place a character. This would give a subject to talk about.

5. Contributions and Limitation.

The contribution of our work is to show why digital technology supports children abilities in terms of a psychological viewpoint. We conducted interdisciplinary discussion between engineers and psychologists. The result of discussion with real prototype reveals that digital dollhouses will support children’s abilities to shift viewpoints. It also suggests that our approach is also improved by applying human-agent interaction technologies. Our interdisciplinary works are just a start point. Further research will be required to verify our hypothesis.

5 Conclusion

In this paper, we propose a digital dollhouse for children who are suffering with learning and developmental disabilities, who require daily training to develop their social skills. It may enhance traditional psychological play therapy with digital sensors and computer graphics.

The digital dollhouse provides an immersive space for children that grows their abilities to hear and speak through their imaginary play. In this paper, we showed details about the digital dollhouse prototype. We also categorized requirements for Digital Play therapy, from psychological viewpoints, as the first stage of our research.

In the discussion we categorized the requirements and became aware of a new requirement character that would promote the training effectively. In the future, we will try to improve the house based on this discussion and include how multimodal interaction environments are effective for children from the viewpoint of those with neuroscience backgrounds [17]. We currently associate the relationship of communication and language as being based on empathy. From the viewpoint of computer science, we consider the effective support in the use of agents.