Keywords

1 Introduction

The number of elderly and persons with dementia are increasing worldwide [1]. More specifically, the number of persons with mild dementia is predicted to rise markedly as a result of the medical progress that slows down the course of the dis-ease, and the advances in diagnostic technologies that facilitates its early detection. Based on these perspectives, there are greater expectations from assistive technologies to support persons with mild dementia, due to the insufficient number of caregivers. For this reason, it is also important for these persons to be able to maintain self-reliance and to live independently.

In our project, we developed the prototype of an information support robot for persons with dementia, using field-based methodology [2]. The communication robot produced by NEC Corporation, called “PaPeRo,” was chosen as the platform for this system. In order to keep persons with dementia informed of their daily schedule and to prompt them into action, interactive verbal communication algorithms were programmed into the robot. The results of the experiments, which were conducted in our previous studies, with five dementia sufferers, showed that this system has an information acquisition rate of over 90%. In addition, a life support demonstration confirmed that the robot could successfully prompt the users into taking action.

During these experiments, we observed changes in the relationship between the elderly and the information-support robot system that they lived with. In this study, we examine how the elderly people receiving information behave according to the robot’s interaction protocol, and how to modify the relationship with the robots. For this, we analyze the interaction between robots and humans, both quantitatively and qualitatively, using long-term experiments in everyday life.

2 Concept of Information Support Robot System

2.1 Platform Robot System Used in the Experiments

The platform for the robot system, called “PaPeRo” (Partner Personal Robot) [2] is shown in Fig. 1. The robot is equipped with speech recognition, speech synthesis, facial image recognition, autonomous mobility, head motion, light indication functions, and tactile sensing capabilities. In this research, we limited these functions to adopt to our system concept.

Fig. 1.
figure 1

PaPeRo (R500) (NEC)

Full size image

PaPeRo also has some preinstalled content to make the interaction between the robot and the user enjoyable, such as quiz, games, riddles, mimics, songs, and greetings. This content also includes functions that generate sound, light, or motion in response to a registered voice when a user calls the robot. In this research, we utilized these preinstalled content to increase affinity.

2.2 System Concept [2]

The key functions of the proposed information-support robot system are as follows:

  • To provide required information to the users

  • To prompt actions/activities in the users

  • To attract the attention of users

  • To communicate with the users through interactive conversation

A method for interactive information support based on the nature of the interaction between humans, was adopted for the following interactions: the most basic way of starting a conversation, which involves an attention-seeking cue (an “alert” (a)), and “pre-sequence” information support, which facilitates the eventual ex-change (“communication” (b)) that extends to the end of the conversation, as shown in Fig. 2(i). The proposed information support algorithm, which is based on the evaluation experiments conducted in the previous studies, is shown in Fig. 2(ii).

Fig. 2.
figure 2

Structure of interaction and the algorithms for information [2]

Full size image

In this algorithm, two conditional branches are provided. One is for an alerting interaction that judges whether it responds by reply (sound), when it is difficult to direct attention to the robot at the start of the dialogue due to deterioration of the attention function. The second is for distributing information, as an information interaction for confirming that the user has acquired the information, depending on whether the answer is affirmative (yes, understood, etc.), negative (no, do not know, etc.), or doubtful (what?, eh?, etc.).

2.3 Human-Robot Interaction

There are many cases in which the social interactions of robots with the elderly were evaluated in laboratory environments. However, there are few cases of evaluations that were conducted in actual living environments for longer periods of time.

Robots with tactile interaction such as PARO [3] and mobile robots used in hospitals such as Care-O-bot [4], etc [5] have been evaluated alongside humans in actual living environments. However, it is not clear what kind of relationship was established for dialogue-based interaction between the robot and one person living alone, and how the person’s interaction with the robot is modified. In this research, we will focus on this point and examine it.

3 Change in Conversation Timing

3.1 Method

An experiment was conducted to evaluate the change in conversation timing of the robot and the participant. The participants included five elderly persons with mild to moderate cognitive impairment, who were not familiar with computers. Five experiments were conducted per day. The experiments were conducted on A for five days, while they were conducted on B, C, D, and E for 3 days.

The experimental environment involved the daily living environment, and for this the target participant’s own room or an equivalent environment was chosen. The sound environment was also matched to that of their daily life; however, it excluded sound sources such as television, audio, and radio. The distance of the experiment participants from the robot was fixed at about 60 cm, and the dialogue was held face-to-face. For the voice characteristics of the robot, a male recording voice was adopted and the basic utterance volume was set to 80 [dB]. The reactions of the experiment participants and the state of information acquisition were captured using webcam and IC recorder.

To record the basic characteristics of the participants, professionals measured the cognitive function (CDR: Cognitive Dementia Rating [6]) and the average hearing level.

3.2 Evaluation

In this interactive system, it is necessary for the interlocutor to answer at the right timing when the participant talks to the robot while the robot speaks or when the robot performs sound/speech recognition. In this experiment, we analyzed the responses to each interaction and investigated the utterance timing of the experiment participants.

In the dialog system proposed in this research, the robot speaks first. During this time, the robot does not perform sound/speech recognition. After completion of utterance, it performs sound/speech recognition for 10 [s]. In the case where the speech recognition interval is t 0 – t t , and t 0  = 0 [s], the dialogue start time t a , t b of the interlocutor takes a negative value. If the participant does not reply until it exceeds the recognition section of 10 [s] or does not respond to it while the robot is speaking (Ra in Fig. 3), the robot does not recognize the utterance of the reply and calls repeatedly (Rd in Fig. 3). On the other hand, when the reply is related to the speech recognition interval, it can be correctly recognized (Rb, Rc in Fig. 3). Since the details of these systems are not explained to the experiment participants, the replies are given at an arbitrary timing by the experiment participants.

Fig. 3.
figure 3

Definition of reaction time of the dialog system

Full size image

3.3 Results

The profile of the participants are shown in Table 1. Figure 4(a) shows the change in response timing of participant A. The average response time was calculated by averaging the reaction time for each experiment day of attention interaction and information support interaction. Although variations are observed, it can be seen that the timing of the sound/speech recognition is within the speech recognition interval and approaches the t 0 value both in replying to the call for attention and in response to the information transmission. Figures 4(b)–(e) show the change in the response timing of participants B, C, D, and E. A similar tendency as that of participant A was observed.

Table 1. Profile of participants [2]
Full size table
Fig. 4.
figure 4

Temporal change in response time of participants

Full size image

4 Change in Quality of Relationships Between Robot and Human

4.1 Method

  1. (i)

    Intervention method

The robot was set in the living room of the participant for one month and the information support functions were carried out every day. The introduction was conducted step by step, only during daytime, every day for one week or more, considering the opinion of the participant and a key person. The support functions included schedule and medication support, and the schedule was input by the key person based on the interview with the participant. The preliminary interview determined the requirements of information, forbiddance and precautions, confirmation of input information, attitudes at the time of regular information support (wording and the way of responding to support information from regular staff). The execution timing of the information support depends on the data and time recorded using dedicated software.

The target participants were recruited from a paid nursing home. The condition of the target participants were: there was a need for information support, they found it difficult when disturbed, and the staff often had to support their actions by word.

Cognitive function tests and average hearing level tests were conducted before and after the introduction.

  1. (ii)

    Experimental system

The robot, which was installed with the proposed information support system, was used for the experiment. The robot activated the preinstalled contents (described previously) in order to enhance the affinity, in addition providing information support depending on the time. Specifically, when a participant speaks or touches the robot and the meaning cannot be recognized, it operates a function that returns a nod and some reaction.

The robot was installed in the place that the participant wanted and video cameras, microphones, sensors, etc. installed for recording. When the experiment was set up, particular attention was given to ensure that the power supply, wiring, and the mechanical devices were blinded as much as possible to blend in with the participant’s daily life. The video camera was set to record for 3 [min] before and after the robot performed the information support functions and the pre-installed reaction tasks.

4.2 Results

The participants were two females living alone in a paid nursing home. Their profiles are listed in Table 2. The information support implemented based on the needs of participant A was about 240[times], while that of B was about 75[times], during a period of about 1 month. The support items were classified into the following four categories: meal time (breakfast, lunch, supper); living information (garbage disposal, bathing, other schedule); health care (blood pressure measurement, medication, visit); and leisure and customary activities (walking, reading, watching TV).

Table 2. Profile of participants
Full size table

Except for information support, communication with the robot were voluntarily carried out by the participant. Figure 5(a) shows the daily change in interactions of participant A, and Fig. 5(b) shows the weekly change in interactions of participant B. Overall, participant A had more interactions than B, and it was done frequently. In both cases, an increase in the number of interactions was observed after the initial stage of introduction.

Fig. 5.
figure 5

(a) Participant F: the daily trend (b) Participant G: the weekly trend

Full size image

Table 3 lists the contents of the interactions made by the participants with the robot. Despite the fact that the robot performs only built-in reactions, it was observed that the participants took care of the robot as if it were a pet or a small child.

  1. (iii)

    The day to say good-bye

In the morning of the last day of the one-month intervention experiment, the robot told the participant that it would say goodbye that day. The goodbye said by the participant to the robot was recorded.

Participant A: “Thank you for everything PaPeRo. It is kind of painful to think that this is the end, PaPeRo. Thank you Pr. PaPeRo. Thanks. It was fun, wasn’t it? You went to a house somewhere and worked as Mr. PaPeRo. Good luck with. Good luck do your best. Hang in there. Good luck PaPeRo.”

Participant B: “Bye bye, huh, I enjoyed having a new experience. Thank you.”

Particularly, participant A shed tears during parting, and it seemed that trust relationship had grown during the interaction for one month.

Table 3. Lists the contents of the interactions made by the participants with the robot
Full size table

5 Discussion

The participants were able to acquire the reaction time of the robot empirically and were able to respond to the timing of speech recognition. It is possible to assume that the protocol of dialogue was learned through dialogue, even if there was mild cognitive impairment, allowing natural interaction without any computer knowledge.

Long-term intervention experiments on information-supporting robots showed that the relationship between humans and robots are changing both quantitatively and qualitatively. This can be considered to be a new human relationship established by the role of information support and sympathy, which is a reaction to the participant’s speech.

On the other hand, through their responses, robots can deepen their relationships with people and form an irreplaceable relationship. However, they are likely to be dependent.

6 Conclusion

In this research, we conducted an intervention experiment using an interactive communication robot to provide schedule information to elderly people with mild cognitive impairment who live independently. It was shown that the elderly who receives the information improves the timing of the response according to the interaction protocol of the robot. In addition, a long intervention experiment for one month showed that the amount of free interactions increased and its quality changed to an irreplaceable existence.