Keywords

1 Introduction

The recent progress of digital technologies dramatically increases one’s cognitive overload. For example, a large amount of information is offered to citizens via public displays located on trains, stations and streets in our everyday commute in big urban cities. Modern social media try to steal our available attention with advanced computing technologies. To make everyday life more comfortable and peaceful, information should be more ambiently and playfully delivered to individuals only when the information is truly necessary.

This paper proposes a notification system named Ambient Bot that intimately and ambiently presents everyday information to people. It requires them to wear a head-mounted display (HMD) through which they see a virtual creature in the real world via augmented reality (AR) technologies. This creature speaks and presents everyday information of possible interest but only when they focus their attention on the creature itself.

2 A Basic Interaction in Ambient Bot

The goal of Ambient Bot is to offer an implicit light-weighted interaction method wherein it is easy to access necessary information, in the any places and situations of everyday life; for example, when a user is in a public space, such as a train station or when walking on the street. Ambient Bot adopts the his/her gaze as the basic input modality, while its visual text messages and spoken audio are the output modality. Interactions using eye contact cannot be used for accessing complex information, but these interactions can enable implicit natural interactions, bypassing explicit interactions like those related to the use of traditional controller devices.

Ambient Bot allows people to acquire information simply by using eye contact with a virtual creature that continually floats within their field of view when they really need information. The virtual creature not only speaks but also displays texts in a translucent window against an image of the real world. Figure 1 shows a user’s perspective, i.e., what he/she sees via an HMD (Nexus 6 used as HMD and Microsoft HoloLens) when using Ambient Bot. When eye contact has not been made, the virtual creature automatically moves to a position that does not interfere with his/her view, ensuring that he/she is not strongly conscious of the creature’s existence. To actualize interaction and for the creature to make an eye contact, it needs float around his/her field of vision as described above. The current design chose a jellyfish character as a floating virtual creature, as shown in Fig. 1 because it would be inappropriate to display humans, dogs or cats as floating creatures.

Fig. 1.
figure 1

An overview of Ambient Bot

Full size image

3 Accessing Multiple Information Sources in Ambient Bot

3.1 Pull-Based and Push-Based Interaction Methods

Two methods are investigated for people to access everyday information; the pull-based interaction method and the push-based interaction method. In the pull-based interaction method, people actively access information that they want to know. In the push-based interaction method, information is actively provided to people and they can passively access the information. The push-based interaction method includes not only notifications but also accidentally receiving information to make people mindless and frustrated. For example, people may acquire information about souvenirs via advertisements when they walk in a train station. Such interaction is classified as push-based interaction because they acquire the information passively from the outside world, and they may not expect to receive the information. We have developed two prototype implementations of Ambient Bot; one is based on the pull-based method and the push-based method is adopted by another.

3.2 Pull-Based Interaction in Ambient Bot

In the prototype based on the pull-based interaction method, since its user’s objective is to access information intentionally, he/she needs to indicate to Ambient Bot what kinds of information he or she is currently seeking. In conventional information access services, people usually select a specific application on the desktop of a personal computer or a smartphone home screen, acquiring the information they want by clicking the target application’s icon. For example, if a user wants to know the weather tomorrow, he/she usually picks up his/her smartphone, taps the weather forecast application icon, and thus, acquires the expected information. However, in the case of the basic Ambient Bot concept, user interaction instead entails making eye contact with a jellyfish. However, this eye-contact approach is likely insufficient for accessing complicated information because it is difficult to select the appropriate information with only eye contact.

This basic interaction design as described in the previous section can be easily enhanced for accessing multiple information sources. A user can configure the types of contents to be accessed in advance, according to the time and situation. This design allows the user to acquire the desired information without hindering the implicit and natural interactions offered by the basic Ambient Bot interaction design. In the enhanced Ambient Bot, multiple virtual creatures appear around a user, with each virtual creature representing a different informational category. Figure 2 shows an overview of this interaction design. A user chooses a desired information category by selecting a virtual creature floating around him/her, and making eye contact with one of them. Showing multiple virtual creatures in the real world where a user is could lead to a cognitive overload, yet this load could also be reduced by carefully arranging the viewable positions of the creatures with respect to the time and specific situation.

Fig. 2.
figure 2

Accessing contents via multiple creatures

Full size image

3.3 Push-Based Interaction in Ambient Bot

In the prototype implementation based on the push-based interaction method, when Ambient Bot wants to provide information, a virtual creature appears in the real space, but the position is almost the edge of a user’s view; thus, the creature’s appearance does not consume his/her cognition too much.

When he/she makes eye contact with the creature, the creature provides information. In this prototype, the appearance of a virtual creature means that the creature wants to notify something to him/her. The creature moves out of his/her sight and disappears after providing the information or its user does not pay an attention to the creature for a certain time.

When a creature is appeared in his/her view, Ambient Bot shows a notification icon that indicates which content it wants to notify above the creature as shown in Fig. 3. For example, when a creature wants to notify a news message, the icon on the creature indicates that there is a news message for him/her. When he/she makes an eye contact on the creature, an appeared message window includes a news message.

Fig. 3.
figure 3

A notification icon above a creature

Full size image

4 User Studies

We conducted the first user study experiment to investigate the feasibility of the Ambient Bot’s pull-based interaction design. We recruited ten participants, including eight males and two females, all of whom are university students between 21–24 years old who usually access information with their smartphones.

Participants freely used Ambient Bot at the beginning of the experiment. Then, they configured the system with the appropriate information, according to the current time and by matching the virtual creature with its corresponding content. Next, participants used Ambient Bot in the following three situations: working on daily tasks at a desk, sitting in a train, and walking outside.

Figure 4 shows results with respect to three situations based on their decisions regarding configurations that either displayed an article within an information window or read the article using a voice synthesizer. In the first situation, when working at the desk, nine out of ten participants displayed an information window, and whether or not they required that information to be voiced depended on the participants involved. Participants who rejected using the voice during work read only the critical part of the article, and then felt annoyed at listening to the entire article. On the other hand, a participant who wanted to listen to the article via voice said, “I was less burdened when I used the voice assistance; rather than reading the whole article by myself, I could concentrate on my work”.

Fig. 4.
figure 4

Information delivery methods

Full size image

In the second situation, when sitting in a train, all participants wanted to display an information window, yet their opinions about using the voice were divided. Participants who refused to use voice noted that, “The voice conflicts with in-train broadcasting,” while participants who wanted to use the voice noted that, “The broadcast information on a train is not necessary for me to listen to.” Unlike music, the reading of an article in a voice, seriously conflicts with in-train broadcasting. Using voice information in Ambient Bot may disturb those who normally use in-train broadcasting to provide their current location. On the other hand, for most of them, visual information on a train is unnecessary and displaying an information window did not disturb their activities on the train.

In the third situation, when walking outside, many participants refused to display an information window. To ensure people’s safety while walking, they needed to carefully distinguish the necessary information from within their eye-sight. As a result, the presentation of a window to disturb their eye-sight, should be avoided.

The second user study to investigate the push-based interaction method was conducted using nine participants (8 males and 1 female, average age: 22.7). In this user study, participants chose their preferred display modes for three typical content types: newsflashes, e-mails and the user’s schedule. For each type of content, the participants selected whether a notification icon is presented or not, and whether a message window is presented or not. The message window presents content, and the content may be read by the creature’s voice.

In this user study, we interviewed the participants and asked them why they chose the configurations. Of course, some participants always turned the window on, but other participants changed the configuration to display the message window according to the content. The reason provided for turning the message window on was mostly because they may miss listening to content via voice only, and the reason for turning it off was because the content in the message window does not matter if participants missed listening or not. For e-mail content, all participants turned on the message window. From the interviews, they did not want to miss the email content. In addition, several participants said they do not need a function to read e-mails via a voice because e-mails are usually read by a user’s eye, and not listened to by a voice.

The icon was turned on by most participants, except participant C. Several participants said, “The icon was turned on in order that Ambient Bot informed what information would be presented.” Participant C selected to display the icon only when the message window was not displayed. He commented in the interview “When the message window comes out, I can read the message quickly; thus, I can fully understand what content it is. In the case of only voice, the icon is displayed soon to see what content it is.” Hence, we understood through the interviews that all participants wanted to know at the beginning what type of content Ambient Bot wanted to present.

The results of the interviews suggested that the modality of information cues should be designed according to the existing services’ modalities.

5 Related Work

Welbo [1] is an AR-based agent system similar to Ambient Bot, where a virtual agent appears in the real world, and the agent can talk to a user. However, in Welbo, a user needs to speak to the virtual agent for interacting with the agent. The approach is heavy weighted because speaking requires more human consciousness than eye contact that is adopted in Ambient Bot.

In past research, a concept named Pervasive Ambient Mirrors reflects people’s current situations to influence their behavior [5]. In [4], slow technologies enable daily objects to ambiently represent some currently useful information for a user. Recently, push-based notifications on smartphones were studied [2] and an ambient notification method using an eyeglass device was reported [3].

6 Conclusion

Ambient Bot is a playful and ambient notification system that uses AR technologies to offer everyday information using casual interactions. This paper described some insights of Ambient Bot, that emerged from the two user studies.