Keywords

1 Introduction

Water is one of the most abundant compounds on earth, and about three-quarters of the world’s area is covered with water. Popular aquatic activities like water sport, water theme park, water game provide players with special experience. Recently, Human-Computer Interaction related research has received significant attention in game experience design. Games research field is advancing from playing with digital content using a keyboard, to using bodies to play with digital content, towards a future where we experience our bodies as digital play [3]. While most previous work has focused on ground-based exercise activities, there has been an immense interest in various water activities [1]. Besides, an underwater robot is designed mostly for industrial, educational and scientific research. In this paper, we are concerned with underwater entertainment applications. Advances in computing, digital and robot technology have inspired the creation of novel water game design. Those advancements let to improved sensing and interacting technologies that support new forms of digital water play.

In this paper, we present AquaBot system towards digital game with physical water. The system supports interaction between human, robot and environment. It includes two parts: underwater robot and wearable wristband. The underwater robot can perceive the environmental information to give corresponding feedback, it can also complete the interactive task according to the commands issued by the player. Now, it can support color, light, vibration and movement changes. Players can send the command to adjust the robot’s state through wristband. We also explore the design factors of interaction with underwater robot from human, device and water environment. Those factors are beneficial to consider when design water-related interaction. Our main contributions are as follows:

  • Proposing a digital water play system supporting multiple interactions and experiences.

  • Using the body to control underwater robot which makes interaction more natural.

  • Exploring the design factors of interaction with underwater robot.

2 Related Work

Water environment is unconventional and sometimes is risky for players to attend, there is less amount of existing HCI work in such an underwater environment than in land domain. Technological advancements such as game console accessories allowing for bodily play, wearable technologies and sensors in mobile phones enabling digital games [3]. In this paper we focus on interactions beyond the screen and land situation, we explore the digital water play and water-related interaction experience.

With the advances in technologies, studies on water games have been made. Water Ball Z is a novel interactive two-player water game that allows kids and young adults to “fight” in a virtual world with actual physical feedback [15]. It engages people in a real and fun experience where a hit is physically manifested via a water spray. The paper by Choi et al. [1] proposed MobyDick, a smartphone-based multi-player game in which a group of swimmers coordinates themselves in order to achieve the common goal of hunting an AR monster in the water. AquaCAVE [5] is a computer-augmented swimming pool with rear-projection acrylic walls that surround a swimmer, providing a CAVE-like immersive stereoscopic projection environment. Bellarbi [7] explored augmented and mixed reality technologies in aquatic leisure activities, he proposed an underwater AR game using the DOLPHYN device which can be used at water surface as well as underwater using a tuba. Jain [6] presents Amphibian VR system that provides an immersive SCUBA diving experience through a convenient terrestrial simulator. Users lie on their torso on a motion platform with their outstretched arms and legs placed in a suspended harness. It could simulate buoyancy, drag and temperature changes through various sensors. In underwater game, wireless data exchange among users and devices need to be supported and it also needs to overcome several key technical challenges as well as human factor constraints.

Recently, the demand for advances in underwater robot technologies is growing and will eventually lead to fully autonomous, specialized, reliable underwater robotic vehicles [9]. Some researchers have successfully applied underwater robots to digital water games. In the experience process, we have to consider the interaction between human, underwater robot, and the environment. Swimoid is a swim support system using an underwater buddy robot [4] to recognize, follow, and present information to the swimmer. While there are some problems in this system, it’s difficult to control the robot with increasing water resistance. Users can’t interact with the display for a long time because of the limitation of breathing capacity. Pell [8] presents Gravity Well an interactive shallow-water system that supports bodily play through water-movement interactions. The robot can provide a real-time visual response to the player’s human-aquatic movement interactions and change color with touch.

Within the context of the underwater environment, there are many factors that may influence the quality of interactions. The existing studies on underwater interaction design strategies may be classified into two categories: giving instructions to robots from human; tracking human using some sensors mounted on robots [4]. Wu [10] proposed a preliminary human factors model to study interactions between human and semi-autonomous underwater vehicles. Raffe [11] proposed a taxonomy of six degrees of water contact and four categories of player and computer design features to be considered during the design and development of digital water play experience.

Inspired by previous researches and technologies, we propose AquaBot an interactive digital game with physical water which can give different color, light, movement feedback according to the interaction with human and environment. We also explore the design factors of interaction with underwater robot with the AquaBot system.

3 System Description

AquaBot system contains two parts, an underwater robot and a wearable wristband. The wristband is a waterproof part worn by the player to response for the interaction with the robot. The underwater robot is the main interactive device and plays a critical role in the whole system because of its interactive property like color, light, movement. Compared to the studies about HCI and HRI in land domains, the biggest challenge of underwater interaction is that some communication channels available in land become unavailable underwater. We need to test and find a suitable communication channel that can be used in our system. Besides, all of our devices need to be waterproof and can be immersed in water.

3.1 Interaction Framework

Figure 1 illustrates the interaction framework of AquaBot system, it includes three essential factors: AquaBot, human and water environment. AquaBot amplifies the interactions between underwater robot, wristband, player, water environment through visual perception, haptic perception and behavior interaction. The system has two data input ways: the monitoring data from the wristband and the data collected from the underwater robot. The wearable wristband is used for detecting hand gesture and send commands to the robot. The underwater robot consists of LED, vibration motor, propeller, temperature sensitive material for interaction output. The interaction between the AquaBot system and the environment is mainly through sound and temperature information. Human interacts with the system is through body gesture and active operation. The interaction between human and water environment is reflected in the contact degree between hand and water: above the water surface, on the water surface, and under the water. Arduino for computational processing, and Bluetooth for data communication.

Fig. 1.
figure 1

Interaction framework of AquaBot

Full size image

3.2 Underwater Robot

The appearance of the underwater robot is a bionic design based on jellyfish and is made by 3D printing (Fig. 2). At the outermost layer of the robot, we applied a temperature sensitive paint. When the temperature is below 31 °C, the paint will show blue, and when the temperature is above 31 °C, the blue paint will become transparent. So that it can interact with the environment to change its color according to temperature.

Fig. 2.
figure 2

(a) Underwater robot of AquaBot; (b) Internal structure of underwater robot.

Full size image

The robot is powered by a 12 V lithium battery and can continue to work underwater for 20 min. We use Bluetooth as the communication method for several reasons. Firstly, our water environment is set in a glass jar with the size of 80 × 45 × 45 cm. Bluetooth module can work in this underwater environment; Another Bluetooth module is installed on the wristband, it’s convenient for data transmission; The connection of Bluetooth is easier than other methods while meeting system function. But Bluetooth can only achieve short-distance data transmission, it needs the player and robot to stay within 5 m.

Under the action of the propeller, the robot can perform floating and dive movement under water. The propeller provides 11 N thrust at 12 V. We use a duct to help it generate more force. We also add counterweights inside the robot to keep it balanced under water. The Arduino Pro Mini is used to control the robot. The water sensor can detect whether the robot is in a water environment. When leaving the water environment, the propeller will stop working. The sound sensor can detect ambient sound with a volume threshold of 50 dB and the robot will be activated when the sound is above 50 dB for more than 1 s. The vibration motor provides a vibrating touch. The LED strip is available in a variety of colors. If no interaction occurs for more than 5 min, the robot will start to “sleep”. The connection part of the robot is waterproofed by a silicone ring.

3.3 Wristband

The wristband needs to be worn by the player to participate in the game (Fig. 3). It acts as an operation tool between the human and the robot. It converts the human action commands into data and sends them to the robot. The wristband is powered by a 9 V dry battery. The Arduino Pro Mini is used as the control system. Data communication is done with the Bluetooth module. We use the MPU-6050 module to detect gestures. The water sensor on the side is used to detect the contact degree between the hand and the water. It can be detected that the hand is not in contact with water, is located on the water surface, and is completely immersed in water. The other electronic components of the wristband are placed in the waterproofed box.

Fig. 3.
figure 3

(a) Wristband of AquaBot; (b) Internal structure of Wristband.

Full size image

3.4 Interaction and Control

AquaBot supports interaction with human and the environment. When the ambient temperature is lower than 31 °C, the robot remains blue, and when the temperature is higher than 31 °C, the robot will change color from blue to transparent (Fig. 4). When the ambient sound is above 55 dB, the robot will wake up and flash the blue light (Fig. 5).

Players need to wear the wristband when interacting with the robot. AquaBot will give feedback based on human gestures. The player claps, the robot is activated when the sound exceeds 50 dB and the light flashes. When the hand is not in contact with the water, the arm raises upwards, the robot will float up; when the arm moves down, the robot will dive; when the user swings the arm to the left or right, the color of the robot can be switched; when the hand puts over the robot and touches the water surface, the robot will float up to touch the player’s palm and the light becomes red; when the hand is completely immersed in the water and close to the robot, the robot will dive down. Catch the robot and take it out of the water, the propeller will stop working. At the same time, the robot will flash the light and the vibration module is triggered to provide the hand with a vibrating touch.

Fig. 4.
figure 4

Robot changes color with temperature: (a) The water temperature is 28.9 °C and the robot remains blue color. (b) The water temperature is 56.3 °C, the blue paint gradually fades and becomes transparent. (Color figure online)

Full size image
Fig. 5.
figure 5

Robot is awakened by the sound and flash light. (Color figure online)

Full size image

4 Interactive Application Scenarios

In order to improve and evaluate the performance of AquaBot and understand how users would use our system, we organized 2 scenario tests for different interaction: body gesture interaction and underwater interaction. The tests were implemented indoors, the robot was placed in a 90 × 45 × 45 cm glass jar. Added water to the glass jar, the depth of the water was 30 cm. All participants are aged from 20 to 32 and were recruited from our university. Users wore wristband device and interacted with underwater robot to accomplish tasks. After each test, we conducted semi-structured interview and asked them some questions about the interaction and experience.

4.1 Scenario 1: Body Gesture Interaction

Scenario 1 involves two environments: the user is in an environment outside the water, and the robot is in the water. The user needs to wear the wristband and then will be introduced the function and operation of the AqucaBot system. In this scenario test, users need to complete two tasks. The first task is to activate the AquaBot and control its floating and dive movements by gestures (Fig. 6a). The second task is to change the color of the robot’s light through gesture control and set it to the color they like (Fig. 6b).

Fig. 6.
figure 6

Body gesture interaction: (a) Raising arm to control the robot float up; (b) Swing arm to change the color of robot. (Color figure online)

Full size image

All users showed great interest in the interaction process. Some users said that when they experienced the AquaBot, they were surprised that they could control the underwater robot without touching the water. P3 said the system was very interactive and provides a new water game experience. P6 said that this system gave her a new understanding of underwater robots. During the test, we found that the task of controlling the robot’s movement was the most popular, and some participants experienced this task many times.

4.2 Scenario 2: Underwater Interaction

In scenario 2, users need to touch the water surface and immerse in the water. The water sensor in the wristband would perceive the situation when hand contacts with the water. In this scenario, users are told to complete 2 tasks. The first task is to put the hand on the top of the robot and let the palm touch the water (Fig. 7a). The second task is to immerse the hand underwater, catch the robot and take it out of the water (Fig. 7b, c).

Fig. 7.
figure 7

Underwater interaction: (a) Putting hand on the water surface, the robot would float up, touch the palm and bright red light; (b) Immersing hand in the water and closing to the robot, the robot would dive down to try to escape; (c) Catching the robot and taking it out of the water, the robot would flash red color and vibrate. (Color figure online)

Full size image

The test results reflect the user’s surprise for the AquaBot, especially when the hand puts on the water surface, the robot floats up to touch the user’s palm, and lights up red. One of the users said: “This experience made me feel like a pet is approaching me, then gently kissing my palm and shy blushing.” This interaction makes the underwater robot emotional.

Another user said that he likes to catch robots underwater. In this process, the robot would dive down when the hand closes to it, and when taking it out of the water, it would vibrate and try to get rid of the arrest. But some users said that the robot’s up and down movement is too singular, and if it can support multiple directions of movement in the water, it will make this interaction more interesting.

In this scenario, we just let the user immerse the hand in the water to interact and not all of the body immerse in the water. This setup is sufficient to reflect the existing functions of the AquaBot, but there are some limitations because we have not fully observed the interaction of the entire body when it is submerged under water.

5 Design Factors

Human-Computer Interaction Design is accepted and rapidly spread over the world, at the preliminary stage, it applies to visual interface design, such as multimedia game and immersive circumstance design, later interaction concern becomes one of the product entities [12]. Through the process of designing AquaBot system, we learned that some designers are unfamiliar with digital water play and interaction with underwater robot. And many land-based digital design principles and methods are not suitable for underwater environment. These issues inspired us to summarize and propose a series of design strategies of digital game with physical water. Digital water play design needs to consider multiple factors, and these factors may influence the quality or even the success of the interaction. In this paper, we discuss three factors: water environment, human, device (Fig. 8) on the interaction of underwater robot.

Fig. 8.
figure 8

Design factors of interaction with underwater robot.

Full size image

5.1 Water Environment

We discuss two environments: the real open water environment such as the ocean and lake, the non-open water environment such as swimming pool and water tank. The temperature, depth, stability, visibility, and light of water are all factors that can influence the outcome of the interaction. These factors can also be used as triggers for interaction under certain conditions. In addition, the complex terrain of the underwater environment and other disturbances around the water environment can also affect the interactive experience.

5.2 Human

Human play an important role in digital water play. For human, the primary consideration is physical condition and water skill in the underwater environment, which is important for some interactions that require full body immersion into the water. Moreover, the educational background, technical level, and operational ability of people are also factors that can affect the system. The ways in which users participate in interaction include body gesture, movement, voice control, sight interaction, brain wave control, facial expression, perception (visual, auditory, olfactory, taste, touch), pulse, body surface temperature, galvanic skin response, etc. If the interaction scenario requires multiple people to complete, then the person also needs to have the ability to work in teams.

5.3 Device

Due to the special environment under water, the device needs to be waterproof, and underwater communication is also a challenge. The four means of communication through underwater are using acoustic waves, EM waves, optical signals, optical fiber cables [13]. In these ways that acoustic waves generally carry information inside the water because of various drawbacks of other signals. In addition to the limitations of the technology itself, noise near the water can also affect the transmission, such as waves, ships, weather, etc. Besides, the multipath channel and Doppler Effect will also affect underwater transmission [13].

The size, appearance, operation interface, operation mode of the device are all factors that need to be considered in underwater interaction design. And they are the most relevant factors to human. The power of the device underwater is mainly from the battery. Generally, we need an operator to control underwater device like a remote controller, a mobile phone, etc. But this is a traditional way of controlling, the interaction is unnatural and lacks some interests.

6 Discussion and Future Work

AquaBot is a digital underwater play system that includes an underwater robot and a wristband. This system can support the natural interaction of people in three different water environments. But there are still some limitations in the current system. First, our systems are currently only used in the shallow and stable water environment, it is not tested in open water environments such as lake and ocean. Then we use Bluetooth as the communication method, which only supports transmission within a few meters of water. There are some problems in the design of the underwater robot that needs to be improved, and the movement of the robot under water is not very stable. During the movement, the robot has some inclination in the water. Moreover, the robot is still unable to support long-term work and needs to be charged in time.

In the future work, we will continue to study the research of digital water play. We will improve the underwater movement of AquaBot, adjust the structural design, and ensure that the robot can maintain stable floating and dive underwater. We will also explore the underwater multidimensional motion of robots to provide more forms of interaction. For underwater communication, we will adopt more mature and stable methods, such as acoustic waves. We will also try to increase the depth of water and study the interaction of complex underwater environments. Study other parts of the body when contact the water and participate in the interaction. We will also enrich the interaction between the environment, human and device, and provide more interesting experiences.

7 Conclusion

This paper introduces AquaBot, a novel digital water play system that provides water play experience with digital technologies. The system includes two parts underwater robot and wristband which support interaction with human and environment. Through the exploration of different scenarios, it can be seen that bringing digital and intelligent elements into water play provides new interactions and experiences. In this paper, we also discuss the design factors of interaction with underwater robot, we hope this will inspire more researchers to study digital water play.