Keywords

1 Introduction

In the concept of the Internet of Things (IoT), industrial environments contain a vast number of electronics and IT systems [1]. It is estimated that by year 2020 fifty billion devices will coexist in our environments, connected and interacting with each other. Industrial equipment has embedded sensors and can communicate real-time sensory data. As the machinery grows increasingly complex, such tasks as configuration and maintenance demand higher human expertise, and operators have to specialize in handling complex systems. Moreover, the IoT concept of connected machines extends beyond industrial environments to daily life. Domestic machines are equipped with chips and sensors as well, and servicing them may be rather demanding.

A calm technology implies that humans are informed but not overloaded with information and that technology should only require the user’s attention when necessary. To that end, based on the biophilic design and calm computing principles, we developed a nature-inspired visual language, BioIoT, to communicate sensor information.

In this work, we explored how to convey information about complex, real-time processes via nature-inspired visualizations. Our objectives were multi-levelled. First, we aimed to achieve a representation that is both easy to understand and aesthetically engaging and that increases the overall well-being. Second, we tested our hypothesis that if machines were perceived to be more like living beings, operators would take better care of them, which ideally would translate into a better (preventive) machine maintenance.

In this paper, we introduce the BioIoT concept and we present two use case scenarios under which it can be applied and we demonstrate its potential benefits in households and at workplaces.

2 (Re)designing Human-Machine Interaction via the Biophilic Hypothesis and Calm Computing

We propose to use nature-inspired representations to communicate IoT information in an appealing and engaging manner. In the BioIoT concept, an IoT device can connect to a living proxy that reflects the general state of device.

The user interacts with both the IoT device and the nature representation, as depicted in Fig. 1.

Fig. 1.
figure 1

BioIoT concept - IoT device generates sensory information, which is communicated through virtual nature-inspired representations.

Full size image

The BioIoT concept is based on four theoretical premises:

  • Biophilic Design: by incorporating contact with virtual nature elements into everyday tasks, we expect to contribute to the user’s overall well-being;

  • Calm Computing: by presenting substantial amounts of IoT data via a single peripherally noticeable representation, we expect to decrease the information overload and only request the user attention when necessary;

  • Persuasive Technology: by exploring and stimulating the human tendency to perceive devices as social actors, we expect users to interact with the devices as though they were alive. As a result, behavior or attitude promoting greater care about the device can be enforced, ultimately improving its performance;

  • Positive Computing: by designing and developing technology that enhances well-being factors, e.g., positive emotions, empathy and compassion, we expect to positively affect the users’ psychological welfare.

2.1 Biophilic Design and Technobiophilia

The biophilia hypothesis states that humans are attracted to nature and all living processes [6]. Research confirms the human preference for natural environments and shows that our well-being, productivity and creativity greatly improve merely by being in direct contact with nature, e.g., gardens, parks and nature window view [7]. The contact with nature analogues, e.g., nature-resembling colors, patterns, materials and art, as well as natural-like light and sounds, also proved to be beneficial [4, 12].

Living in urban environments does not satisfy this human inclination. Efforts have been made to re-design cities, buildings, workplaces and homes to include natural elements. Biophilic design [4] is a method of constructing work and living spaces in a way that fulfils the humans’ fundamental need to connect with nature.

The positive effects of biophilic design extend beyond the workers’ well-being and job satisfaction and can be translated into economic advantages for employers since human productivity increases upon contact with nature. Both direct (e.g., the number of pieces produced and time required for task completion) and indirect metrics (e.g., illness, absenteeism, staff retention, learning rates, stress levels and fatigue) productivity metrics clearly indicate a remarkable growth, upon which organizations can capitalize [12, 13].

The notion of technobiophilia within the biophilia hypothesis refers to the ‘innate tendency to focus on life and lifelike processes as they appear in technology’ [22]. Technology can provide access to nature when nature is not available and can help to fulfil the human’ attraction to nature [22].

Nature representations, e.g., artificial gardens and pets, as well as easily accessible feedback to data, can lead to reflection and posterior behavioral change [1, 2, 11]. For example, Data Fountain uses water jets to show the currency rates of Euro, Yen, Dollar to achieve a general ‘feeling’ of interconnection between the currencies [15]. Similarly, plant displays have successfully been employed to represent data and provide affective feedback [3].

2.2 Calm Computing, Persuasive Technology and Positive Computing

Weiser’s vision of calm technology [16, 17] is now within reach due to the Internet of Things. Calm technology emphasizes calm, suggesting that humans need to be informed but not overloaded with information. As such, technology should be transparent and only require attention when needed. For example, in industrial environments, the perceived system complexity from the operators’ perspective should be reduced [18].

With that regard, establishing the balance between esthetic and informational quality is crucial. Some messages (e.g., fire alarms) are too urgent to be subtle and must be delivered unambiguously. Yet even in the case of potential disaster, the level of urgency can gradually be built up over time. This increasing urgency should be presented ‘calmly’ in the environment in order to create optimal conditions for vigorous human action to control the damage or even prevent the accident from happening [15].

Research has been carried out to study how people interact with machines, e.g., robots [8]. Human perception of the machines’ intelligence and consciousness, combined with anthropomorphic factors, i.e., appearance, gestures and emotions, is known to change the dynamics of human-machine interactions. A possible explanation is that machines tend to be perceived as social actors. In that context, some studies [9] indicate that social rules and dynamics should be applied to design systems that have the potential to change the user’s behavior. For example, the reciprocity rule “If you help me, I feel that I should help you” is also valid for human-machine interaction [10]. Humans are hardwired to be social and to experience emotions towards others humans and living beings, e.g., empathy and compassion.

Bartneck et al. designed a study in which two groups of users were asked to play a game collaboratively with a talking cat robot [19]. In one group the robot was very helpful, while in the other one not so much. Eventually, the groups were instructed to turn the robot off. The users’ perception of the robot’s intelligence and agreeability had a strong effect on the ease of switching it off. The group with the most helpful robot hesitated almost three times as long to switch it off since ‘killing’ the machine was difficult because of an emotional connection with the machine [20].

Persuasive technology refers to technology designed to change the users’ attitudes or behavior through persuasion and social influence [21]. B.J. Fogg proposes the functional triad framework for thinking about the roles technology plays. The functional triad shows that technologies can function as tools (increasing capability), media (providing experience) or social actors (creating a relationship). These functions capture how people use or respond to any technology and most technologies are a mix thereof.

A tool can be persuasive, for example, by making a target behavior easier to establish. A medium can be persuasive by allowing people to rehearse a behavior and a social actor can be persuasive by rewarding people via a positive feedback or modeling a target behavior or attitude.

Technology should be designed not only to impact productivity but also to enhance the humans’ well-being. Positive computing is a research area and practice that uses well-established methods in such fields as psychology, neuroscience and economics to design and develop new technologies that foster psychological well-being and human potential [14].

Positive computing relies on psychological principles of motivation, engagement, relatedness, autonomy, competence and compassion to design technology that enhances human lives. The objectives of our work were to communicate sensory information of machines in a meaningful way based on the concepts of biophilia and positive computing and to add value by increasing human well-being via representations of nature.

3 Concretizing the BioIoT Concept

The BioIoT concept contains four main elements, as depicted in Fig. 2:

Fig. 2.
figure 2

BioIoT concept: an IoT device generates sensory data communicated through virtual nature-inspired representations.

Full size image

  • an IoT device that generates sensory data and communicates it using a lightweight messaging protocol, e.g., MQTT - Message Queue Telemetry Transport;

  • a backend application that handles communication and encodes the IoT data using a nature-inspired language;

  • an output-specific user interface, e.g., display with 2D visualization, a head-mounted display with 3D augmented reality visualization;

  • a nature-inspired language for encoding the IoT data, e.g., visual elements, sounds, smells.

3.1 Nature-Inspired Visual Language: A Virtual Tree

To implement our concept we designed a virtual tree model for the following reasons:

  • Trees are appreciated and recognized as a fundamental part of a natural environment.

  • Changes in a tree’s condition are easily recognized by everyone, regardless of culture. We intended to minimize the effort required to learn a new language by exploring this prior knowledge.

  • Trees have a limited number of parts that can change: flowers, branches, leaves, trunk, and roots. This is of advantage while designing the virtual tree model since it reduces the design complexity. However, this can also be considered a disadvantage since number of variations is limited.

The appearance of the designed virtual tree depends on the foliage density, the foliage color, the presence and size of flowers, and the size of the trunk. The roots of the tree were not considered since we aimed to create a realistic environment, in which they are not visible. In addition, to enhance the virtual tree model we integrated the wind and sun light effects.

We created the virtual tree using the SpeedTree modeler and made it available in 2D and 3D versions.

The proposed virtual tree model has several levels of encoding complexity.

Foliage Color

Tree foliage colors typically vary between green and red. Figure 3 shows two virtual trees with green and red foliage. To achieve a realistic foliage, the leave colors were not completely uniform.

Fig. 3.
figure 3

Two examples of virtual trees with different foliage colors. Left: a tree with green leaves; Middle: a tree with red leaves; Right top: detail of the green leaves; Right bottom: detail of the red leaves. (Color figure online)

Full size image

Foliage Density

Tree foliage densities vary greatly. Figure 4 provides four examples of virtual trees with different degrees of foliage density.

Fig. 4.
figure 4

Four examples of virtual trees with different degrees of foliage density. Left top: foliage not present; Left middle: low foliage density; Left bottom: medium foliage density; Right: high foliage density.

Full size image

Flowers

In the tree model, the flowers can be present or absent. When present, the flowers can have three sizes (small, medium and large) (Fig. 5).

Fig. 5.
figure 5

Four examples of virtual trees with/without flowers. Left top: flowers absent; Left middle: small flowers; Left bottom: medium-size flowers; Right: large flowers present.

Full size image

Sun Light and Wind Animation

The sunlight and the wind animation add movement to the virtual tree model, creating a more realistic effect. The wind had various degrees of intensity (low, medium and intense), and the sunlight effects could be present or absent (Fig. 6).

Fig. 6.
figure 6

Two virtual trees without (left) and with (right) of the sunlight effect.

Full size image

4 Use Case Scenarios – Bringing Nature to Our Lives

The following use case scenarios describe two applications of the BioIoT concept.

Peter and Anna live and work in an urban area and, like many of us, have little or no chance of contact with nature in their daily lives. However, technology and data seem to be present at all times.

4.1 Use Case A: Peter Has a Green Thumb

Peter works in an office and every morning enjoys a cup of coffee while checking his calendar and messages. The coffee machine in Peter’s office is a new IoT model, but the machine is a nuisance for him: ‘It’s a nightmare! No one services the machine or performs maintenance unless it stops brewing coffee! The other day all warning lights were on… Everyone wants to drink coffee, but no one cares for the machine until it stops working’.

The following day, next to the coffee machine there was a screen with a beautiful virtual tree. Everyone was informed that the tree reflects the coffee machine’s condition.

A few days later, Peter notices that his colleagues (and he himself) began to interact with the coffee machine in a more positive way, making sure that it was serviced and maintained in time.

After reflecting on his own actions, Peter explains “I just want to make sure that the tree looks its best. And it seems that no one wants to be responsible for killing it!”

Due to this change of behavior and attitude, Peter is now able to fully appreciate his morning coffee.

Study: A Green IoT Coffee Machine

We conducted a study to validate the participants’ ability to understand the proposed nature-inspired visual language (virtual tree) and evaluate the appeal of the BioIoT concept [5, 23].

To that end, we created an IoT coffee machine prototype enhanced with sensors and able to communicate the IoT data to a MQTT server. In addition, we developed a desktop application in Unity to encode the IoT data from the coffee machine, creating a multi-surface projection of the tree (four cameras) onto a holographic display.

We encoded the IoT coffee machine’s sensors data as follows:

  • The foliage density represents the three fill levels of the water tank.

  • The foliage color represents the temperature.

  • The presence or absence of flowers represents the maintenance status.

  • The size of the flowers represents the time until maintenance is required.

  • The dead tree (no leaves) represents critical error.

  • The presence/absence of the sun light animation represents the presence/absence of a coffee capsule.

Twelve individuals (8M, 4F) participated in the study, which confirmed that the nature-inspired encodings are readable. The participants recognized the status of the machine based on the virtual tree features. However, the comprehension declined when a large amount of sensor information was displayed in a single visualization. All participants considered the interaction with the BioIoT pleasant and agreed that the machine states representations were overall aesthetically satisfying. They also felt more inclined to perform machine maintenance (take “care” of the machine) than they would if classic state representations were to be applied.

4.2 Use Case B: Anna Enjoys Walking Between the Trees

Anna works in a warehouse and is responsible for equipment maintenance, e.g., conveyors, picking stations, storage towers. Anna performs several tasks daily to ensure that the operations run smoothly. Some examples of Anna activities are: installing and (re)configuring equipment, performing preventive maintenance and diagnosing and solving problems (e.g., repairs, spare parts). In addition, Anna has to constantly monitor the warehouse to identify problems with the equipment. She typically wears a head-mounted display and uses the augmented reality performance support system. According to Anna, the system has three very useful features: step-by-step instructions while servicing a machine, direct communication with experts for solving specific problems and access to in-context information dashboards about the warehouse.

A new version of this system offers a nature view feature, with information encoded using virtual trees. Anna personalizes the encoding and defines which equipment or clusters of equipment are connected with each tree. Now, when Anna is monitoring the warehouse, the nature view is activated and she can enjoy walking through a virtual forest.

Anna is very satisfied: ‘It is much nicer to be informed via the trees than via red lights! And it is gratifying to see the beautiful trees after solving the issue.’

Study: Scaling the BioIoT Concept

We conducted a study to validate the participants’ ability to perceive visual changes and fast-changing states on a larger number of IoT devices encoded using the nature- inspired visual language [20, 23].

Following on Healey et al. work on pre-attentive processing of visual features [24], we investigated accomplishing target detection tasks by relying on variations in hue and in form. The former was represented by the foliage color green or red, and for the latter we applied our virtual maple tree model and a palm tree. We developed a factory model composed of 49 IoT machines and a Unity mobile application that randomly-generated states of the machines and overlaid an augmented reality tree next to the machine.

Sixteen individuals (12M, 4F) participated in the study, which showed that the BioIoT concept can be applied to a larger number of IoT devices and that changes in the hue and form features are pre-attentively perceived (200 ms or less).

5 Discussion

As we increasingly engage with IoT technologies it becomes necessary to find new ways to interact with such large amounts of data. The BioIoT concept proposes to communicate sensory information through nature inspired representations.

As referred, the BioIoT concept is grounded in four theoretical premises: biophilic design; calm computing; persuasive technology; and positive computing.

The biophilic design and calm computing are the base of our concept, which aims to impact positively the wellbeing of the users. In addition, our concept persuades the user to change their behavior.

Accordingly with Fogg’s taxonomy persuasive technologies can function as tools, media, or social actors [12]. The BioIoT concept comprehends these three functions.

Firstly, it functions as a tool by providing all the necessary information to infer the general state of an IoT device in a single visualization, which allows an earlier detection of issues. For example, if the user notices that the tree foliage density is decreasing, it is likely that there is lack of water in the machine. Therefore the user can act before the lack of water becomes a critical problem.

Secondly, it functions as a media by allowing users the opportunity to explore cause-and-effect relationships, e.g. if the user does not take care of the machine, the tree will degrade or even die. Up to now we used nature inspired visual representations, e.g. virtual trees in augmented reality. However, we consider to extend it to enrich the user’s experience, e.g. addition of audio or scents.

Lastly, it has a strong social actor functional role. It persuades the user in multiple manners, such as: reinforcement of the emotional connection with the IoT devices, e.g. empathy and kindness towards the virtual tree; the act of caring for the IoT device rewards the user through an aesthetical and pleasant visualization, e.g. a beautiful tree is shown if the device is timely serviced and maintained, which motivates the user to establish or strengthen the desired behavior.

So far we performed two studies to validate the BioIoT concept design. Both studies had a strong focus on perceptual aspects of decoding information encoded using our nature inspired representations. Our representations were understood by users, even while performing an activity, e.g. coffee machine maintenance. The BioIoT concept can be scaled to a larger number of devices (forty nine devices), and still users are able to detect changes in the status of these devices.

In the future, we intend to conduct long-term studies to investigate the impact of the BioIoT concept in the users’ behaviors and attitudes towards the IoT devices.