Keywords

1 Introduction

The average American spends 87% of her or his time in buildings and 6% in vehicles, amounting to 93% of their life inside [1]. In the next decade, rapid technological disruption is expected to bring new levels of intelligence to both automobiles and buildings, which will have a transformative impact on our interactions with these technologically sophisticated environments. There will be thousands of autonomous vehicles deployed in the U.S. by 2020 and it is forecasted that 21 million autonomous vehicles will be sold in the U.S. in the next 20 years [2]. The smart building market is expected to grow from $6B in 2016 to $25B by 2021, affecting lighting, HVAC, communication, and security systems [3]. The rapid deployment of smart technologies in buildings and automobiles means that, in less than 20 years, Americans could be spending over 90% of their lives interfacing with intelligent environments.

The incentive for this rapid adoption of smart technologies is two-fold: First, it is the urgent need to improve the systems-level efficiency of our buildings and automobiles in the face of a resource-constrained world. Second, smarter environments present opportunities to meaningfully improve the experience of the occupants in automobiles (safety) and buildings (environmental quality). In this proceedings, we discuss the potential systems- and occupant-level benefits of integrating wearable thermal technology, connected wearable accessories that deliver precise sensations of heating or cooling, with smart buildings and autonomous electric vehicles. In Sect. 2, a wearable thermoelectric system, EMBR WaveTM, is presented that has been designed specifically for integrating personalized thermal sensations into the internet of things. In Sect. 3, we utilize EMBR WaveTM to demonstrate the ability of wearable thermal technology to improve the situational awareness of passengers in autonomous vehicles. Finally, in Sect. 4, we will discuss the value that can be unlocked by integrating wearable thermal technology with Smart HVAC systems. Together, these examples highlight the new opportunities for human-computer interactions enabled by the convergence of intelligent environments and wearable technology. By introducing thermosensation into the internet of things, new channels of interaction are created that can improve the operation of the complex systems while simultaneously enhancing the experience of the occupants.

2 Wearable Thermal Technology

Wearable thermal technology, the concept of wearable technology that is designed to deliver precise, localized thermal sensations, is now feasible and attractive as a wearable human computer interface due to the significant technological and scientific progress of the last 20 years. Wearable thermal technology has become technologically feasible owing to the same advances currently enabling the Internet of things: Energy-efficient wireless communication, the miniaturization of computational systems, and the increasing energy and power density of batteries. In parallel with these technological advancements, scientific advancements have elucidated astounding relationships between the human body and the experience of thermal sensations that suggest thermal sensations can be used to convey information [4], relieve personal discomfort [5] and even influence the experience of emotions [6]. In this section, we present EMBR WaveTM, a wearable platform technology for integrating personalized thermosensation with smart environments and the Internet of things.

2.1 EMBR WaveTM: A Smart Platform for Wearable Thermal Technology

EMBR WaveTM is a platform technology developed at EMBR Labs for introducing precise, localized, and personalized thermal sensations into connected and wearable accessories. EMBR WaveTM utilizes a thermoelectric module to generate precise and dynamic temperature profiles directly on the wearer’s skin. Thermoelectric modules are conventionally considered inefficient and high-power devices, but through iterative prototyping (Fig. 1) we have demonstrated a custom thermoelectric element with size and power specifications that enable wearable form factors with battery included. In previous attempts at wearable heating and cooling, power consumption and thermal dissipation have been major obstacles to designing devices that are sufficiently small and light for everyday use. EMBR WaveTM can be powered from a lithium battery, comparable to what is currently used in smartphones and smart watches (250–1000 mAh), and uses under 2 W of power. Furthermore, the heat generated can be dissipated using natural convection and a passive aluminum heat sink.

Fig. 1.
figure 1

(a) Original EMBR WaveTM demonstration at MADMEC competition in 2013 (Cambridge, MA). (b–e) Examples of wrist-worn prototypes developed 2014 - 2016. (f) Current wrist-worn adaptation of EMBR WaveTM, called Wristify, being commercialized in 2017.

Full size image

EMBR Labs has developed patent-pending architectures for this comprehensive system that are robust and enable all components to be packaged into wearable form factors appropriate for everyday use. The thermoelectric heat pump is packaged with multiple temperature sensors that provide a resolution around 0.1 °C and the thermal system can controllably create rapid temperature profiles at the skin in the range of 0.1–1 °C/s. Finally, EMBR WaveTM heating and cooling modules are equipped with wireless communications and onboard computing to enable sophisticated systems-level integration with Smart Environments.

2.2 The Power of Thermal Sensations

When you feel something warm or cold, a lot more is going on than just a thermal sensation. When we wrap our hands around a warm mug of tea, or a cool breeze blows across our face, the temperature changes are detected by two different kinds of receptors in the skin, known as cold and warm thermoreceptors. These thermoreceptors send signals to the brain, which translates the thermal stimuli into thermal sensations through the same neural networks that are also responsible for touch, pleasure, thermoregulation, emotion, and the balance of the autonomic nervous system [7].

Through this complex network of regions in the brain, localized thermal sensations have the unique potential to influence the human experience through both exteroceptive and interoceptive pathways. Like conventional haptic devices, thermal sensations can be used to convey information [4], albeit with limited information density and temporal resolution compared to audio or visual feedback. Thermal sensations, however, can offer occupants much more meaningful interactions than just notifications. For example, experiencing localized thermal sensations can influence the experience of emotions [6, 8, 9] and personal comfort (See Sect. 4) [5]. Integrating wearable thermal technology with smart environments unlocks new potential value through these relationships between temperature and human psychophysiology. In the following sections we present two ongoing collaborations in which, utilizing EMBR WaveTM, wearable thermal technology is being used to improve the functionality of smart environments and the experience of the occupants or passengers.

3 Improving Situational Awareness in Autonomous Vehicles with Wearable Thermal Technology

Ninety percent of automobile collisions are due to human error with 20–40% of those collisions resulting from driver distraction [10]. The adoption of autonomous vehicles is expected to significantly reduce automobile crashes by removing or reducing the potential influence of human error [11]. Autonomous vehicles present new challenges, however, because as vehicles become increasingly autonomous the passengers may pay less direct attention to their surroundings. An important component of interfacing within an autonomous vehicle is the ability of the vehicle to capture the driver’s attention and provide her or him with the necessary information to rapidly and accurately respond to unexpected events.

Multiple resource theory states that communicating information over multiple channels can help individuals better perform multiple tasks [12]. As the number of sensory features given to a piece of data increases, the amount of information a person can receive also increases [13]. Multimodal information presentation has demonstrated clear benefits, such as faster reaction time combining audio and visual alerts [14]. In this study we investigate the potential value of conveying situational information to passengers in autonomous vehicles through multimodal sensory inputs, including thermal sensation.

The Immersive Situation Awareness system (isaWear), developed at Draper (Cambridge, MA), includes visual, auditory, thermal, and haptic feedback. (Figure 2) Thermal feedback is provided on the inside surface of each wrist by two EMBR WaveTM wristbands, allowing for directional information to be intuitively conveyed by thermal cues. Our hypothesis is that imbuing situational information with features across multiple human senses will increase human capacity for information perception, which translates to faster and more accurate responses by the driver when presented with unexpected events that require attention and rapid decision making. Two experiments will be presented that evaluate the use of multisensory information to convey situational awareness and aid human response and decision-making.

Fig. 2.
figure 2

The Immersive Situation Awareness system (isaWear), including augmented reality in a heads up display, 3-dimensional audio, and EMBR WaveTM wearable thermal technology on each wrist.

Full size image

The first experiment was an object identification test. Participants were placed in the middle of eight large screens with a forest scene projected on all sides (Fig. 3). Users received signals through auditory, thermal, and a combination of auditory and thermal channels that were counterbalanced to account for learning and ordering effects. All signals had dangerous and benign signals associated, and were presented in all conditions. Their task was to turn to the source screen and identify whether the source was dangerous or benign. We collected response time and accuracy. After experimenting with a 3 × 2 experiment design over 24 participants, the single presentations of information performed worse than the combined signals, both in reaction times and in accuracy. Figure 3 presents the experiment set-up (a) and the accuracy results (b). Overall, by pairing visual and thermal signals, there was an order of magnitude reduction in error in identifying dangerous or benign signals.

Fig. 3.
figure 3

(a) The experimental setup of a hand-held response, augmented reality glasses, and the EMBR Wave™ heating and cooling wristbands. (b) The chart demonstrates the percentage of incorrect responses for each condition. The paired-presentation experiment compared error rates using a presentation device using single channels (either thermal or visual) to a paired presentation of visual and thermal presentation. The paired presentation resulted in a 10× error reduction.

Full size image

The second experiment was a driving simulator in which participants wearing isaWear had to react to obstacles encountered en route to arriving at a goal location (Fig. 4(a)). The participants were instructed to drive through a maze with various objectives, including avoiding animals, dealing with vehicle overheating, and getting to the goal within a certain time. Participants received visual signals through augmented reality goggles simulating a heads up display, auditory signals through earphones that placed sounds in a three-dimensional location, and directional thermal signals from EMBR WaveTM wristband worn on each wrist. Signals were presented alone, in pairs, and in a combination of three to understand any combinatory effects of the signals. All users saw all instances of the obstacles and all of the designs of the signals. The ordering of these conditions was counterbalanced between participants to account for learning or effects of overwhelming the user. Researchers measured performance of the participants by time to react to the signal. Figure 4(b) shows the average reaction time of participants in response to combinations of sensory signals. The experiment demonstrates that combinations of signals across modalities can improve reaction times in high-stress environments. This trend is consistent with Multiple Resource Theory and supports the feasibility of leveraging wearable thermal technology as a complementary interface between passengers and autonomous vehicles.

Fig. 4.
figure 4

(a) Driving Simulator with isaWear kit and 5-panel 180° display. (b) Reaction time to different categories of obstacles presented in multimodal driving simulation. Comparing control (no isaWear, visual and dashboard-only signals) to different combinations of sensory inputs.

Full size image

The aforementioned investigations utilizing isaWear and EMBR WaveTM wearable thermal technology highlight the potential value that multisensory feedback systems could have in ensuring the situational awareness of the passengers of autonomous vehicles. Directional thermal cues, delivered by wearable thermal technology, have several practical advantages in the context of interacting with autonomous vehicles. First, wearable thermal technology was found to pare favorably with augmented reality (Fig. 4(b)), which was meant to imitate the type of heads up displays expected to be prevalent in autonomous vehicles. Second, relying on wearable technology to provide sensory feedback ensures that the occupant receives the notifications independent of their hand or body locations. Third, thermal cues can be provided without obfuscating other visual or auditory information that is necessary for situational awareness.

There are limitations to temperature as a haptic medium that make thermal cues better suited to be a complementary sensory input than a primary input. First, the resolution of information that can be conveyed by thermal signals is limited compared to auditory or visual cues. In this study, a thermal system was worn on each wrist and this allowed directional information to be intuitively included in the thermal cues. Even under high-stress environments, participants were able to consistently distinguish warm from cold and assign directional information to the cues. However, when additional information was encoded in the thermal stimulus, such as varying intensity or temporal profiles, subjects exhibited difficulty differentiating between temperature profiles during the immersive and high-stress simulation. Second, the meaning ascribed to thermal sensations is often mixed and context dependent [15]. We found that the interpretation of thermal signals varied depending on many variables: user preconceived notions, expectations relating to the scenario, environmental cues, and implicit associations with heat and cold. Combining thermal cues with other sensory inputs reduces the ambiguity of thermal cues and allows for a broader range of affective tones when providing haptic feedback [16]. Cross-modal matching and the effective pairing of thermal cues with other sensory inputs for clear meaning is an area of ongoing research.

4 Reducing Energy Consumption and Improving Occupant Comfort and Productivity in Smart Buildings with Wearable Thermal Technology

Current approaches to building-scale heating and cooling are causing both environmental and economic damage. Every year over 12 Qbtu of energy, more than 12% of all energy used domestically, is used to heat or cool spaces with the goal of maintaining occupant comfort [17]. Despite enormous energy consumption put into tight environmental control in the built environment, more office occupants are dissatisfied (42%) than satisfied (39%) with their office environment [18]. In a study focused on thermal comfort, 50% of the subjects preferred a change in their thermal state, 38% of subjects in winter were dissatisfied with thermal conditions, and almost 50% of the thermal conditions during summer were outside of the thermal comfort zone [19].

The smart building market is expected to grow from $6B in 2016 to $25B by 2021, [3] and this trend has the potential to significantly reduce the amount of energy used to condition our indoor environments and improve occupant comfort. In Smart HVAC, this is being demonstrated [20,21,22,23] by integrating a combination of sensors that can provide information about the environment and the occupants, the collection of self-reported comfort data from occupants, and developing sophisticated models for aggregating this data and using it to optimize HVAC operation. It is important to recognize, however, that building-scale solutions are incapable of solving the core issue underlying occupant dissatisfaction: Different people have different standards for what environmental conditions are comfortable at any given time. Current ASHRAE guidelines suggest that, even under “ideal” conditions, only 80% of occupants will be comfortable [24]. Building-scale HVAC systems, whether intelligent or not, face fundamental challenges with regard to maintaining comfort across populations of people.

Introducing wearable thermal technology as a human-computer interface in smart buildings has the potential to both improve occupant comfort and further reduce the energy consumption of building-scale heating and cooling. Contrary to mainstream belief, improving personal comfort in moderate environments does not require changing the heat balance equation of the body. Personal comfort, while closely related to thermoregulation, is a distinct psychophysiological concept [25]. The most significant advancement in the field of comfort science in the past 20 years has been the shift from a physical, steady state and deterministic model of comfort to an adaptive comfort model [26]. The adaptive comfort model recognizes that personal comfort is not just a heat balance equation but depends on more complex mental and adaptation processes [27, 28]. Accordingly, the ASHRAE Standard 55 defines thermal comfort as: “that condition of mind that expresses satisfaction with the thermal environment” [24].

Leveraging this full psychophysiological model of thermal comfort, it has been demonstrated that localized, transient thermal sensations can improve personal comfort [5]. In this comfort model, if a local thermal sensation is experienced, so long as the occupant has some control over their thermal experience and the thermal experience is transient, then the overall comfort experienced by the user becomes the average of the 2 most uncomfortable regions and the maximum comfort vote (the transient thermal sensation) [29]. By taking a personalized and transient approach to thermal sensations, there is an opportunity to improve whole-body comfort while using much less energy than is necessary to heat or cool the entire body.

EMBR Labs is developing a wearable personal comfort system that can provide up to 8 h of use (1 work day) and could expand a building occupant’s thermal comfort zone by 1–3 °C. Such personalized comfort performance would have a transformative impact on both building-scale energy consumption and occupant productivity: The energy required to heat or cool a building is reduced by 7-10% for every 1 °C that the occupants’ float zone is extended, [30] suggesting a potential for greater than 20%+ reduction in building energy consumption. Analyses have suggested that personalizing temperature in a ±2 °C range would lead to a building-wide increase of 3% in the performance of both logical thinking and very skilled manual work and a 7% increase in typing performance [31].

Equipping occupants with a connected and wearable personal comfort system creates a new channel for interacting with Smart HVAC systems. The use of a personal comfort system, whether in heating or cooling, is in itself an indicator of personal discomfort that can be immediately communicated to the Smart HVAC system. Tracking occupants’ behavioral response to discomfort will provide a rapid and accurate form of comfort reporting to complement Smart HVAC operation. The smart HVAC system (or facilities manager) will know at any given time how many people are hot or cold and to what degree, giving them high-resolution insight into the most important HVAC performance criteria: occupant comfort. Wearable thermal technology has the potential to provide instantaneous thermal relief, allow occupants to suit their own comfort needs, and provide real-time comfort data to help inform more intelligent building-scale HVAC operation. For these reasons, it has the potential to be the defining experience of the smart building of the future.

5 Summary

The last 20 years of progress in mobile technology and in the science of thermal sensations are converging to enable wearable thermal technology that serves as a tool for the wearer and an interface for interacting with smart environments. We have demonstrated the feasibility of wearable thermal technology through EMBR WaveTM, a wearable thermoelectric platform designed for integrating thermosensation with the Internet of things. In collaboration with Draper, we have validated the capability of EMBR WaveTM for improving situational awareness in autonomous vehicles, in particular when thermosensation is integrated with a multisensory system such as IsaWear. In the built environment, wearable thermal technology has the potential to improve occupant satisfaction and productivity while reducing the energy consumed heating and cooling buildings. EMBR Labs, together with UC Berkeley, has received a Phase I NSF STTR to demonstrate wearable personal comfort systems that can improve occupant comfort and generate real-time comfort data for smart buildings. These examples highlight the new opportunities for human-computer interactions enabled by intelligent environments and connected wearable thermal technology. Introducing thermosensation into the Internet of Things creates new channels of interaction that can improve the operation of the complex systems while simultaneously augmenting the experience of the occupants. Potential collaborators interested in using EMBR WaveTM to introduce thermosensation into their connected systems should contact the corresponding author at EMBR Labs.