A User-Centric Design Framework for Smart Built Environments

Abstract

Smart Built Environments (SBEs) empowered by the Internet of Things (IoT) dramatically augment the capabilities of traditional built environments by imbuing everyday objects with computational and communication capabilities. SBEs primarily consist of three types of components: architectural elements, embedded technology (smart objects) and enhanced interaction modalities. As smart objects hold the ability to change the state of the environment, inefficient design of smart configurations can lead to potentially harmful conditions affecting the safety and security of the inhabitants. The interaction scenarios and space use pattern of SBEs are also notably different from traditional built environments. But, to the best of our knowledge, there has been limited work on developing a consolidated design framework addressing the three interdependent SBE elements and evaluating the safety and security of the IoT application environment. We propose an SBE design framework based on the traditional architectural design process. The framework combines the technological aspects of SBEs with the traditional architectural design process while leveraging Building Information Modeling (BIM) and participatory design. We describe a Mixed Reality(MR)-based reference framework implementation that is particularly helpful for representing, visualizing and modeling the vast amount of data, digital components and novel SBE interaction scenarios.

1 Introduction

Imagine a built space that is empathetic to your needs; a physical environment that goes beyond obvious user-interaction and has the ability to derive your cognitive state and activity pattern and respond accordingly. The idea of such smart environments is not new, but the emergence of IoT has dramatically broadened the scope of SBEs and given rise to revolutionary ideas like a smart, connected world. This emerging idea of the SBE is set to be the future of all built environments and it needs a trans-disciplinary design approach as it encompasses fields like computer science, electrical engineering, architecture, industrial design etc.

IoT-based SBEs include fundamentally different and enhanced capabilities compared to the traditional built environments. Traditional built environments consist of basic building elements and plain physical objects offering primitive interactions, basic use cases and direct affordances. SBEs on the other hand, consist of three major components– basic building elements, embedded technologies and enhanced interaction modalities. The computational and communication capabilities embedded with everyday physical objects enable augmented affordances and multi-modal interactions, thus affecting users’ spatial usage pattern and interaction scenarios [17]. As a result the spatial design is dependent on smart functionality.

In-spite of these inherent differences, SBE designers still follow the traditional architectural design processes and implement the architectural design, technology design and interaction design as three separate processes. The segregation of processes result in the three components being merely layered on top of each other rather than being completely merged towards a unified goal. Consequently, the potential of an SBE to enhance its users’ overall spatial experience or positively impact their spatial use pattern is not fully utilized.

Therefore, it is imperative for SBE designers to adopt a trans-disciplinary approach and adopt a unified design framework that considers the inter-dependency of the three key elements from the very beginning of the SBE design process. But to the best of our knowledge, there has not been much work on developing such a comprehensive framework for SBE design, previous efforts being focused mostly on addressing technological issues associated with SBE.

We describe a unified SBE design framework by integrating embedded technology perspectives and user-centered interaction design principles with the traditional architectural design process. The proposed framework leverages Building Information Modeling (BIM) and enables participatory design by having users as active participants in the design process. It also helps ensure a safe, secure and user-centric design approach for SBEs.

We also describe a mixed reality (MR)-based reference implementation of the proposed framework. The use of MR is particularly helpful for representing and modeling the vast amount of data, digital components and novel interaction scenarios associated with SBEs.

2 Related Work

Architecture is increasingly becoming a major concern for smart environment design and interaction with SBE because when embedded interactive technologies work as architectural elements, it influences the activity flow of the occupants and functional layout of the built environment. Wiberg et al. [30] for example, describe a restaurant in Umea, Sweden, where the inclusion of an RFID-device-based ordering service changed the layout of the restaurant to become significantly different from those of traditional restaurants. Hence architectural design process needs to be an integral part of SBE design. In this section, we first discuss the traditional architectural design process.

Fig. 1.

Traditional architectural design process [6].

The traditional built environments (TBE) consist of basic building blocks (i.e., wall, column, floor, window) and plain physical objects (i.e., furniture, fixture) [6]. For TBEs, the defining elements are– places (points of activities with a sense of boundary), paths (space characterized by a tendency towards mobility), domains (well defined areas consisting of places and paths), thresholds (functional and physical boundaries between spaces) and objects (elements that define a space) [20, 23, 25, 28]. So, the TBE design process primarily focuses on the design issues of the physical environment and we briefly describe a comprehensive outline of the TBE design process based on the related literature [5, 9, 10, 18, 22].

As illustrated in Fig. 1, the first step consists of problem definition and program analysis for understanding the functional requirements, usage pattern, budget, client’s perspective, etc. Step 2 is information collection to understand the topography, climate, regulations, etc. Step 3 consists of concept development and schematic design analyzing client’s lifestyle, spatial use pattern and cultural preferences. Step 4 is mass design, structural design, etc. by drawing a flow diagram and finally developing an architectural design. Step 5 consists of presenting the proposed design to the stakeholders using models, 2D drawings, rendered images, etc. Step 6 is detail design and construction documents. Finally, the construction phase begins.

As for SBE, the related research and current practices do not yet offer a comprehensive framework for SBE design. We briefly discuss the related works that addresses different SBE design issues and proposes novel approaches for addressing those issues. Zhang et al. [31] integrate BIM with the smart functionality design and management of SBEs by including smart object profiling and information exchange data. They also implement a framework to identify possible defects in the design. Lertlakkhanakul et al. [19] worked on a data model for a building and a virtual platform simulating smart home services. The integration of context-aware data model, digital representation of place and user and web services enable a user-oriented approach to visualize invisible services and configuration of smart capabilities. Inada et al. [15] emphasize on the ECA (event, condition, action) rules for designing sensor-driven services where possible conflicts need to be addressed. Guinard et al. [14] describe the importance of a detailed description of the building environment for designing an efficient indoor wireless sensor network. Jeng et al. [17] propose a 3D smart space design concept where space (furniture, fixture, etc.) is viewed as one dimension. The other two dimensions are ubiquitous computing technology and living (safety, security, etc.). Different devices influencing a common set of environmental factors need to be considered based on their spatial context as architectural features effect the efficiency of a built form [31].

In a framework for human computer interaction (HCI) and sustainable home technology, Makonin et al. [21] proposed an ecosystem consisting of the occupants, components of the home, context and dependencies between these. They mentioned the case study of North House to show how to achieve net-zero performance using customized energy systems, smart facade and automated optimization. But the house was not successful in balancing occupancy comfort with optimal energy efficiency, ultimately leading the house occupants to disable the system in favor of their own personalized settings for the house. Rowland et al. [26] emphasize on understanding the primary users, stakeholders and the consequences of networked technologies for designers of connected products. Dourish et al. [11] discuss studying tangible interfaces and interactive behavior together with ubiquitous computing. Weiser’s idea of UbiComp [29] proposes that the most successful technologies are those that are invisible and yet pervasive. This theory exploits our natural skills and activities and tries to make technology blend into our environment. SBE designers would benefit from keeping this in mind while designing smart environments.

We also explore novel assistive technology for supporting the design process. Lertlakkhanakul et al. [19] note there is limited research into introducing virtual reality or web services to the SBE design process to simulate complex, invisible smart services to end users or even designers. They introduce a web-based virtual platform to engage end users in the design process by allowing them to configure smart services.

There has been previous research into the use of immersive technologies for traditional architectural design as they enable visualization and exploration of the designed space before it is constructed [4, 32]. It also has the potential to aid in surveying a model of the site, topography, etc. without having to be there physically [2]. An omni-directional treadmill allows users to move in the virtual environment. Campbell et al. [5] studied and compared designs of a built form designed with virtual reality (VR) and more traditional methods and reported the advantages and shortcomings of VR systems.

MR technologies can overcome some shortcomings of VR. MR devices allow for the projection of the designed space onto the real world in real scale and allow for 3-dimensional interaction with them [2]. MR based social interactions testbed can be used to study users’ situated interaction in an SBE [8]. Virtual twins of the smart objects can also be used to interact with the physical objects in an SBE [13].

3 Problem Definition

In an SBE, the use pattern of a physical space maps to the underlying computing infrastructure. So the pattern of situated interaction is inherently different from a traditional architectural space. But, to the best of our knowledge, there is no existing defined framework for SBE design. As a result, the traditional architectural design approach is still being used for the physical design of an SBE. The technological aspects of the SBE are designed as an entirely separate process.

Additionally, the literature review shows that the related works on SBE design mostly address only the technological aspects of SBE design. As a result SBEs often do not cater to a smart living pattern and human-human/human-object relationship aspects [16, 17].

So, the research problems addressed in this paper are as follows:

• Develop a framework for assisting the SBE design process by modifying the traditional architectural design process leveraging BIM and user-centred design approach. We use a trans-disciplinary approach including architectural design, smart functionality design and interaction design.

• Describe an immersive technology based reference implementation of the framework. We emphasize on overcoming the shortcomings of traditional 2D representations for represent the enhanced capabilities and abstract components of SBEs.

4 Proposed Approach

We developed a unified, trans-disciplinary framework addressing the interdependency of architectural elements, embedded technology and interaction-modalities by modifying the traditional architectural design process. Our aim is to assist in the holistic SBE design process by combining architectural design and smart functionality design within the same process.

Smart functionality design means the design of the smart capabilities of the space. A smart space is able to perceive user’s presence and activity and change the configuration of the environment accordingly. For example, we consider a flexible smart living room in home office context that enables maximum use of space in a small house. The living room transforms into a home office by day and to an expanded living room by evening using an automated movable wall in the design.

The design process of such flexible, smart space needs to consider architectural aspects as well as the design of automatic configuration. Our framework provides a step by step process describing necessary aspects ranging from user requirement collection to an overview of system architecture for an SBE.

An SBE has the following three dimensions [17]:

• 1. Built Environment: Physical components like wall, floor, furniture, appliances, etc.

• 2. Embedded Technology: Sensing technology, networking technology, display, etc.

• 3. Living Requirements: Safety, efficiency, etc.

Our described framework (Fig. 2) modifies the traditional architectural design process (Fig. 1) for meeting the needs of SBE design. This framework consists of four phases- schematic design phase, design development phase, presentation/evaluation phase and construction phase. Each phase consists of multiple steps including detailed guidelines described below:

Fig. 2.

SBE design framework.

Fig. 3.

Left: Phase 1—Schematic design. Right: Phase 2—Design development.

PHASE 1—Schematic Design (Fig.  3 Left):

Step 1—Program Analysis: Understanding the client’s requirements for the SBE along with the usage pattern, context, ecological factors, socio-cultural factors, etc. Ecological factors are the components of the operational network of an SBE (e.g., third party application developers).

Step 2—Site Analysis, Contextual Information: Operational context is important because SBE designs for urban areas, rural areas or the wilderness would need to address different sets of constraints (e.g., unhindered Internet access and power supply). As the spatial arrangement and architectural design support the functionality of the device [26], standard architectural dimensions need to be considered for installing IoT devices.

Step 3—HCI Models: Users’ time-based routines, user-user/user-device relationships and psychological aspects etc. are necessary to understand for avoiding superficial and unnecessary technological intervention. Fully automated smart home devices might make users uncomfortable if they feel like they are always being watched [26]. The SBE designer needs to create a balance between learned automation, programmed automation and fully automatic or user-initiated actions, based on user’s preference, e.g., North house did not balance optimal energy efficiency with occupancy comfort resulting in the users option to disable automation [21]. Hence, we introduce HCI models [26] within the SBE design framework. These models include—(1) Elicitation activities (e.g., personas, activity time-line etc.). (2) Field visits (e.g., observing situated interaction) and (3) Generative Methods (e.g., co-design workshops).

Step 4—Concept Development and Schematic Design: The architectural design of an SBE needs to accommodate smart functionality meaningfully. SBE components that influence the space design are smart objects (walls, floors, etc.), smart devices and furniture/fixtures enabled with smart functionality (smart fridge, smart meter, etc.). Schematic design of the space needs to follow the mapping of situated interaction with these smart objects to ensure adaptability [4].

PHASE 2—Design Development: Designing the components of an SBE based on the context study and client’s requirements (Fig. 3 Right):

Step 1—Architectural Design: The architectural layout needs to be responsive to the novel interaction and activity diagram of the smart space. For example, flexible space or movable wall scenarios accommodated by architectural design. In SBEs, architectural components work as interaction modalities and smart objects function as architectural components. Hence, planning and layout of sensors and actuators need to be integrated into the architectural design process of SBE.

Fig. 4.

Left: BIM model for SBE design. Right: Interaction modalities.

Step 2—Leveraging BIM For Integration of technology with physical environments: Traditional BIM consists of information about the physical infrastructure. We propose that the BIM model (Fig. 4 Left) needs to include smart device data and inter-dependency of smart functionality, context, user and devices. Also the constraining factors of sensors to address issues that affect performance of a smart object. For example, placing a temperature sensor too close to a furnace/cooler causes interference with its functionality. Also, trigger/affect information needs to be included for generating warnings to detect conflicting services. For example, occupancy sensor instructing curtain to open and turn on light simultaneously. The designer needs to program specific instructions to handle these scenarios. We promote participatory design by introducing users’ preference data in the BIM by including their preferences on event–condition–action. For example, each resident of the house gives input on their preferred automated setting of light, temperature, etc. for different tasks.

Step 3—Interaction and Interface Design: The boundary between physical and digital space is slowly disappearing with dynamic interfaces being integrated into everyday objects. As the nature of HCI dramatically changes, a major challenge in SBE design is developing an well defined interaction model which does not pose cognitive burden to the user or confuse them. A clear boundary needs to be defined for starting and end points and multi modal interaction needs to be supported by the model. A natural mapping between action and perception helps users understand and feel comfortable in a space.

There are two types of interaction modalities—direct interaction (switches, input devices, etc.) and indirect interaction (gesture, voice command, automation etc.). We suggest a balanced, hybrid user-side (direct) and SBE side (indirect) interaction (Fig. 4 Right). For example, the lighting and HVAC system can be semi-automated with options for the users to override the system using voice command or manual switch.

Step 4—Acquiring Data from the Environment: Smart objects gather data about the state of the objects and respond to changing conditions and user-interaction. Typically three types of sensors are used in SBEs:

• 1. Location sensors: Detect human presence using web cameras, optical and magnetic sensors, etc.

• 2. Mobile sensors: Detect gestures, motion, etc.

• 3. Environmental sensors: Measure humidity, temperature, etc.

The major challenge in system design lies in successfully combining the heterogeneous sensors and actuators with a software platform to develop a responsive environment and smooth user experience. The steps in our proposed system are collecting the sensor data, integrating them in a central system and programming the cause and effect (Fig. 5). The proposed framework uses a singular protocol for sending data from smart objects and dynamically creating digital representations for them. The framework supports both cloud and local installation enabling the system to be completely autonomous and independent.

Fig. 5.

Acquiring data from SBE.

Step 5—System Architecture: An integrated platform controls the whole system making the SBE responsive to a changing environment. Sensors and actuators send data to a server, an application accesses the data and determines the role and behavior of smart devices. Data analysis tools help in improving the building performance (Fig. 5).

There are three layers in the system architecture:

• 1. Spatial system: Spatial planning of the environment.

• 2. Sensor networks: Collecting environmental parameters like temperature, humidity, etc.

• 3. Services and Application layer: Using collected data for controlling and monitoring building’s conditions.

There are some other necessary aspects that need to be considered in the SBE design process. They are stated below:

Energy Efficiency: Additional objectives of an SBE includes improving comfort, operational cost reduction, energy consumption reduction, etc. [24]. So, efficient use of building systems, improving life cycle of building utilities, etc. are necessary criteria for SBE design.

Laws and Regulations: It is very important to know about the relevant laws and regulations before embarking on the design process because SBEs collect a lot of personal data from the users [26]. As it is a comparatively new field the regulations are still not very concrete.

Privacy and Security Aspect: The unique characteristics of SBE enabled by IoT, i.e., use of distributed control, heterogeneous attack surfaces and scale make it hard to provide security and privacy. Eavesdropping is easier as majority of the communication is Wireless. IoT devices have low computing capability and limited energy resource, so complex schemes cannot be implemented for enabling security [3]. End devices belong to various organizations making the management of passwords a challenging task. So there needs to be unified human centered approach for solving this issue. A major concern regarding privacy is the uneasiness among users at being constantly watched or listened to by smart devices. The increasingly pervasive collection of data is a serious privacy concern as it gives away a virtual biography revealing behavioral and lifestyle patterns.

User Safety: In an SBE, it is common to have multiple actuated devices that are capable of acting independently, without user supervision. In such a space with several independently-acting smart objects there is a possibility that the interaction of these smart objects might produce safety hazards for the SBE inhabitants. For example a collision between the SBE user and a moving wall could take place while changing the spatial configuration of the SBE. To prevent hazards like this, the SBE needs to have a system in place that is capable of supporting real-time hazard detection [12]. Such a system would have to constantly monitor the state of the SBE and its inhabitants in order to warn the inhabitants and to take mitigative action against these safety hazards.

PHASE 3—Presentation and Evaluation: The steps in this phase are discussed below:

Step 1—Presenting the Ideas for Feedback and Evaluation: Computer drafting, drawing and 3D models are predominantly used for development and presentation of architectural ideas [4]. But these tools have limitations in case of SBE design. Novel immersive simulation techniques can assist in evaluating the enhanced SBE capabilities and also as input-output modality. Re-configurable spaces, automated configurations etc. can be simulated to understand capability and spatial impact.

An immersive platform has potentials for remote and in-situ collaboration with other consultants. Figure 6 shows an immersive walk through a home interior. Incorporating editing capabilities within the immersive platform allows the designer to make necessary changes and test different iterations of the design at different scales [1, 27]. This technology can in fact reinvent the architectural/SBE design process [2].

Step 2—Improve the Design and go back to Phase 1: Based on the feedback from stakeholders, the designer needs to return to the first step and reconsider design decisions for a proper balance between events and actions.

PHASE 4—Construction Phase:

Step 1—Detail development and construction documents: Construction documents include detail working drawings and specifications for guiding construction.

Fig. 6.

Visualizing the designed space on top of user’s physical environment.

Step 2—Bidding and Construction: After selecting a contractor the designer oversees the construction.

5 MR-Based Reference Implementation of the Framework

We describe a MR-based implementation of the SBE design framework in a smart home context. Since SBE prototype building is expensive and challenging, using immersive MR technology helps simulating the abstract affordances of a smart space to benefit both the user and designer. It also makes the user’s participation in the design process easier. Figure 7 highlights the steps that incorporate immersive technology (I.T.) in the SBE design framework.

This implementation assists in the SBE design process by leveraging the MR platform for incorporating the immersive simulation with the user’s physical surrounding. The user is able to test the smart functionalities and make design choices for preferred configuration of smart environment for different activities.

SBE design process also requires selection of interaction modalities like gesture, voice command, etc. based on user preference. The MR based implementation allows the user to test interaction scenarios like voice commands and hand gestures to control the state of a physical smart object by manipulating its virtual counterpart and decide upon a preferred interaction method. Moreover, the SBE designer can use the MR based implementation for visualizing different architectural and smart functionality design options.

Fig. 7.

Steps involved in the MR based reference implementation of the SBE design framework.

The services provided by the implementation include immersive visualization, object manipulation at different scales, multimodal interactions (gesture, gaze, voice command), visual representation of interconnected smart functionality, conflict detection based on the data model, digital representation of users, audio/visual cues, collaboration capabilities, situated interaction, view manipulation, navigation and physics modeling/simulation. Object manipulation also includes controlling physical objects by manipulating their virtual counterparts.

Phase 1 Implementation: A 3D digital model created based on a preliminary conceptual architectural design provides the context for the MR application interface. The first phase requires the designer to collect user’s preferences on smart functionality. The “User Configuration Mode” allows the designer to create HCI models (e.g., persona) by directly taking inputs from the users about automated smart-functionality. Figure 8 (Left) shows an example scenario where users explore and choose configurations of smart environment for different activities in an immersive simulation, e.g., recording preferred illumination and volume for activities like watching movies, reading etc. in a game-like application setup. This mode also allows the users to test voice-commands and hand-gesture for controlling physical objects using their virtual counterparts. This functionality helps the user to decide upon a preferred interaction modality. The designer later uses these requirements to program the automation and interaction of SBE.

Phase 2 Implementation: The second phase assists in designing smart functionality and physical environment. A BIM-inspired data model is developed from information collected using “User Configuration Mode”. The data model includes the context, user, smart objects, constraints and the interaction information.

The MR application consists of a “Architectural Design Mode” which allows manipulation of the virtual models and exploration of different layouts of the physical design of the house. Designer is able to see the object from different perspectives and in different scales. Modification capabilities like copy, transform, scale, etc. are provided. A small scale holographic representation of the house appears in front of the user along with a library of modules floating over the base model (Fig. 9 Right). Spatial mapping is used for placing the virtual models on top of a physical surface selected by the user. The user can drag and drop the modules to try out different possible layouts (Fig. 10). Testing voice commands to control lighting is also allowed in this mode.

The “Designer Configuration Mode” allows the designer to explore different possible combinations of smart devices in the immersive visualization. The data model is used to validate and notify designers if the collection of smart devices is safe, secure and functional.

Fig. 8.

Left: User Configuration Mode—Customize configuration of ambient environment to accommodate user’s personal preference. Use of data model for configuration of SBE. Right: Designer Configuration Mode—Explore combinations of smart objects and resolve conflicts.

Fig. 9.

Left: Preview mode: visualizing design in outdoors. Right: Architectural design mode: visualizing design and module library in MR environment.

It is used to identify conflicts if some event triggers multiple contradictory actions. If any combination of smart functionality violates the safety and security of the environment because of overlapping/conflicting trigger-affect, a warning message is generated for the designer to resolve the conflict by introducing appropriate conditions while programming.

Figure 8 right shows an example scenario where a designer receives warning messages if she attempts to combine conflicting services. Here the smart curtain and the smart bulb are conflicting as both affect lighting.

Another example is that, functionality of a sprinkler system can be hampered by a water leak detection system in case the latter turns off water supply after detecting water pouring into a room with potentially disastrous consequences.

Fig. 10.

Architectural design mode: use of MR in the SBE design process.

Phase 3 Implementation: Phase 3 (presentation and evaluation phase) consists of testing the proposed interaction modalities for controlling devices and the overall SBE functionality using the “Preview Mode”. In “Preview Mode”, the designer/client is able to navigate through the interior of the proposed building in real scale in immersive visualization for understanding the spatial quality. They are also able to visualize the design on actual site (Fig. 9 Left). This helps in smart facade design based on sun path and wind flow to utilize natural light and wind for making it energy efficient.

Phase 4 Implementation: Phase 4 leverages the “Preview Mode” for immersive visualization for assistance in construction management.

MR Application: System Architecture

Figure 11 shows an example of the MR application architecture using light control as the use case.

Fig. 11.

MR application architecture.

The work-flow for developing a similar MR application consists of three steps:

STEP 1—Preparing the Virtual Components:

The built environment is designed and modeled using digital drafting tools (AutoCAD) and 3D-modeling tools (SketchUp). The models need to be generated in the origin so that transformation in game engine is easier.

STEP 2—Designing the behavior of application:

Application Design: Capabilities like manipulating basic shapes (e.g., change dimensions, add, move), changing color, texture, etc are useful design tools. The virtual model includes BIM information, energy performance, smart functionality and interdependency information with other smart objects. After designing the behavior and capabilities of the application, hand gesture, gaze and voice command are used as interaction modalities. The application provides audio/visual cues in the UI to assist in the design process, e.g., selection menus, shape and color palettes, etc.

System Overview: The interaction model is informed by user data (data from client/designer), smart object data, physical environment data and context data (Fig. 12 Left). The proposed framework creates a virtual twin of SBE using a data model for immersive visualization (Fig. 12 Right). To verify service conflicts, each service registers triggering/affecting factors and the information processor performs reasoning based on space and context.

STEP 3—Developing the application:

Programming the behavior of the application includes developing embodied interaction and testing them using a game engine. For collecting environmental and smart object data, the application needs to connect to the archive and overlay on the virtual model. Building and testing the application for immersive technology platform (VR, MR) requires use of game engines (e.g., Unity, Unreal Engine).

Fig. 12.

Left: System overview of MR based SBE design process. Right: Integration of data model with immersive technology.

6 Use Cases and Evaluation

This section discusses the reference implementation of the proposed approaches. An SBE was designed based on the approach described in [7]. The computational and physical infrastructure were perceived as interdependent from the beginning of the design process.

Schematic Design: First, the program analysis step is followed to identify the aim, which is to develop a responsive and energy efficient home with concepts of aging in place and flexible space. Then information collection and HCI modeling is followed to get an overall idea about the conceptual clients, site, context, user’s preferences. In a traditional architecture process each basic activity needs a dedicated physical space for supporting functionality. Here we conceptualize an automated, transformable space with multi-functional use. Based on the user’s activity, the room would change configuration. For example, the occupants can turn the bedroom into a home office using voice command. Dining space can be turned into family living and formal living into Home Theater.

Design Development: Two major design proposals were developed according to the schemes from phase 1.

• 1. Modular, off-site construction using integrated technology: The proposed design consists of a prefabricated modular design approach combined with site-built components (Fig. 9 Right). The core functional spaces of a house, like kitchen, bathroom, etc., are designed as modules and constructed remotely in a factory. They are constructed fully equipped with the embedded technology like sensors, actuators. Then they are brought to site and anchored to the site built foundation system.

  The proposed system architecture focuses on connecting smart objects, collecting usage data, storage and exploring usage pattern (Fig. 5). Interaction modalities include hand gesture, touch screen, switches and immersive technology based interaction. Touch screen displays are embedded with walls, tables, etc. physical components of the house.

• 2. Data model: The proposed data model consists of semantic information depicting spatial relationships among users, objects and context in addition to geometry information (Fig. 13). User’s contextual preferences for different events, smart device information and physical information are modeled here.

Fig. 13.

Proposed user centered data model.

7 Conclusion and Future Work

In SBEs, ‘things’ can autonomously interact with each other and change the state of the physical world. SBEs provide a transformed task-space mapping with changed use-pattern. So during the design of SBE, simply incorporating smart devices into a space without focusing on the holistic design process leads to a rigid and reduced spatial and experiential quality. But current SBE design practices focus primarily on computational capabilities with less importance on holistic approach, resulting in a rigid and sometimes impractical setting. We describe a user centered SBE design approach modified from the traditional architectural process to achieve the desired spatial quality. The proposed framework addresses the issues of situated interaction and underlying technology along with the physical environment design process. A novel data model emphasizing human-space interaction is introduced in the layered framework focusing on increasing user participation in SBE design.

We also describe an immersive technology based reference implementation of the proposed framework using participatory design principles. This implementation allows visualization of invisible services and real-time configuration based on individual preferences before deploying them. Smart home clients can experience the immersive visualization and interact with the designed environment using a game-like application. Data input by actual users is integrated in the data-model assigning an active role to users in the design process. The framework helps testing novel interaction scenarios and complex affordances whereas testing situated interaction is challenging and expensive by following traditional prototyping methods. Moreover, immersive visualization of data generated by smart objects enables more efficient usage of the data in the design process.

Overall, the contributions of our work are as follows:

• A framework for assisting in SBE design process developed by modifying the traditional architectural design process and incorporating user-centred BIM.

• An immersive technology based reference implementation of the framework.

We are working on employing the framework as an education tool for next generation SBE designers. An ongoing user study tests the usability of the application for both the designers and users of SBEs. The study is conducted on two groups—architecture students and people from general population. Architecture students are considered as subject matter experts within the context of SBE design. General people are considered as users of SBE. Participants complete three tasks using traditional methods and mixed reality based implementation of the framework. The first task for the study participants is to configure the ambient environment setting (e.g., illumination, temperature, volume) in a living room context for watching movies. The second task is to use different interaction modalities (e.g., voice command, hand gesture, automatic) for selecting and controlling a device (e.g, lights). The third and final task is to choose smart objects from a menu and place them within the living room context. The participants are provided a feedback in case of a conflict. Finally, the participant completes a survey about their preferences and opinions for the study.

Overall, ours is a novel and comprehensive approach addressing the major aspects associated with SBE design from an interdisciplinary point of view. We hope that the proposed framework would help reduce design failures in SBEs during occupancy period.