1 Introduction

The lifecycle of a building comprises four main stages: Design, Build, Operate and Maintain. The first two stages cover a building from its conception, through to construction until hand over to building owners and facility management teams for use. Operation and Maintenance (O&M) is relevant to all assets from infrastructure to high value products and can incur up to 60% of the total lifecycle costs for a building [1]. Thus, effective management is crucial to reduce costs and extend lifespans.

Lifecycle management within O&M aims to provide a safe working environment, minimise lifecycle costs and optimise for desired building function. To help achieve these aims, Building Information Modelling (BIM) is employed to improve collaboration, information retrieval and decision making across the entire supply-chain and product lifecycle. BIM is the process of generating, integrating and collaborating on digital models and data to support building management to support design, building, maintenance and operation. BIM is defined by four levels that feature the level of detail, data and collaboration involved in the management of a building. For example, level 0 involves 2D drafting and plans and little to no collaboration, while level 3 requires the sharing and collaboration across the prime contractor, customer and entire market place of all 3D models. Level 3+ BIM can enable, for example, advanced analysis such as clash detection and interrogation of construction plans. In the UK, government projects are generally required to conform to Level 2 BIM, which demands the exchange of models and data using standards such as COBie (Construction Operations Building Information Exchange). While large contractors for major projects endeavour to comply with Level 2 BIM, the majority of small projects achieve Level 1 BIM compliance by documenting some 3D CAD work for concepts, 2D drawings for statutory documentation and some form of common data repository for access and re-use. Given that the majority of projects conform at least to Level 1 BIM standards, the focus of this paper is in enhancing the re-use of this data within O&M. This is achieved by visualising the information and models through a low-cost, lightweight platform suitable for small BIM projects.

Emerging Mixed Reality (MR) technologies can provide superior visualisation through more immersive and interactive environments. MR refers to technology that enhances reality by integrating it with the digital world, such as Augmented Reality (AR), Virtual Reality (VR) and spherical panoramas. AR and VR are becoming increasingly commonplace and research shows they could be effectively applied to support Small and Medium enterprise (SME) O&M [2, 3]. This paper introduces the use of spherical panoramas (a.k.a. Photospheres), which are commonly used within Google Street View, as a means to provide the visual navigation and search of BIM information one would expect from a BIM level 2+ compliant project.

Small and Medium enterprises (SMEs) are generally defined as firms with less than 250 employees and turnover of below €50 million [4]. This limit of company size will in turn have an effect on the building it occupies and the resources dedicated to management of the building lifecycle. Hence some visualisation tools suited to larger facilities may be too expensive, require too much expert knowledge and require technical pre-requisites that are not available within small facilities. Whilst research has been performed into the use of mixed reality tools within Facility Management and O&M, there has been no little to no consideration of their suitability for SMEs.

This paper aims to understand the different requirements for visualisation tools between large and small enterprises (facilities), and present recommendations on the most suited MR tools for smaller companies. Finally, a number of recommendations are developed for the selection of an appropriate strategy dependent upon maturity of BIM adoption.

2 Operation and Maintenance

The purpose of this section is to highlight the need for Mixed Reality tools within the management of the Operation and Maintenance stages, commonly known as Facilities Management (FM). FM is an organisational process focused on maintaining and enhancing building infrastructure within O&M. Typical responsibilities of a facility manager within the O&M cycle focus on three key areas: cost, performance and risk. Through employing Building Information Modelling (BIM), FM tasks can be improved as maintenance data is more readily available, and decision and collaboration with internal and external stakeholders is enhanced.

FM is a complex process, encompassing many cross-organisational activities, many stakeholders and large amounts of data. The potential applications of BIM within FM were highlighted through online user surveys and were found to be: locating building components, facilitating real-time data access, visualisation and marketing, checking maintainability, and creating digital assets [5]. Hence, in this paper, MR tools will be explored based on how they can be used within these scenarios.

To service assets and perform corrective maintenance, important building assets must be frequently located and inspected. Through providing asset location in 3D models, the exact location can be easily visualised, helping reduce maintenance time. Subsequently, asset information, along with repair instructions and maintenance schedules, must be accessed to perform maintenance. Providing this information in an easy to comprehend format and in-situ with the maintenance location can help reduce errors and increase time spent performing repairs. Achieving such interoperability is one of the key tenets of BIM solutions that aim to bring all relevant information into a single platform.

The use of BIM within FM is often limited by the availability of digital models from the design and build phase, which are often either non-existent, not accessible or not up-to-date. For example, 80% of European buildings were created pre 1990, before mainstream BIM software to create building models was available [6]. Further, despite the inception of BIM, not all building projects use digital models and many models are either not kept up-to-date or not transferred to facility managers in the handover process. The lack of pre-existing models to underpin BIM is a major barrier to BIM-enabled FM. Therefore, to start utilising BIM within FM, digital models of assets need to first be created in order to provide traceability. Maintainability is the ease with which a product can be repaired to perform its desired function. Within buildings, assets must be easily accessible so that maintenance teams can perform repairs quickly with minimum cost and where possible, designed sustainably to preserve asset life span.

To evaluate how applicable different visualisation tools are, the stakeholder requirements for implementation and use must first be considered. [2] highlighted the main issues for implementation of BIM and mobile AR in FM. Issues relevant to technology were cost, lack of BIM models, lack of BIM-trained personnel and interoperability. Similarly, [7] review of barriers to BIM adoption highlighted the main issues to be: cost of investment, learning curve for BIM technologies, lack of executive buy-in, poor collaboration among participants and poor interoperability among BIM software. In terms of technology specific requirements, [6] presented the requirements for data capturing techniques within BIM as being: applicability in existing buildings, cost, time, spatial accuracy, level of detail, influence of size and complexity of the scene, influence of environmental conditions, importability into BIM, data volumes, degree of automation, operability, equipment portability, equipment durability and robustness. Through synthesis of these existing studies, this study considers six aspects of MR tool implementation within the context of O&M, these are:

Cost: Must be minimised to be attractive to companies with limited budgets. This includes both initial hardware costs and running costs.

Time: The tool should require as little time commitment as possible. Through increased automation of tasks and faster data retrieval, visualisation tools enable the reduction of time spent performing maintenance tasks.

Interoperability: The ability to integrate with current BIM systems and maintenance information is fundamental for ease of implementation.

Technical requirements: Required skills and expert knowledge should be low, in order to minimise the amount of specific training time and increase the number of staff who can use the tools.

Pre-existing data: BIM data must exist before using visualisation tools; the lower the data requirements (level of detail and completeness), the easier a visualisation tool is to implement and use.

Degree of Interactivity: The level to which the environment can be interacted with and the ease with which relevant information can be retrieved.

3 Operation and Maintenance in Small-to-Medium Enterprises

Large businesses are able to commit more resources to implementing technology to aid processes and are usually the first to benefit from technological advances. BIM is an area which has been primarily focused towards large companies and projects, however SMEs, which contributed £1.8 Trillion to the UK in 2014, can also benefit from implementing it to support O&M [4]. Within FM, the maintenance process is the same as larger companies, where maintenance procedures and extensive collaboration between internal and external stakeholders is necessary. Visualisation tools present a method of improving these collaborations, which includes sharing and modification of data, information and models, and as barriers to entry are lower, they become increasing viable for small companies.

A challenge for smaller firms is the ability to provide cost-effective FM, which when combined with non-adaptive employees and a lack of understanding of the capabilities of BIM, can prevent the adoption of new technologies. Visualisation tools that require specific hardware, perform limited tasks and have high running costs would therefore be unsuitable to SMEs. In a large company, many employees would be committed to maintenance and can afford to spend more time in training and keeping BIM related data and models up-to-date. The creation of visualisation tools that require minimal training and are as time efficient as possible is therefore increasingly important.

As discussed in Sect. 2, implementing BIM and visualisation tools in FM requires pre-existing BIM data and models and is a challenge for small enterprises who would not possess or have access to digital models. Rather, paper-based floor plans are often the main source of documentation on an existing building. In order to create digital models, current data capture technologies are laser scanning, photogrammetry, RFID and Barcode tagging. Laser scanning produces the most accurate 3D models but is very expensive. Photogrammetry is a much cheaper method of creating models but with lower resolution. RFID and Barcodes are used to mark assets to provide quick information retrieval rather than creating digital models. Whilst a lower cost solution, tagging methods are limited to the applications they can be used for and it is time-consuming to mark every asset. For a SME, the cost of creating, curating and marking-up digital models and assets is a major barrier, which limits the visualisation tools that can be implemented to aid the FM process. Thus, the visualisation tool in turn must be suited to the level of BIM adopted by the company, which maybe relatively low (Level 0).

4 Mixed Reality

The following section will cover the state-of-the-art in Mixed Reality (MR) and its suitability for supporting the O&M activities of SMEs. MR refers to technology that involves the merging of real and virtual worlds and can vary from real environments to fully virtual environments [8]. The term incorporates technologies that enhance real environments with virtual objects, known as Augmented Reality (AR), and fully simulated environments, known as Virtual Reality (VR). MR offers benefits in terms of visualisation compared to standard screen based methods. Users are not limited to visualising 3D objects on 2D screens and by integrating the real world with technology data can be linked to real objects. Three different MR technologies will be reviewed in the following sections and appraised based on the maintenance activities they are suited to support and how they satisfy the requirements given in Sect. 2.

4.1 Augmented Reality

Augmented Reality involves the enhancement of the real world by overlaying virtual objects via a computer. Two different types of AR are considered; monitor-based devices and ‘See-through’ head mounted AR devices.

Research into the use of Augmented Reality for BIM and FM, has mainly looked at the effectiveness of information access using hand-held AR devices and applications in which it can be implemented. Within construction, site inspection and hazard identification can be enhanced by using computer vision to detect, and AR to highlight unsafe environments for workers [9]. Hidden utilities, such as underground pipes can be revealed, to help prevent collisions in laying pipework [10]. Augmenting assembly instructions can help improve construction speeds and reduce mistakes compared to paper-based approaches. A study in electrical construction found mistakes were reduced by 75%, whilst a study in using AR for piping assembly found a time and error reduction of 50% [11, 12].

Using either hand-held or head-mounted devices, AR is suitable for on-site work. Head Mounted devices such as the Microsoft HoloLens use a combination of inertial measurement unit and depth sensors to create basic 3D models, perceive distance of objects and recognise environments. This allows users to walk around and interact with objects, such as marking defective features and retrieve data in-situ. Therefore, AR would be effective in asset tracking, facilitating real-time data access and checking maintainability of assets, as they are on-site tasks.

Locating building components requires the process of the user localising themselves relative to assets within a building. Localising assets can be achieved using SLAM (Simultaneous Localisation And Mapping) or tagging. The current standard method involves either linking RFID or visual markers to assets, meaning users locate assets, and by detecting markers, information and assets can be augmented automatically. This provides a faster method of information access, however, assets must be found initially which can be time consuming. Using localisation methods, AR devices can be used to understand a user’s location in a building and provide directions to the desired asset. Through object recognition methods, the asset could be automatically identified, preventing the need to manually install asset tags. Manually installing RFID or visual markers is a very time-consuming process and requires regular servicing to ensure they remain in working condition. A comparison of mobile and head-mounted AR is shown in Table 1.

Table 1. Mobile and Head mounted AR against user requirements
Full size table

Two types of AR are presented, mobile and head-mounted. Both types are suited for in-situ maintenance, but vary in the cost, hardware requirements and level of interactivity. Mobile AR is the lowest cost and lowest skill solution, but users are limited to screen based visualisation with no spatial understanding capability.

4.2 Virtual Reality

Virtual Reality is the full simulation of an environment allowing a user to interact with it, usually via a head-mounted display. VR aims to fully immerse a user within a virtual world, even providing sensory feedback to enhance the degree of realism. VR headsets comprise two main types from mounted smartphones to more expensive and powerful computer-tethered devices.

A major focus of VR for BIM is within the design stage, before any physical infrastructure exists. Designs can be viewed in actual scale and explored fully, providing a better sense of how users will interact with the building and visually detect any design flaws. Further use of VR lies in training, where users can be put in specific scenarios too dangerous to experience in real settings, such as identifying construction hazards before going on-site [13]. A benefit of VR use in BIM is that within building simulations, user experience can be tested with much greater realism than conventional methods. For example, by using VR to simulate lighting configurations of buildings, the user can visualise room atmosphere and optimum lighting setup, which wouldn’t be possible by simply viewing on a computer screen [3]. Within FM, VR has been shown to be able to be used as a multi-user network to allow remotely situated stakeholders to collaborate using virtual building models to aid in maintenance decision-making [14].

Due to VR being a fully simulated environment, digital models are required. Thus, to implement, building models must either be pre-existing or be generated using data capture techniques. The use of head mounted devices is something that is unfamiliar to many people as methods of control are dissimilar to controlling smartphones or computers. Virtual worlds must also be specially designed to consider motion sickness and fatigued eyes. These factors mean that use of VR will need both specialist training and there is likely to be resistance to uptake, as people don’t feel confident in using the devices. VR enables off-site inspection; therefore, time can be saved through preventing physical inspections. The suitability of VR with respect to the user requirements is shown in Table 2.

Table 2. Virtual Reality against user requirements
Full size table

Mobile and tethered VR have similar capabilities, but tethered VR has higher processing power as they use powerful computer hardware and motion tracking to increase interactivity and allow users to move around virtual environments.

4.3 Photospheres

Spherical panoramas, known as Photospheres, are 360-degree spherical images, most commonly used in Google Street View. They can be created by using specialist cameras, such as the Ricoh Theta S, or by stitching multiple images on smartphones. Although comprised as static images stitched together, Photospheres can be considered to be a sub-set of virtual reality, as a real environment is recreated in a virtual environment. This provides a greater level of immersion than standard images and a higher level of resolution than common Virtual Reality environments.

At present, the use of Photosphere beyond Google Street View remains limited. The capability to better understand an environment has been demonstrated through virtual tours mainly in museums and galleries [15]. The potential in extracting information automatically has also been explored, an example being mapping out road sign location using computer vision within Street View [16]. Within the area of BIM, more advanced 3D models can be generated using photogrammetry and further used as a tool to compare deviation of buildings from the ‘as-built’ to ‘designed as’ models [17, 18].

Photospheres have the benefit of a high level of detail, meaning users can easily relate their actual environment to the digital model. If a user was to stand in the same place as the Photosphere, what they observe on their device should be identical to the real world. This easy to understand environment could aid the localisation of maintenance issues by building users. By integrating with building plans, from 2D plans to 3D models, a greater degree of realism to BIM systems can be introduced.

Unlike Virtual Reality, the computing power required to create, view and manipulate Photospheres is very low due to the static image format and ability to be viewed interactively on modern web browsers. Capturing a building with Photospheres is a predominantly manual process. Similar to Street View with cars driving through streets taking photos, a person is required to walk around a desired building and capture the panoramas of each room. However, as only images are taken in rooms, the process is much quicker than taking manual measurements or laser scanning to create digital models. Table 3 shows the suitability of Photospheres to meet the requirements in Sect. 2.

Table 3. Photosphere against user requirements
Full size table

Photospheres offer a lost cost, low skill visualisation method. Users are constrained to visualisations of buildings based on capture location, but environments are easy to understand as they are photograph based.

5 Discussion

The three visualisation methods vary in their potential applications and their ability to meet the requirements developed in Sect. 2. Table 4 shows the comparison of each MR tool presented against the elicited set of user requirements from Sect. 2.

Table 4. Comparison of MR tools based on user requirements
Full size table

Both VR and head mounted AR devices require higher technical skill to use, due to methods of control many people are unfamiliar with. Due to the unusual methods of use, cultural resistance to implementation is likely to be greater than the other tools. Both Photospheres and AR are compatible on smartphones and tablets, therefore have low skill requirements and have low initial costs as no specific hardware is required. Further, on-going costs are likely to be low due to open source toolkits and standards. Whilst VR and head mounted AR have the highest skill requirements, they also provide the highest level of interactivity, through 3D environments that can be manipulated and navigated easily. Mobile AR is limited to screen-based visualisations but with a higher degree of interactivity than photospheres. Photospheres are image-based therefore cannot be manipulated and environment visualisation is limited by the number of Photospheres captured. AR and Photospheres are web-based meaning there is higher compatibility across different data types and platforms including desktop and mobile devices [19].

VR has the highest data requirements, as digital models are needed to visualise building information. Companies must have a BIM Level 2+ (detailed CAD models and asset information) or use data capture methods to generate models. AR can be used with either full digital models or with asset tagging, a lower cost solution. Future AR may employ localisation and object recognition to allow building navigation and remove the need for models or asset tagging. Photospheres employ a more manual approach to asset location and information retrieval, therefore have much lower data requirements. Given the capabilities and limitations of each technology, the applications they are suited to varies, which is summarised in Table 5.

Table 5. Suitability of mixed reality tools to maintenance activities
Full size table

AR and Photospheres are generally more portable than VR and can integrate with real environments and thus can be used effectively in in-situ maintenance tasks. VR however, is limited to off-site use only as the visualisation is a full virtual environment.

There is no clear MR tool that would be the most effective in FM, with the most suitable tool being dependent on the company needs, desired application and current level of BIM maturity. For smaller companies with low budgets, the current level of BIM adoption is likely to be low. Without up-to-date 3D building models, use of VR tools would not be advantageous.

Using Photospheres to locate building components requires no extra hardware and can be used with any level of BIM maturity. In buildings, where only 2D plans are present, Photospheres can be integrated to give a level of 3D visualisation without the need for full digital models. Whilst benefits are gained in cost and pre-existing data required, there is a low level of interactivity as Photospheres are image based. The process of setting up Photospheres within a building is also time consuming, as photos must be captured and assets manually linked within each Photosphere.

Where slightly larger budgets and workforces are accessible, locating building assets and data access could be facilitated using AR. Current AR tools require assets being marked using either RFID or visual markers, which along with initial setup time, require regular servicing to ensure markers work correctly.

In a company with an already high level of adoption, the use of VR should be considered. With a high level of maturity already present, using VR can aid in collaboration, training and checking assets off-site.

6 Conclusion

Mixed Reality tools present a method of enhancing the management of buildings within Operation and Maintenance. This paper explored the requirements for implementation and use of mixed reality tools, and the current state of the art in Virtual Reality, Augmented Reality and Photospheres.

VR was found to be best suited for visualisation and collaboration activities, however high cost and high pre-requisite data requirements limit its applicability to SMEs. Both AR and Photospheres could be applied to locating building components, facilitating real-time data access and visualisation activities. Mobile AR and Photospheres are low cost solutions, which can run with both web-based applications and smart phones. Their low cost also relates to their implementation where they have little to no on-going costs. High device compatibility means there are low technical skill requirements and high interoperability, therefore would be well suited to smaller companies. Current AR requires either assets to be tagged or digital models, whereas Photospheres can be used with any level of BIM adoption and buildings can be quickly mapped using smart phones or specific cameras. For a small company with low levels of BIM maturity (level 0 or 1), Photospheres can most easily be implemented to support the O&M process.

Whilst there are benefits to using Mixed Reality tools to support maintenance within small companies, current barriers limit their widespread implementation. Future development in the technologies could see them becoming used regularly within building management.

In terms of the long-term outlook, the conceptual implications of integrating Photospheres with 2D plans may offer the opportunity to leapfrog existing BIM level 0/1 projects to level 4 projects. Further, advanced image processing techniques may allow for automatic asset identification and the construction if 3D models. These aspects are being considered by the Authors in their future research.