Keywords

1 Introduction

Cultural Heritage (CH) represents an expression of the ways of living that were developed by people inside a community and passed on to the new generations. CH encompasses several components, such as customs, places, practices, artistic expressions and values. UNESCO (United Nations Educational, Scientific and Cultural Organization) classifies heritage in three main categories: cultural heritage (tangible or intangible), natural heritage and heritage in the event of armed conflict [1]. Tangible cultural heritage is further defined as having three components: movable cultural heritage, such as manuscripts, paintings, coins, sculptures; immovable cultural heritage, for example archaeological sites and monuments; underwater cultural heritage, which is represented by underwater ruins and cities, shipwrecks [2].

Computer applications applied to cultural heritage are mainly focused at improving the processes of digitization and documentation of artifacts and sites, digital preservation and exploration [3]. In the past modern technologies were used only by CH professionals, such as archaeologists, architects and civil engineers. Recently, more and more museums, archaeological places and exhibitions have begun to explore the use of new technologies to create new types of interaction with the aim to enhance the user experience (UX). There are several interactive solutions that are used in CH, such as location-aware audio guides, online and mobile applications, games, interactive multi-touch displays, virtual/augmented reality systems, 3D virtual worlds and other types of installations which include even kinesthetic control [4].

The term “augmented reality” has been used since the 60’s [5] and represents a bridge between the real environment and virtual reality. The best way to understand AR is with the help of the “Virtuality Continuum” concept introduced in 1995 by [6]. The term refers to a scale that starts from a real environment and shows the “road” to a totally virtual environment, passing through augmented reality and augmented virtuality. Therefore, augmented reality is the first stage in which virtual objects are superimposed on a real scene. The advances of technology are allowing developers to create user friendly AR applications that work on mobile devices, thus bringing AR closer to the general public.

Project Tango is a platform for augmented reality that offers a simple API for non-experts in computer vision. Its primary use case is to create interactive applications that have the capability to recognize the environment. The devices that support this technology need to be equipped with a complex package of cameras consisting of an RGB camera, a depth camera and a fisheye camera for motion tracking. A capable processor is required to analyze and fuse the data from the cameras and from inertial sensors of the device. The depth camera can be used to detect distances to surfaces in the environment. Another interesting use case is to create 3D models very fast and convenient. However, the 3D objects obtained using a Project Tango device are of poor quality and with limited functionality. The ability to recognize the environment allows developers to create applications that will no longer require the use of visual markers, thus enabling a more natural interaction with cultural heritage. As the name suggest, this technology was only a project and was shut down since the 1st of March in 2018. Developers can still create AR applications based on Tango, but there is no support provided and no future releases.

ARCore is based on Project Tango with fewer features that are not dependent on specialized cameras, thus allowing a larger number of devices to be compatible with its technology. The main drawbacks of ARCore is the lack of Area Learning and occlusion detection. There is a wide variety of Android phones running Android 7.0 (Nougat) and later that support this technology, as opposed to the two commercial devices that are compatible with Project Tango.

The main purpose of this study is to evaluate the users’ perception regarding two mobile augmented reality applications that allow the exploration of a 3D representation of a fortified church. The questionnaire includes empirical items to determine if participants noticed any significant differences between the two AR apps and to find out which they prefer. A secondary focus is to highlight the importance of preserving cultural heritage and to show how the use of advanced technology can bring tangible and intangible CH back to life.

The remainder of the paper is structured as follows. Section 2 provides theoretical background on technological approaches in cultural heritage, presents a comparison of frameworks for mobile AR development and also shortly approaches the digitization techniques that are commonly used in CH. Section 3 presents the methodology of the study, including the questionnaire that was used. Results are presented and discussed in Sect. 4, while the conclusions are organized in Sect. 5.

2 Theoretical Background

2.1 Technological Approaches in Cultural Heritage

Preservation, education, and entertainment are essential points regarding to cultural heritage sites, and technology integration is viewed as an essential element of service delivery in a museum environment [7]. Thus, new ways have been exploited to enhance the communication between users and heritage sites. The aim of technological approaches can vary from preservation and valorization of cultural heritage, support for artistic creation, facilitating access to and involvement in culture, protecting pluralism, freedom of expression, and cultural diversity [8].

A variety of different technological solutions are found in previous studies in order to improve the relationship between cultural heritage content and users. Among the several solutions proposed, it is found different kinds of games like: educational games, e.g. the role-play digital game named “Taiwan Epic Game” where it was intended to create the historical context of Southern Taiwan in the late nineteenth century [9]; or location-based games, e.g. “Gossip at Palace,” a location-based mobile game developed for an Italian historical residence to communicate its 18th-century history to teenagers [10]. Another technological solution frequently found are virtual reality explorations, e.g. a 3D model of the lost township of Caen [11]; or a virtual exploration on mobile devices based on a natural interaction approach of cultural heritage sites which are not accessible [12]; or a proposal for better access and communication of the Cultural Heritage information through to the visualization and disclosure of 3D digital contents [13]. A set of the main interactive technologies and interaction styles are presented in a study published by Koutsabasis, namely, 3D game engine, mobile, kinesthetic interaction, mobile AR, virtual reality, web, multi-touch display, physical computing, multimedia, virtual world, location-based audio, wearable, and AR [4].

Notwithstanding the riches facilitated by the use of the various technological solutions, regarding to AR in cultural heritage sites, it has economic, experiential, social, epistemic, cultural and historical, and educational value from both internal and external stakeholders’ perspectives [14].

2.2 Mobile AR in Cultural Heritage

The adoption of AR in cultural heritage began as early as 1999 with the MARS project [15] and the ARCHEOGUIDE Project [16], which were considered as mobile AR although the large amount of heavy devices needed to carry in these experiments. In the last few years, mobile AR refers to mobile devices easily portable such as smartphones or tablets [17]. These pioneer projects were followed by other research projects related to mobile AR applied to cultural heritage sites aiming at exhibition enhancement, being followed by reconstruction and exploration [18].

Looking at some of the several studies that tested the use of AR in cultural heritage with the aim of improving the visitors’ experience among these spaces, it is common to find acceptance studies to understand the users’ acceptance and intention to use a technology like the study of acceptance and use of AR in cultural heritage outdoors through the exhibition of photographs [19], or a study made to test users’ satisfaction and intention to recommend marker-based augmented reality applications in Jeju Island, South Korea [20].

Regarding to some recent AR technology approaches, a virtual restoration of the religious heritage objects through 3D AR technology content was proposed [21]. The CHESS project introduced an overview of handheld AR in museums, aiming to design and evaluate personalized interactive stories for visitors of cultural sites [22]. The potential of AR for supporting mobile tourism applications is emphasised with CorfuAR, a mobile augmented reality tour guide presented by Kourouthanassis et al. [23]. An adaptive mixed reality system was proposed aiming to achieve the visualization of in-situ virtual ancient building reconstructions with the MixAR project [24]. A mobile augmented reality guide for cultural heritage to examine user experiences was evaluated in the historic city centre of Brno in Czech Republic [25]. KnossosAR is an outdoors mobile AR guide implemented for the archaeological site of Knossos which intended to improve the user’s perception about its surrounding space hiding or utilizing appropriate visual metaphors for occluded objects/locations [26].

2.3 Comparison of Frameworks for Mobile AR Development

A research related to the frameworks available to implement AR mobile apps on mobile devices was made and an overview of this research is described below. According to this study concern, all frameworks presented provide the development of its AR application for Android or iOS devices. Among this research, the frameworks which provided their latest versions before 2016 were dropped. The following description is made considering the characteristics that each framework points out the most. Considering specific features, the results of the comparative study is presented in Table 1.

Table 1. Comparative study of the SDKs available to implement AR mobile systems.
Full size table

In the following, we will summarize the most popular AR platforms available on the market, presenting them in a concise manner, in alphabetical order as they appear in the table.

ARKit [27] is an Apple’s platform that combines different techniques and algorithms, like motion tracking, camera scene capture or advanced scene processing in order to allow users to easily build AR experiences. ARCore [28] is Google’s version of AR platform that uses different APIs, enabling the device to sense its environment, understand the world and interact with information.

ARToolKit [29] is an open-source AR software library providing support for three categories of tracker: natural feature tracking (NFT), traditional template square maker, and 2D Barcode Markers. ARToolKit was acquired by DAQRI since 2015 and, for this reason, this framework will not be considered in the comparative study. Another AR open-source minimal library for detecting squared fiducial markers in images is ArUco [30]. It is written in C++, is extremely fast and can calibrate cameras to make camera pose estimation (static or moving).

Augmented Pixels [31] has proprietary simultaneous localization and mapping technology (SLAM SDK) optimized for low CPU usage (Raspberry Pi 3 with standard Raspberry Pi camera sensor for SLAM real time processing). Also, it gets autonomous navigation by connecting to a drone or a robot through a Vision Processing Unit. Catchoom’s toolbox, from the Catchoom CraftAR Pixels [32] offers Image Recognition and AR tools in their branded apps targeted to transform the way consumers discover and shop for products in the real world using visual search.

EasyAR SDK [33] is a free AR engine developed by VisionStar Information Technology (Shanghai) which provides a number of cutting-edge features (such as SLAM, 3D tracking, and screen recording). Kudan AR SDK [34] can support marker or markerless tracking and location requirements, having a very robust single-camera SLAM tracking technology.

MAXST AR SDK [35], a cross-platform AR engine, also provides features and environments needed to develop AR apps. The main focus of this platform is on Natural Feature Tracking. Based on ARToolKit, NyARToolkit [36] is a free open-source project developing a vision-based AR written in Java.

Tango [37] was a phone and tablet-based mobile AR solution that relied on advanced camera hardware to build 3D meshes of spaces developed by a division of Google. Google shut down Tango in order to focus on the more mass market ARCore product. Vuforia [38] allows to develop AR apps with advanced vision and recognition of a range of everyday images, objects and environments.

The last but not least, Wikitude’s all-in-one AR SDK [39] combines instant tracking technology (SLAMFootnote 1), object recognition and tracking, top-notch image recognition and tracking, as well as geo-location AR for mobile, tablets and smart glasses.

In order to get to know all of the cited frameworks in this study, a comparative study was made and it is resumed in Table 1.

Observing Table 1 it is possible to realize that the great majority of the frameworks listed allows the image 2D tracking and the majority provides a package to implement the technology using Unity software. The usage of a 3D object to accomplish the tracking process is limited to few frameworks, where, in the case of the EasyAR it’s confined to the Pro version. The usage of sensors to find the localization of the device, typically, the frameworks which appeal to GPS also provide IMU sensors usage. In the case of Vuforia, that does not provide geo-localization with GPS sensors, it still uses IMU sensors to undertake the AR experience. The SLAM is assured by a large part of the frameworks presented, whereas the occlusion is covered by a small part of them, namely, ArUco and Tango. Albeit this two, ArUco does not provide an SDK to implement the technology which is important to help its implementation to reach a larger number of developers.

2.4 Digitization Techniques in Cultural Heritage

The digital preservation of CH is more important than ever due to several factors: the deterioration of the materials, natural phenomena like earthquakes, hurricanes, tsunami or other, armed conflicts and other human related problems. The main motivations for preserving CH are: (1) to create a digital replica; (2) to create a database with information related to the shape and appearance of an object; (3) to create new types of applications that are based on digital media collections to enhance the users’ experience [40]. A brief state of the art regarding the 3D digitization process related to cultural heritage is presented in Table 2.

Table 2. State of the art solutions for the 3D digitization process applied to cultural heritage
Full size table

Structure from Motion or multi-image photogrammetry is a practical and versatile technique that is being adopted widely for accurate digital capture of 3D objects and surfaces related to cultural heritage [42,43,44,45]. Photogrammetry is a method that analyzes and matches features detected in overlapping photos, and is fundamentally based on trigonometry. There are a few guidelines that need to be taken into consideration when any object is photographed for 3D reconstruction that ensure accurate and reliable results:

  • The most important rule is to keep an overlapping percentage of 60% to 80% between successive photos;

  • The light and white balance should be the same or with few changes. In the case of large scale objects, such as buildings or other monuments, a cloudy sky offers the best lighting conditions due to a uniform spread of light, thus avoiding the appearance of hard shadows;

  • Use a professional digital camera with at least 24 megapixels (relatively good results can be obtained also with a smartphone camera), at a small aperture to have a “large” depth of field.

3 Methodology

The aim of the experiment is to obtain a subjective evaluation of two mobile augmented reality applications that use different technologies. The virtual object was obtained using photogrammetry and it represents the Prejmer fortified church from Brasov, a UNESCO monument.

3.1 Participants

For this study 38 participants (average age = 24.08, SD = 5.43) have used the AR applications. There were no requirements necessary in order to participate. Participants have confirmed that they use a smartphone very frequently, however only 18 have experienced augmented reality prior to the experiment.

3.2 Procedure

The experiment took place at a public event organized by the Transilvania University of Brasov. Participants were first presented the aim of the study after which they received instructions on how to use the applications. The AR application based on Project Tango was installed on a Lenovo Phab 2 Pro (see Fig. 2), while the one based on ARCore was tested using the Huawei P20 Pro (see Fig. 1). Half of the participants started with the Phab 2 Pro, while the other half started with the P20 Pro. Their task was to explore the 3D model of the fortified church without having a time limit. After using one technology participants were asked to complete an evaluation questionnaire and then move on to the second technology.

Fig. 1.
figure 1

Screenshot from the Huawei device

Full size image
Fig. 2.
figure 2

Screenshot from the Lenovo device

Full size image

For the Project Tango application we used the Ikariotikos package, while for the ARCore we used package version 1.2. The only difference in the usage of the two applications is that for the ARCore version, users first have to scan the surroundings in order to detect a surface on which they can then place the virtual object.

3.3 Questionnaire Design

The questionnaire is adapted from the Handheld Augmented Reality Usability Scale (HARUS) [46] and contains 21 items based on a 7-point Likert scale, ranging from 1 – “strongly disagree”, to 7 – “strongly agree” (see Table 3). There are two questions (Q1–Q2) that are meant to reveal the familiarity of the participants with AR applications and their normal usage of a mobile device. In order to evaluate the comprehensibility parameter we used eight items (Q3–Q10) and seven items (Q11–Q17) to assess how difficult or physical demanding it was to interact with the two applications. Four items (Q18–Q21) reflect if the participants found the applications pleasant or boring. Using the last three questions (Q19–Q21) we wanted to find out their general perception of the experiment, if they believe that an AR application can help them learn more about cultural heritage and if they notice any significant differences between the two devices.

Table 3. Evaluation questionnaire
Full size table

4 Results

Results showed that the application is easy to use, intuitive and with relevant information. Related to comprehensibility, the scores computed for the Project Tango device were slightly better due to the bigger size of the display. A large display offers a better user experience, however in the case of handheld devices this can cause discomfort in a long term usage. Regarding manipulability, the ARCore device was rated a bit better, mostly because of its size and weight.

There was no significant difference when it comes to enjoyment. Using augmented reality allowed users to explore cultural heritage in a new and exciting way (Figs. 3 and 4).

Fig. 3.
figure 3

Box plots showing the results from Project Tango

Full size image
Fig. 4.
figure 4

Box plots showing the results from ARCore

Full size image

Other observations regarding the empirical questions:

  • Participants declared that the AR application increased their interest in learning about cultural heritage

  • The experience using the AR application was very positive

  • Some participants declared that the Tango enabled device was more stable, however most of the users did not notice any significant differences between the two technologies.

5 Conclusions

Cultural heritage stands for more than just an object, it represents the way we used to live and could even be considered a national treasure. Finding new methods to preserve, document and explore CH using modern technologies is of great interest and many breakthroughs have been made by archaeologists, researchers or museum curators.

In this paper we developed an augmented reality application that gives users the chance to visualize and explore a 3D model of a fortified church. The virtual model was obtained by our project team using photogrammetry with the aim to obtain a digital replica of a recognized UNESCO monument.

Project Tango has gained the attention of many entities during its development as it offered a practical and easy to use technology to implement augmented reality. With the help of specialized sensors and powerful hardware it yielded great results. However, it was more of a niche solution with relatively small potential to be adopted widely by regular consumers. Nonetheless, several museums and other commercial companies have developed attractive and interesting applications using this technology. Project Tango had a lot of potential, however it was replaced by ARCore in a move to boost the number of mobile AR applications.

The present study shows that there is no significant difference between the two technologies in simply exploring a virtual object. Things would be different in the case of a marker-less application or trying to use the device for 3D reconstruction, where only Project Tango is feasible.