Automatic Creation of a Virtual/Augmented Gallery Based on User Defined Queries on Online Public Repositories
Virtual museums are becoming increasingly popular, especially thanks to the recent spread of low-cost immersive technologies enabling a richer technology-based cultural offer. However, creating a virtual museum commonly requires a lot of effort, especially if a certain visual quality is required.
The aim of the project described in this paper is to verify the effectiveness of automatic strategies to create a virtual museum of paintings, whose digital images are retrieved directly from repositories freely available in the network based on specific user queries. The same approach is then applied in order to create an augmented reality application transforming any environment, purposely instrumented with markers, in a virtual museum.
KeywordsVirtual reality Augmented reality Cultural heritage Virtual museums Procedural generation Public repository
Virtual museums are becoming increasingly popular as they allow “to complement, enhance, or augment the museum through personalization, interactivity, user experience and richness of content” . Creating a virtual museum commonly requires a lot of effort in terms of design, content management, and visual rendering, effort unavoidable whenever a given quality of the experience is to be achieved. However, it is possible to think at digital collections which, even if not carefully designed as proper virtual museums, provide an immersive experience richer and, expectantly, more engaging than digital galleries as commonly found in standard web pages.
The aim of the project described in this paper is to verify the effectiveness of automatic strategies to create a virtual museum of paintings, whose digital images are retrieved directly from repositories freely available in the network. The virtual museum is based on a virtual environment, initially empty, that can be populated with various digital elements based on specific user queries. Since the number of artworks can be variable, the structure of the museum is modular, i.e. it changes shape based on the number of works selected by the user. The goal of this project is to provide a tool enabling the exploration of a wide variety of artworks taken from public heritage repositories such as Europeana and Google Art Project, or from other generic repositories of multimedia information like Google Images.
2 Previous Work
Since the advent of VRML, 3D virtual museums have shown their potential as a tool for disseminating culture and knowledge using digital technologies and taking advantage of the advanced features provided by VR such as immersion and presence.
One of the first VR projects applied to cultural heritage was the Nu.M.E, or “New Electronic Museum of the city of Bologna” . Launched in the first half of the 1990s, it can be considered one of the reference points for subsequent works in the field of virtual technologies applied to cultural heritage.
The MUVI  (Virtual Museum of Daily Life of the twentieth century) stems from a series of reflections on the memory of people and the events that preceded us above all in relation to the “danger” of losing all the collective heritage that brings together the knowledge related to everyday life and the identity of the various communities. The project, launched in 1999 at the VisIT Lab of CINECA, focuses on the transmission and conservation of past knowledge that can be favored by appropriate technological solutions.
A different approach was proposed in the Museum of Pure Forms : in addition to presenting a VR system including graphic software and sterescopic visualization, it also includes a haptic interface to provide touch feedback. This enables a novel way of presenting historical/artistic artifacts to visitors, allowing them to touch artifacts, albeit in a virtual environment.
The ARCO project  aims to develop a series of technologies to “help museums create, manipulate, manage and present digitized cultural objects in virtual exhibitions accessible both inside and outside museums”. In the project there is a database storing information regarding digital representations, which can be managed by a system called ACMA (acronym of “ARCO Content Management Application”). ACMA associates each 3D model with a series of metadata outlining four main aspects of the museum’s objects: the curatorial aspect, the technical aspect, the exploration of resources and administration.
An investigation on how to go beyond 3D web-based virtual museums and embed virtual technologies in the offer of real physical museums is instead proposed in .
An approach similar to the one proposed in this paper was pursued in DynaMus . In this case, also, the virtual content is dynamically downloaded from open data repositories, albeit the generated museum is made available on standard web browsers without any immersive/augmented option.
3 Architecture of the VR Museum
In the second phase the requested information based on the selected search criteria are retrieved. A C++ library is used to this purpose.
The third and last phase consists in the creation of the virtual museum, using the XVR  software.
3.1 Search Criteria
Depending on the repository chosen, the user can perform two types of searches: enter the name of an artist or, in the case of Wikimedia Commons, a collection of works of art. Afterwards the user can choose the number of paintings to be visualize inside the virtual museum (in order to keep the download time within reasonable limits it was decided to set a maximum of 50 artworks, although no investigation has been performed about the optimal value of this parameter).
In addition to the searching on Google Images and Europeana, currently about 255 collections are indexed through an XML file created starting from Wikimedia Commons collected information regarding the works on the Google Cultural Institute. Each component of the web page is controlled by JS code in order to allow only valid queries. Once the user has outlined the parameters to perform the search, the XVR graphics engine has to be started.
the first element is an ID corresponding to the chosen repository (Google Images, Wikimedia Commons, Europeana);
the second element is a string corresponding to the name of the artist or the name of the art gallery;
the third element is the amount of artworks to be inserted in the virtual museum.
These elements are concatenated into a single text string that is passed to the XVR plugin.
3.2 Resource Retrieval
Once the user XVR application has started, a preliminary phase begins consisting in the retrieval of information that takes place by means of a DLL library written in C++ compiled as a XVR external module, in charge of communicating with the server API of the selected repository.
In the case of Google Images, information retrieval is not a very laborious process, as the only information to be recovered is that relating to the URLs of the images to be downloaded. The solution adopted is to make a simple GET Request to Google’s servers to return an HTML page containing high-resolution images links. The HTML page is parsed in order to fill an array containing all the needed links, which are first examined and checked prior to the actual download of the image.
the resource is actually reachable and returns a response code of 0;
the image to be downloaded is not be empty;
the images size is at least 10 K in order to avoid low-resolution resources.
A very different process is used to retrieve resources from Wikimedia Commons. In fact, it is possible to interface with an API in order to request, in addition to links to high-resolution images, also metadata containing information about the works. In addition, instead of receiving an HTML page in response to the GET REQUEST, a JSON object is received that is easily “parsed” through additional library methods.
the first phase consists in the getting a list of the items present in a given collection;
the second phase consists in the retrieval of the connection to high resolution images and related metadata.
In Europeana the process is somehow similar to that of Wikimedia Commons. Initially a list of items is obtained accessing the Europeana search engine and, subsequently, the application queries each item in order to retrieve related metadata and multimedia resources. Europeana provides several tools to retrieve information programmatically; dedicated APIs are available to perform searches in databases or to query entities and other open data. The tool used for file retrieval is the Search API.
4 Dynamic Generation of Virtual Museum
After retrieving all the information and resources related to the artworks, the next step is the creation of the virtual museum: a dynamic structure with three-dimensional models and multimedia resources placed in the appropriate spaces.
One of the peculiarities of the gallery is modularity: it does not have a fixed size, but varies according to the number of works requested by the user. 3DS Max software was used for the creation of three-dimensional models.
The virtual museum is divided into blocks: its general shape, corresponding to a box, is initially divided into four parts, with models for the top right, top left, bottom right and bottom left parts. Subsequently, “corridor” blocks are added to create a larger museum, with a shape depending on the side they are placed.
An assisted navigation mode facilitates the exploration of the virtual museum and improves the user experience in visiting the environment. The system consists in labeling the rooms of the museum and providing a list of the artworks present in the different rooms, so that the visitor can move from one room to another to see the desired paintings without having to complete the whole itinerary.
5 Architecture of the AR Museum
5.1 First Itinerary: Panels
The mobile augmented reality version of the procedural museum has been implemented as an Android app, using the Unity3D and Vuforia tools.
It can be seen that, with respect to the VR museum, in this case only two repositories are supported. The procedure of data retrieval and resource downloading is similar in many respects to the one used in the VR museum. Once the data has been collected from the various activities, the Unity3D graphical engine is started.
In this case, Unity3D is used in combination with Vuforia, a library allowing to easily marker-based augmented reality apps. Vuforia makes available to Unity an AR Camera and imports the defined set of markers as asset files. The Unity 3D scene is composed of quads on which markers are placed following a predefined order. Prior to the execution of the real-time application, each marker is associated to a certain downloaded resource. Once a marker is recognized, it will be substituted by the painting automatically associated to the marker ID.
6 Performance Test
7 Pilot User Study
The obtained results show a good potential of the methodology adopted and of the pilot implementation sketched out and used in this study.
The VR museum constitutes a good alternative of showing images of paintings in a sort a virtual 3D slideshow. However better results in terms of engagement could be obtained by presenting the environment in a fully immersive setup such as an HMD or a CAVE, while in this first pilot tests users were using a desktop environment.
Particularly promising seems the use of an AR paradigm as it has aroused great curiosity and proved to be extremely engaging, with the users amazed to transform the environment they actually are inside in an on-demand digital gallery.
A possible improvement will be the implementation of a vocal interface to perform queries or some actions within the museum, in order to increase immersion. An interesting feature in the augmented reality application would be the use of a markerless implementation, in order to avoid a preliminary instrumentation of the physical space hosting the experience.
This paper is supported by European Union’s Horizon 2020 research and innovation programme under grant agreement No 692103, project eHERITAGE (Expanding the Research and Innovation Capacity in Cultural Heritage Virtual Reality Applications).
- 3.Ponti, F.D., Guidazzoli, A., Liguori, M.C.: Virtual domestic environments and historical narration: a methodological hypothesis. In: EVA 2009 Florence, Conference Proceedings (2009)Google Scholar
- 4.Loscos, C., et al.: The museum of pure form: touching real statues in an immersive virtual museum. In: VAST, pp. 271–279, December 2004Google Scholar
- 5.Wojciechowski, R., Walczak, K., White, M., Cellary, W.: Building virtual and augmented reality museum exhibitions. In: Proceedings of the Ninth International Conference on 3D Web Technology, pp. 135–144. ACM, April 2004Google Scholar
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.