Encyclopedia of Computer Graphics and Games

Living Edition
| Editors: Newton Lee

Mixed Reality, Gamified Presence, and Storytelling for Virtual Museums

  • George Papagiannakis
  • Efstratios Geronikolakis
  • Maria Pateraki
  • Victor M. López-Menchero
  • Michael Tsioumas
  • Stella Sylaiou
  • Fotis Liarokapis
  • Athina Grammatikopoulou
  • Kosmas Dimitropoulos
  • Nikolaos Grammalidis
  • Nikolaos Partarakis
  • George Margetis
  • Giannis Drossis
  • Martha Vassiliadi
  • Alan Chalmers
  • Constantine Stephanidis
  • Nadia Magnenat-Thalmann
Living reference work entry
DOI: https://doi.org/10.1007/978-3-319-08234-9_249-1

Synonyms

Definition

Mixed reality as display technology, gamification as motivational element, and storytelling as interaction metaphor while maintaining the feeling of presence are identified as key elements for defining the next generation of virtual museums.

Introduction

Storytelling, presence, and gamification are three basic fields that need to be taken into account when developing novel mixed reality applications for cultural heritage, based on the recent renaissance of commercial VR and AR hardware. This survey aims to cover a gap in the bibliography and the last relevant surveys of Papagiannakis et al. (2008), Jung et al. (2011), Foni et al. (2010), and Anderson et al. (2009) which are more than 7 years old, whereas some of these topics are mentioned in several different chapters within Ioannides et al. (2017) but not in single, comparative...

This is a preview of subscription content, log in to check access

References

  1. Anderson, E.F., McLoughlin, L., Liarokapis, F., Peters, C., Petridis, P., de Freitas, S.: Serious games in cultural heritage. In: Ashley, M., Liarokapis, F. (eds.) The 10th International Symposium on Virtual Reality, Archaeology and Cultural Heritage VAST – State of the Art Reports, St Julians, Malta, 22–25 Sept 2009, pp. 29–48Google Scholar
  2. Ardito, C., Buono, P., Desolda, G., Matera, M.: Empowering CH experts to produce IoT-enhanced visits. In: UMAP ‘17 25th Conference on User Modeling, Adaptation and Personalization, Bratislava, 9–12 July 2017, pp. 327–328. ACM, New York (2017)Google Scholar
  3. Bimber, O., Encarnação, L.M., Schmalstieg, D.: The virtual showcase as a new platform for augmented reality digital storytelling. In: IPT/EGVE03 Imersive Progection Technologies/Eurographics Virtual Environments, Zurich, 22–23 May 2003, pp. 87–95. ACM, New York (2003)Google Scholar
  4. Bowman, D.A., McMahan, R.P.: Virtual reality: how much immersion is enough? IEEE Comput. 40(7), 36–43 (2007)CrossRefGoogle Scholar
  5. Bugalia, N., Kumar, S., Kalra, P., Choudhary, S.: Mixed reality based interaction system for digital heritage. In: VRCAI ‘16 The 15th International Conference on Virtual-Reality Continuum and its Applications in Industry, Zhuhai, 3–4 December 2016, pp. 31–37. ACM, New York (2016)Google Scholar
  6. Bujari, A., Ciman, M., Gaggi, O., Palazzi, C.E.: Using gamification to discover cultural heritage locations from geo-tagged photos. Pers. Ubiquit. Comput. 21(2), 235–252 (2017)CrossRefGoogle Scholar
  7. Casillo, M., Colace, F., De Santo, M., Lemma, S., Lombardi, M., Pietrosanto, A.: An ontological approach to digital storytelling. In: MISNC, SI, DS 2016 The 3rd Multidisciplinary International Social Networks Conference, SocialInformatics 2016, Data Science 2016, Union, 15–17 August 2016, p. 27. ACM, New York (2016)Google Scholar
  8. Cavazza, M., Charles, F., Mead, S.J.: Character-based interactive storytelling. IEEE Intell. Syst. 17(4), 17–24 (2002)CrossRefMATHGoogle Scholar
  9. Christopoulos, D., Mavridis, P., Andreadis, A., Karigiannis, J. N.: Using virtual environments to tell the story: the battle of Thermopylae. In: 2011 Third International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES), pp. 84–91. National Technical University Great Amphitheater, Athens (2011)Google Scholar
  10. Dimitropoulos, K., Manitsaris, S., Tsalakanidou, F., Denby, B., Buchman, L., Dupont, S., Nikolopoulos, S., Kompatsiaris, Y., Charisis, V., Hadjileontiadis, L., Pozzi, F., Cotescu, M., Ciftci, S., Katos, A., Manitsaris, A. and Grammalidis, N.: A Multimodal Approach for the Safeguarding and Transmission of Intangible Cultural Heritage: The Case of i-Treasures. IEEE Intell Syst, pp. 1–1 (2018)Google Scholar
  11. Dong, Y., Webb, M., Harvey, C., Debattista, K., Chalmers, A.: Multisensory virtual experience of tanning in medieval Coventry. In: EUROGRAPHICS Workshop on Graphics and Cultural Heritage, Graz, 27–29 Sept 2017Google Scholar
  12. Drossis, G., Grammenos, D., Bouhli, M., Adami, I., Stephanidis, C.: Comparative evaluation among diverse interaction techniques in three dimensional environments. In: International Conference on Distributed, Ambient, and Pervasive Interactions, pp. 3–12. Springer, Berlin/Heidelberg (2013a)Google Scholar
  13. Drossis, G., Grammenos, D., Adami, I., Stephanidis, C.: 3D visualization and multimodal interaction with temporal information using timelines. In: IFIP Conference on Human–Computer Interaction, pp. 214–231. Springer, Berlin/Heidelberg (2013b)Google Scholar
  14. Drossis, G., Ntelidakis, A., Grammenos, D., Zabulis, X., Stephanidis, C.: Immersing users in landscapes using large scale displays in public spaces. In: International Conference on Distributed, Ambient, and Pervasive Interactions, pp. 152–162. Springer, Cham (2015)Google Scholar
  15. Foni, A., Papagiannakis, G., Magnenat-Thalmann, N.: A taxonomy of visualization technologies for cultural heritage applications. ACM J. Comput. Cult. Herit. 3(1), 1–21 (2010)CrossRefGoogle Scholar
  16. Gimeno, J., Olanda, R., Martinez, B., Sanchez, F.M.: Multiuser augmented reality system for indoor exhibitions. In: IFIP Conference on Human–Computer Interaction, pp. 576–579. Springer, Berlin/Heidelberg (2011)Google Scholar
  17. Grammenos, D., Michel, D., Zabulis, X., Argyros, A.A.: PaperView: augmenting physical surfaces with location-aware digital information. In: TEI ‘11 Fifth International Conference on Tangible, Embedded, and Embodied Interaction, Funchal, 22–26 January 2011, pp. 57–60. ACM, New York (2011)Google Scholar
  18. Grammenos, D., Zabulis, X., Michel, D., Padeleris, P., Sarmis, T., Georgalis, G., …, Adam-Veleni, P.: Macedonia from fragments to pixels: a permanent exhibition of interactive systems at the archaeological museum of Thessaloniki. In: Euro-Mediterranean Conference, pp. 602–609. Springer, Berlin/Heidelberg (2012)Google Scholar
  19. Hamari, J., Koivisto, J., Sarsa, H.: Does gamification work? – a literature review of empirical studies on gamification. In: Proceedings of the 47th Hawaii International Conference on System Sciences (HICSS), IEEE Computer Society, Waikoloa, pp. 3025–3034 (2014)Google Scholar
  20. Huang, C.H., Huang, Y.T.: An annales school-based serious game creation framework for Taiwanese indigenous cultural heritage. J. Comput. Cult. Herit. 6(2), 9 (2013)CrossRefGoogle Scholar
  21. Ioannides, M., Magnenat-Thalmann, N., Papagiannakis, G. (eds.): Mixed Reality and Gamification for Cultural Heritage. Springer, Heidelberg (2017).  https://doi.org/10.1007/978-3-319-49607-8Google Scholar
  22. Javornik, A., Rogers, Y., Gander, D., Moutinho, A.: MagicFace: stepping into character through an augmented reality mirror. In: CHI ‘17 CHI Conference on Human Factors in Computing Systems, Denver, 6–11 May 2017, pp. 4838–4849. ACM, New York (2017)Google Scholar
  23. Jung, Y., Kuijper, A., Fellner, D.W., Kipp, M., Miksatko, J., Gratch, J., Thalmann, D.: Believable virtual characters in human–computer dialogs. Eurographics (STARs). 2011, 75–100 (2011)Google Scholar
  24. Jung, T., tom Dieck, M.C., Lee, H., Chung, N.: Effects of virtual reality and augmented reality on visitor experiences in museum. In: Information and Communication Technologies in Tourism 2016, pp. 621–635. Springer, Cham (2016)Google Scholar
  25. Jung, T.H., Jung, T.H., tom Dieck, M.C., tom Dieck, M.C.: Augmented reality, virtual reality and 3D printing for the co-creation of value for the visitor experience at cultural heritage places. J. Place Manag. Dev. 10(2), 140–151 (2017)CrossRefGoogle Scholar
  26. Kitsikidis, A., Kitsikidis, A., Dimitropoulos, K., Uğurca, D., Bayçay, C., Yilmaz, E., Tsalakanidou, F., …, Grammalidis, N.: A game-like application for dance learning using a natural human computer interface. In: International Conference on Universal Access in Human–Computer Interaction, pp. 472–482. Springer, Cham (2015)Google Scholar
  27. Klingensmith, M., Dryanovski, I., Srinivasa, S., Xiao, J.: Chisel: real time large scale 3D reconstruction onboard a mobile device using spatially hashed signed distance fields. In: Robotics: Science and Systems, vol. 4 (2015).  http://dx.doi.org/10.15607/rss.2015.xi.040
  28. Kosmalla, F., Zenner, A., Speicher, M., Daiber, F., Herbig, N., Krüger, A.: Exploring rock climbing in mixed reality environments. In: CHI ‘17 CHI Conference on Human Factors in Computing Systems, Denver, 6–11 May 2017, pp. 1787–1793. ACM, New York (2017)Google Scholar
  29. Koutsabasis, P., Vosinakis, S.: Kinesthetic interactions in museums: conveying cultural heritage by making use of ancient tools and (re-) constructing artworks. Virtual Reality. pp 1–16. (2017).  https://doi.org/10.1007/s10055-017-0325-0
  30. Li, N., Duh, H.: Cognitive issues in mobile augmented reality: an embodied perspective. In: Huang, W., et al. (eds.) Human Factors in Augmented Reality Environments. Springer, New York (2013).  https://doi.org/10.1007/978-1-4614-4205-9_5Google Scholar
  31. Liarokapis, F., Petridis, P., Andrews, D., de Freitas, S.: Multimodal Serious Games Technologies for Cultural Heritage, Mixed Reality and Gamification for Cultural Heritage, Part V, pp. 371–392. Springer International Publishing (2017a).  http://dx.doi.org/10.1007/978-3-319-49607-8_15
  32. Liarokapis, F., Kouřil, P., Agrafiotis, P., Demesticha, S., Chmelík, J., Skarlatos, D.: 3D modelling and mapping for virtual exploration of underwater archaeology assets. In: Proceedings of the 7th International Workshop on 3D Virtual Reconstruction and Visualization of Complex Architectures and Scenarios, ISPRS, Napflio, pp. 425–431 (2017b)Google Scholar
  33. Lu, W., Zeng, D., Pan, J.: An XML-based scene description language for 3D virtual museum. In: 30th International Conference on Information Technology Interfaces, 2008, ITI 2008, Croatia Hotel, Cavtat/Dubrovnik, pp. 445–450. IEEE (2008)Google Scholar
  34. Mancuso, S., Muzzupappa, M., Bruno, F.: Edutainment and gamification: a novel communication strategy for cultural heritage. J Pub Archaeology. 1, 79–89 (2017)Google Scholar
  35. Margetis, G., Ntoa, S., Antona, M., Stephanidis, C.: Interacting with augmented paper maps: a user experience study. In: Proceedings of the 12th Biannual Conference of the Italian SIGCHI Chapter (CHITALY 2017), Cagliari, 18–20 Sept 2017Google Scholar
  36. Marshall, M.T., Dulake, N., Ciolfi, L., Duranti, D., Kockelkorn, H., Petrelli, D.: Using tangible smart replicas as controls for an interactive museum exhibition. In: TEI ‘16 Tenth International Conference on Tangible, Embedded, and Embodied Interaction, Eindhoven, 14–17 February 2016, pp. 159–167. ACM, New York (2016)Google Scholar
  37. Mortara, M., Catalano, C.E., Bellotti, F., Fiucci, G., Houry-Panchetti, M., Petridis, P.: Learning cultural heritage by serious games. J. Cult. Herit. 15(3), 318–325 (2014)CrossRefGoogle Scholar
  38. Nakevska, M., van der Sanden, A., Funk, M., Hu, J., Rauterberg, M.: Interactive storytelling in a mixed reality environment: the effects of interactivity on user experiences. Entertain. Comput. 21, 97–104 (2017)CrossRefGoogle Scholar
  39. Ott, R., Thalmann, D., Vexo, F.: Haptic feedback in mixed-reality environment. Vis. Comput. 23(9), 843–849 (2007)CrossRefGoogle Scholar
  40. Papaefthymiou, M., Papagiannakis, G.: Gamified augmented and virtual reality character rendering and animation enabling technologies. In: Ioannides, M., Magnenat-Thalmann, N., Papagiannakis, G. (eds.) Mixed Reality and Gamification for Cultural Heritage, pp. 333–357. Springer (2017).  https://doi.org/10.1007/978-3-319-49607-8
  41. Papagiannakis, G.: Gamification and serious games. In: Lee, N. (ed.) Encyclopedia of Computer Graphics and Games. Springer, Cham.,  https://doi.org/10.1007/978-3-319-08234-9_90-1, ISBN 978-3-319-08234-9 (2017)Google Scholar
  42. Papagiannakis, G., Schertenleib, S., Ponder, M., Arevalo-Poizat, M., Magnenat-Thalmann, N., Thalmann, D.: Real-time virtual humans in AR sites. In: IEE Visual Media Production (CVMP04), London, pp 273–276 (2004)Google Scholar
  43. Papagiannakis, G., Singh, G., Magnenat-Thalmann, N.: A survey of mobile and wireless technologies for augmented reality systems. J Comput. Anim. Virtual Worlds. 19(1), 3–22 (2008)CrossRefGoogle Scholar
  44. Partarakis, N., Antona, M., Zidianakis, E., & Stephanidis, C.: Adaptation and content personalization in the context of multi user museum exhibits. In B.N. De Carolis, C. Gena, T. Kuflik, & F. Nunnari (Eds.), Proceedings of the 1st Workshop on Advanced Visual Interfaces for Cultural Heritage (AVI*CH 2016), Bari, Italy, 7-10 June. Published by: CEUR-WS (Vol. 1621) (2016)Google Scholar
  45. Partarakis, N., Grammenos, D., Margetis, G., Zidianakis, E., Drossis, G., Leonidis, A., Stephanidis, C.: Digital cultural heritage experience in ambient intelligence. In: Mixed Reality and Gamification for Cultural Heritage, pp. 473–505. Springer, Cham (2017)Google Scholar
  46. Pedersen, I., Gale, N., Mirza-Babaei, P., Reid, S.: More than meets the eye: the benefits of augmented reality and holographic displays for digital cultural heritage. J Comput. Cult. Herit. 10(2), 11 (2017)CrossRefGoogle Scholar
  47. Petrelli, D., Dulake, N., Marshall, M.T., Pisetti, A., Not, E.: Voices from the war: design as a means of understanding the experience of visiting heritage. In: CHI ‘16 CHI Conference on Human Factors in Computing Systems, San Jose, 7–12 May 2016, pp. 1033–1044. ACM, New York (2016)Google Scholar
  48. Petrelli, D., Marshall, M.T., O’Brien, S., McEntaggart, P., Gwilt, I.: Tangible data souvenirs as a bridge between a physical museum visit and online digital experience. Pers. Ubiquit. Comput. 21(2), 281–295 (2017)CrossRefGoogle Scholar
  49. SpatialStories: A toolset for real-time interactive VR/AR experiences. http://apelab.ch/spatialstories (2018). Accessed 6 Jan 2018
  50. Sylaiou, S., Liarokapis, F., Kotsakis, K., Patias, P.: Virtual museums, a survey and some issues for consideration. J. Cult. Herit. 10(4), 520–528 (2009)CrossRefGoogle Scholar
  51. The Museum of Stolen Art.: http://mosa.ziv.bz/ (2017). Accessed 22 Dec 2017
  52. Tisserand, Y., Magnenat-Thalmann, N., Unzueta, L., Linaza, M.T., Ahmadi, A., O’Connor, N.E., Zioulis, N., Zarpalas, D., Daras, P.: Preservation and gamification of traditional sports. In: Ioannides, M., Magnenat-Thalmann, N., Papagiannakis, G. (eds.) Mixed Reality and Gamification for Cultural Heritage, pp. 421–446. Springer, Cham (2017)CrossRefGoogle Scholar
  53. Tsekleves, E., Darby, A.: The role of playfulness and sensory experiences in design for public health and for ageing well. In: Sensory Arts and Design. Bloomsbury p. 49–66. 18 p (2017)Google Scholar
  54. Vassileva, J.: Motivating participation in social computing applications: a user modeling perspective. User Model. User-Adap. Inter. 22(1–2), 177–201 (2012)MathSciNetCrossRefGoogle Scholar
  55. Vogel, D., Balakrishnan, R.: Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. In: UIST ‘04 17th Annual ACM Symposium on User Interface Software and Technology, Santa Fe, 24–27 October 2004, pp. 137–146. ACM, New York (2004)Google Scholar
  56. Williamson, J.R., Sundén, D.: Enter the circle: blending spherical displays and playful embedded interaction in public spaces. In: Proceedings of the 4th International Symposium on Pervasive Displays (PerDis ‘15), Saarbruecken, 10–12 June 2015, pp. 195–200. ACM, New York (2015)Google Scholar
  57. Zidianakis, E., Margetis, G., Paparoulis, S., Toutountzis, T., Stratigi, K., Paparoulis, G., Stephanidis, C.: Turning an electric cargo vehicle into a portable interactive information kiosk. In: International Conference on Human–Computer Interaction, pp. 463–469. Springer, Cham (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • George Papagiannakis
    • 1
    • 2
  • Efstratios Geronikolakis
    • 1
    • 2
  • Maria Pateraki
    • 2
  • Victor M. López-Menchero
    • 3
  • Michael Tsioumas
    • 4
  • Stella Sylaiou
    • 5
  • Fotis Liarokapis
    • 6
  • Athina Grammatikopoulou
    • 7
  • Kosmas Dimitropoulos
    • 7
  • Nikolaos Grammalidis
    • 7
  • Nikolaos Partarakis
    • 2
  • George Margetis
    • 2
  • Giannis Drossis
    • 1
    • 2
  • Martha Vassiliadi
    • 8
  • Alan Chalmers
    • 9
  • Constantine Stephanidis
    • 1
    • 2
  • Nadia Magnenat-Thalmann
    • 10
  1. 1.Computer Science DepartmentUniversity of CreteHeraklionGreece
  2. 2.Foundation for Research and Technology HellasHeraklionGreece
  3. 3.Spanish Society of Virtual ArchaeologySevilleSpain
  4. 4.Hellenic Ministry of Culture and Sports, Service of Modern Monuments and Technical Works of Central MacedoniaThessalonikiGreece
  5. 5.Hellenic Open UniversityPatrasGreece
  6. 6.Masaryk UniversityPonava, BrnoCzech Republic
  7. 7.Information Technologies Institute, CERTHThessalonikiGreece
  8. 8.School of Philology, Aristotle University of ThessalonikiThessalonikiGreece
  9. 9.WMG, University of WarwickCoventryUK
  10. 10.MIRALabUniversity of GenevaGeneva 4Switzerland