Passive Profiling and Natural Interaction Metaphors for Personalized Multimedia Museum Experiences

  • Svebor Karaman
  • Andrew D. Bagdanov
  • Gianpaolo D’Amico
  • Lea Landucci
  • Andrea Ferracani
  • Daniele Pezzatini
  • Alberto Del Bimbo
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8158)

Abstract

Museums must balance the amount of information given on individual pieces or exhibitions in order to provide sufficient information to aid visitor understanding. At the same time they must avoid cluttering the environment and reducing the enjoyment of the exhibit. Moreover, each visitor has different interests and each might prefer more (or less) information on different artworks depending on their individual profile of interest. Finally, visiting a museum should not be a closed experience but a door opened onto a broader context of related artworks, authors, artistic trends, etc. In this paper we describe the MNEMOSYNE system that attempts to provide such a museum experience. Based on passive observation of visitors, the system builds a profile of the artworks of interest for each visitor. These profiles of interest are then used to personalize content delivery on an interactive table. The natural user interface on the interactive table uses the visitor’s profile, a museum content ontology and a recommendation system to personalize the user’s exploration of available multimedia content. At the end of their visit, the visitor can take home a personalized summary of their visit on a custom mobile application. In this article we describe each component of our approach as well as the first field trials of our prototype system built and deployed at our permanent exhibition space at Le Murate in the city of Florence.

Keywords

Computer vision video surveillance cultural heritage multimedia museum personalization natural interaction 

References

  1. 1.
    Baber, C., Bristow, H., Cheng, S.L., Hedley, A., Kuriyama, Y., Lien, M., Pollard, J., Sorrell, P.: Augmenting museums and art galleries. In: Human-Computer Interaction, INTERACT, vol. 1, pp. 439–447 (2001)Google Scholar
  2. 2.
    Bagdanov, A.D., Del Bimbo, A., Landucci, L., Pernici, F.: MNEMOSYNE: Enhancing the museum experience through interactive media and visual profiling. In: Grana, C., Cucchiara, R. (eds.) MM4CH 2011. CCIS, vol. 247, pp. 39–50. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  3. 3.
    Bay, H., Fasel, B., Van Gool, L.: Interactive museum guide: Fast and robust recognition of museum objects. In: Proceedings of the First International Workshop on Mobile Vision (May 2006)Google Scholar
  4. 4.
    Bimbo, A.D., Lisanti, G., Masi, I., Pernici, F.: Person detection using temporal and geometric context with a pan tilt zoom camera. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 3886–3889. IEEE (2010)Google Scholar
  5. 5.
    Bruns, E., Brombach, B., Zeidler, T., Bimber, O.: Enabling mobile phones to support large-scale museum guidance. IEEE MultiMedia 14(2), 16–25 (2007)CrossRefGoogle Scholar
  6. 6.
    Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2005, vol. 1, pp. 886–893. IEEE (2005)Google Scholar
  7. 7.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press (2004) ISBN: 0521540518Google Scholar
  8. 8.
    Hatala, M., Wakkary, R.: User modeling and semantic technologies in support of a tangible interface. Journal of User Modeling and User Adapted Interaction 15(3-4), 339–380 (2005)CrossRefGoogle Scholar
  9. 9.
    Karaman, S., Bagdanov, A.D.: Identity inference: Generalizing person re-identification scenarios. In: Fusiello, A., Murino, V., Cucchiara, R. (eds.) ECCV 2012 Ws/Demos, Part I. LNCS, vol. 7583, pp. 443–452. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  10. 10.
    Karaman, S., Lisanti, G., Bagdanov, A.D., Del Bimbo, A.: From re-identification to identity inference: labelling consistency by local similarity constraints. In: Person Re-identification (to appear, 2013)Google Scholar
  11. 11.
    Kuflik, T., Stock, O., Zancanaro, M., Gorfinkel, A., Jbara, S., Kats, S., Sheidin, J., Kashtan, N.: A visitor’s guide in an active museum: Presentations, communications, and reflection. Journal on Computing and Cultural Heritage (JOCCH) 3(3), 11 (2011)Google Scholar
  12. 12.
    Wang, Y., Stash, N., Sambeek, R., Schuurmans, Y., Aroyo, L., Schreiber, G., Gorgels, P.: Cultivating personalized museum tours online and on-site. Interdisciplinary Science Reviews 34(2-3), 2–3 (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Svebor Karaman
    • 1
  • Andrew D. Bagdanov
    • 1
  • Gianpaolo D’Amico
    • 1
  • Lea Landucci
    • 1
  • Andrea Ferracani
    • 1
  • Daniele Pezzatini
    • 1
  • Alberto Del Bimbo
    • 1
  1. 1.Media Integration and Communication Center (MICC)University of FlorenceFlorenceItaly

Personalised recommendations