Advertisement

Object Recognition and Tracking for Smart Audio Guides

  • Lorenzo Seidenari
  • Claudio Baecchi
  • Tiberio Uricchio
  • Andrea Ferracani
  • Marco Bertini
  • Alberto Del Bimbo
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 806)

Abstract

In this paper we address the problem of creating a smart audio guide that adapts to the actions and interests of tourists. Our guide performs automatic recognition of artworks and allows the users instant or deferred fruition of multimedia content. We use a compact CNN as computer vision system to back the whole application to performs object classification, localization and recognition. Tracking is used to improve the recognition accuracy over sequences of detections. We also provide an automatic pipeline for dataset creation based on the same tracking algorithm. The system, deployed on an NVIDIA Jetson TK1 and an NVIDIA Shield Tablet, has been tested in a real world environment.

Keywords

Object recognition Cultural heritage 

References

  1. 1.
    Bowen, J.P., Filippini-Fantoni, S.: Personalization and the web from a museum perspective. In: Proceedings of Museums and the Web (MW) (2004)Google Scholar
  2. 2.
    Karaman, S., Bagdanov, A.D., Landucci, L., D’Amico, G., Ferracani, A., Pezzatini, D., Del Bimbo, A.: Personalized multimedia content delivery on an interactive table by passive observation of museum visitors. Multimedia Tools Appl. 75(7), 3787–3811 (2016)CrossRefGoogle Scholar
  3. 3.
    Seidenari, L., Baecchi, C., Uricchio, T., Ferracani, A., Bertini, M., Bimbo, A.D.: Deep artwork detection and retrieval for automatic context-aware audio guides. ACM Trans. Multimedia Comput. Commun. Appl. 13(3s), 35:1–35:21 (2017)CrossRefGoogle Scholar
  4. 4.
    Baraldi, L., Paci, F., Serra, G., Benini, L., Cucchiara, R.: Gesture recognition using wearable vision sensors to enhance visitors’ museum experiences. IEEE Sens. J. 15(5), 2705–2714 (2015)Google Scholar
  5. 5.
    Abowd, G.D., Atkeson, C.G., Hong, J., Long, S., Kooper, R., Pinkerton, M.: Cyberguide: a mobile context-aware tour guide. Wireless Netw. 3(5), 421–433 (1997)CrossRefGoogle Scholar
  6. 6.
    Wang, Y., Stash, N., Sambeek, R., Schuurmans, Y., Aroyo, L., Schreiber, G., Gorgels, P.: Cultivating personalized museum tours online and on-site. Interdisc. Sci. Rev. 34(2–3), 139–153 (2009)CrossRefGoogle Scholar
  7. 7.
    Zancanaro, M., Kuflik, T., Boger, Z., Goren-Bar, D., Goldwasser, D.: Analyzing museum visitors’ behavior patterns. In: Conati, C., McCoy, K., Paliouras, G. (eds.) UM 2007. LNCS (LNAI), vol. 4511, pp. 238–246. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-73078-1_27 CrossRefGoogle Scholar
  8. 8.
    Kuflik, T., Boger, Z., Zancanaro, M.: Analysis and prediction of museum visitors’ behavioral pattern types. In: Krüger, A., Kuflik, T. (eds.) Ubiquitous Display Environments. Cognitive Technologies, pp. 161–176. Springer, Heidelberg (2012).  https://doi.org/10.1007/978-3-642-27663-7_10 CrossRefGoogle Scholar
  9. 9.
    Eliseo, V., Martine, L.: Ethnographie de l’exposition. Études et recherche, Centre Georges Pompidou, Bibliothèque publique d’information (1991)Google Scholar
  10. 10.
    Keil, J., Pujol, L., Roussou, M., Engelke, T., Schmitt, M., Bockholt, U., Eleftheratou, S.: A digital look at physical museum exhibits: designing personalized stories with handheld augmented reality in museums. In: Proceedings of Digital Heritage International Congress (DigitalHeritage) (2013)Google Scholar
  11. 11.
    Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar

Copyright information

© Springer International Publishing AG 2018

Authors and Affiliations

  • Lorenzo Seidenari
    • 1
  • Claudio Baecchi
    • 1
  • Tiberio Uricchio
    • 1
  • Andrea Ferracani
    • 1
  • Marco Bertini
    • 1
  • Alberto Del Bimbo
    • 1
  1. 1.University of FlorenceFlorenceItaly

Personalised recommendations