Multimedia Tools and Applications

, Volume 53, Issue 2, pp 391–429 | Cite as

Towards the semantic and context-aware management of mobile multimedia

  • Windson VianaEmail author
  • Alina Dia Miron
  • Bogdan Moisuc
  • Jérôme Gensel
  • Marlène Villanova-Oliver
  • Hervé Martin


Users of mobile devices can nowadays easily create large quantities of mobile multimedia documents tracing significant events attended, places visited or, simply, moments of their everyday life. However, they face the challenge of organizing these documents in order to facilitate searching through them at a later time and sharing them with other users. We propose using context awareness and semantic technologies in order to improve and facilitate the organization, annotation, retrieval and sharing of personal mobile multimedia documents. Our approach combines metadata extracted and enriched automatically from the users’ context with annotations provided manually by the users and with annotations inferred by applying user-defined rules to context features. These new contextual metadata are integrated into the processes of annotation, sharing and keyword-based retrieval.


Semantic web Spatial reasoning Context awareness Multimedia annotation Semantic image retrieval Index expansion 


  1. 1.
    Aizawa K, Hori T, Kawasaki S, Ishikawa T (April 20, 2004) Capture and efficient retrieval of life log, Pervasive 2004 Workshop on Memory and Sharing Experiences pp.15-20, Vienna, AustriaGoogle Scholar
  2. 2.
    Aleksy M, Butter T, Schader M (2008) Architecture for the development of context-sensitive mobile applications. Mobile Inf Syst 4(2):105–117Google Scholar
  3. 3.
    Ames M, Naaman M (2007) Why we tag: motivations for annotation in mobile and online media. CHI '07: Proceedings of the SIGCHI conference on Human factors in computing systems. CHI '07. ACM, New York, NY, 971-980Google Scholar
  4. 4.
    Bloehdorn S, Petridis K, Saathoff C, Simou N, Tzouaras V, Avrithis Y, Handschuh S, Kompatsiaris Y, Staab S, Strintzis MG (2005) Semantic annotation of images and videos for multimedia analysis, in: Proceedings of the 2nd European Semantic WebGoogle Scholar
  5. 5.
    Buscaldi D, Rosso P, Peris P (August 2006) Inferring geographical ontologies from multiple resources for geographical information retrieval. In: C Jones, R Purves (eds) Proceedings of 3rd SIGIR Workshop on Geographical Information RetrievalGoogle Scholar
  6. 6.
    Carvalho RF, Chapman S, Ciravegna F (2009) Attributing semantics to personal photographs. Multimed Tools Appl 42(1):73–96CrossRefGoogle Scholar
  7. 7.
    Christensen CB (2001) Place and experience: a philosophical topography. Mind, Oxford University Press, Volume 110, Number 439, 1 July 2001, pp 789-792(4)Google Scholar
  8. 8.
    Dey AK, Abowd GD (2000) Towards a better understanding of context and context-awareness. HUC '99: Proceedings of the 1st international symposium on Handheld and Ubiquitous Computing, Springer-Verlag, London, UK, 304-307Google Scholar
  9. 9.
    Filho JB, Martin H (2008) QACBAC: an owner-centric QoC-aware context-based access control model for pervasive environments. In Proceedings of the SIGSPATIAL ACM GIS 2008 international Workshop on Security and Privacy in GIS and LBS (Irvine, California, November 04 - 04, 2008). SPRINGL '08. ACM, New York, NY, 30-38Google Scholar
  10. 10.
    Frank AU (1996) Qualitative spatial reasoning: cardinal directions as an example. Int J Geogr Inf Sci 10(3):269–290CrossRefGoogle Scholar
  11. 11.
    Goyal RK, Egenhofer M (2001) Cardinal directions between extended spatial objects. IEEE Trans Knowl Data Eng (in press)Google Scholar
  12. 12.
    Gulliver SR, Ghinea G, Patel M, Serif T (2007) A context-aware Tour Guide: user implications. Mobile Inf Syst 3(2):71–88Google Scholar
  13. 13.
    Guo B, Satake S (2008) Michita Imai: home-explorer: ontology-based physical artifact search and hidden object detection system. Mobile Inf Syst 4(2):81–103Google Scholar
  14. 14.
    Hammiche S, Lopez B, Benbernou S, Hacid M-S (2008) Query rewriting for semantic multimedia data retrieval. Adv Comput Intel Ind Syst, 351-372Google Scholar
  15. 15.
    Heesch D (2008) A survey of browsing models for content based image retrieval. Multimed Tools Appl 40(2):261–284CrossRefGoogle Scholar
  16. 16.
    Hendler J (2008) Web 3.0: chicken farms on the semantic web. Computer 41(1):106–108CrossRefGoogle Scholar
  17. 17.
    Hollink L, Nguyen G, Schreiber G, Wielemaker J, Wielinga B, Worring M (2004) Adding spatial semantics to image annotations. Proc. of 4th International Workshop on Knowledge Markup and Semantic AnnotationGoogle Scholar
  18. 18.
    Jones CB, Alani H, Tudhope D (2001) Geographical information retrieval with ontologies of place. In Proceedings of the international Conference on Spatial information theory. Lecture Notes In Computer Science, vol. 2205. Springer-Verlag, London, 322-335Google Scholar
  19. 19.
    Koolwaaij J, Tarlano A, Luther M, Nurmi P, Mrohs B, Battestini A, Vaidya R (2006) Context watcher: sharing context information in everyday life. Web Technologies, Applications, and Services: WTAS 2006: Acta Press 2006Google Scholar
  20. 20.
    Liu Y, Zhang D, Lu G, Ma WY (2007) A survey of content-based image retrieval with high-level semantics. Pattern Recognit 40(1):262–282CrossRefzbMATHGoogle Scholar
  21. 21.
    Lux M, Klieber W, Granitzer M (2004) Caliph & Emir: semantics in multimedia retrieval and annotation, Proceedings of 19th International CODATA Conference: The Information Society: New Horizons for Science, Berlin, GermanyGoogle Scholar
  22. 22.
    Matellanes A, Evans A, Erdal B (May, 2006) Creating an application for automatic annotations of images and video. Proc. of 1st International Workshop on Semantic Web Annotations for Multimedia (SWAMM), Edinburgh, ScotlandGoogle Scholar
  23. 23.
    Miron AD, Gensel J, Villanova-Oliver M (2007) Towards the geo-spatial querying of the semantic web with ONTOAST. 7th International Symposium on Web and Wireless GIS (W2GIS 2007), Cardiff, UKGoogle Scholar
  24. 24.
    Moisuc B, Davoine PA, Gensel J, Martin H (2005) Design of spatio-temporal information systems for natural risk management with an object-based knowledge representation approach. Geomatica 59Google Scholar
  25. 25.
    Monaghan F, O'Sullivan D (2006) Automating photo annotation using services and ontologies, MDM'06: Proceedings of the 7th International Conference on Mobile Data Management. IEEE Computer Society, Washington, pp 79–83Google Scholar
  26. 26.
    Naaman M, Harada S, Wang Q, Garcia-Molina H, Paepcke A (2004) Context data in geo-referenced digital photo collections. MULTIMEDIA '04: Proceedings of the 12th annual ACM international conference on Multimedia, 196-203, New York, NY, USAGoogle Scholar
  27. 27.
    O'Hare N, Gurrin C, Jones GJF, Lee H, O'Connor NE, Smeaton AF (2007) Using text search for personal photo collections with the MediAssist system. In Proceedings of ACM SAC 2007Google Scholar
  28. 28.
    Raento M, Oulasvirta A, Petit R, Toivonen H (2005) 2005. ContextPhone: a prototyping platform for context-aware mobile applications. Pervasive Computing, IEEE 4:51–59CrossRefGoogle Scholar
  29. 29.
    Ressler J, Dean M (2007) Geospatial ontology trade study. Ontology for the Intelligence Community (OIC-2007), Columbia, MarylandGoogle Scholar
  30. 30.
    Riley M, Heinen E, Ghosh J (2008) A text retrieval approach to content-based audio retrieval, Proceedings of the Ninth International Conference on Music Information Retrieval, SeptemberGoogle Scholar
  31. 31.
    Salton G, Wong A, Yang CS (1975) A vector space model for automatic indexing. Commun ACM 18(11):613–620CrossRefzbMATHGoogle Scholar
  32. 32.
    Sarvas R, Herrarte E, Wilhelm A, Davis M (2004) Metadata creation system for mobile images. MobiSys '04: Proceedings of the 2nd international conference on Mobile systems, applications, and services, ACM, Boston, MA, USA, 36-48Google Scholar
  33. 33.
    Schroeter R, Hunter J, Kosovic D (2003) Vannotea, A collaborative video indexing, annotation and discussion system for broadband networks, in: Proceedings of the K-CAP 2003 Workshop on Knowledge Markup and Semantic Annotation, October 2003, FloridaGoogle Scholar
  34. 34.
    Schweer A, Hinze A. The Digital Parrot: Combining Context-Awareness and Semantics to Augment Memory, Workshop "Supporting Human Memory with Interactive Systems", HCI Conference, September 4th, 2007, Lancaster, UKGoogle Scholar
  35. 35.
    Tuffield M, Harris S, Dupplaw DP, Chakravarthy A, Brewster C, Gibbins N, O'Hara K, Ciravegna F, Sleeman D, Wilks Y, Shadbolt NR (2006) Image annotation with Photocopain. In: First International Workshop on Semantic Web Annotations for Multimedia (SWAMM 2006) at WWW2006, May 2006, Edinburgh, United KingdomGoogle Scholar
  36. 36.
    Uren V, Cimiano P, Iria J, Handschuh S, Vargas-Vera M, Motta E, Ciravegna F (2006) Semantic annotation for knowledge management: requirements and a survey of the state of the art. Web Semant Sci Serv Agents World Wide Web 4:14–28CrossRefGoogle Scholar
  37. 37.
    Viana W, Castro RMC, Machado J, Filho B, Magalhaes K, Giovano C (2004) Mobis: a solution for the development of secure applications for mobile device. In: ICT: International Conference on Telecommunications, 11th, Fortaleza-CE, Brazil. Lecture Notes in Computer Science, Vol. 3124, pp 1015-1022, 2004Google Scholar
  38. 38.
    Viana W, Andrade RMC (2008) XMobile: a MB-UID environment for semi-automatic generation of adaptive applications for mobile devices (2008a). J Syst Softw 81(3):382–394CrossRefGoogle Scholar
  39. 39.
    Viana W, Bringel F, Gensel J, Villanova-Oliver M, Martin H (2007) PhotoMap automatic spatiotemporal annotation for mobile photos. Lect Notes Comput Sci 4857:187–201CrossRefGoogle Scholar
  40. 40.
    Viana W, Bringel F, Gensel J, Villanova-Oliver M, Martin H (2008) PhotoMap: from location and time to context-aware photo annotations. J Location Based Serv 2(3):211–235CrossRefGoogle Scholar
  41. 41.
    Voorhees EM (1999) The TREC-8 question answering track report. In Proceedings of the 8th Text Retrieval Conference, Gaithersburg, Maryland, USA, pp 77-82Google Scholar
  42. 42.
    Wang S, Jing F, Jing F, He J, Du Q, Zhang L. IGroup: presenting web image search results in semantic clusters. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '07. ACMGoogle Scholar
  43. 43.
    Yamaba T, Takagi A, Nakajima T (2005) Citron: a context information acquisition framework for personal devices, Proc. of 11th International Conference on Embedded and real-Time Computing Systems and Applications, 489-495Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Windson Viana
    • 1
    Email author
  • Alina Dia Miron
    • 1
  • Bogdan Moisuc
    • 1
  • Jérôme Gensel
    • 1
  • Marlène Villanova-Oliver
    • 1
  • Hervé Martin
    • 1
  1. 1.Laboratory of Informatics of Grenoble(LIG), STEAMER TeamSaint Martin d’HèresFrance

Personalised recommendations