Media annotation

  • CL. Chrisment
  • FL. Sedes
Part of the Multimedia Systems and Applications Series book series (MMSA, volume 22)


Advanced multimedia systems require the development of complex processes and tools to enable the underlying structure and content to be easily understood and to facilitate access and manipulation of such information. Clearly defined semantics is an essential characteristic in this context. Modelling must provide, through the use of pattern recognition, indexing or classifying tools, high level descriptions or abstractions of multimedia data, content and structure. A query support must allow the user to query by the idea he has of their appearance rather than by their exact content. Querying by example (and counter-examples) or allowing for flexible queries seem to be natural in this case. Our proposal is a conceptual model which enables adaptable and reusable multimedia content. Particular emphasis has been put on detecting characteristic features of multimedia documents seen as semi-structured data and querying them, for instance with examples. The different uses of current query languages applied to the model are detailed. This chapter is organised into three sections. The first one deals with the generation of describers. The second one enhances the faceted description, according to the different dimensions of the media. The third part is dedicated to multimedia querying, the interpretation of which relies on describers.


Multimedia System Medium Annotation Multimedia Document Semantic Labelling Video Indexing 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


Generation of Describers

  1. [AL96]
    Ahanger, G., Little, T. D. C., ‘A Survey of Technologies for Parsing and Indexing Digital Video’. Visual Communication and Image Representation, 7(1), 28–43, 1996.CrossRefGoogle Scholar
  2. [AL97]
    Ardizzone, E., La Casia M., ‘Automatic Video database Indexing and Retrieval’, Multimedia Tools and Application, 4(1), 29–56, 1997.CrossRefGoogle Scholar
  3. [AND00]
    André-Obrecht, R. ‘Quelques mots à propos d’Indexation de Documents Sonores par le contenu’, GDR 13, GT 7.1 ‘Documents multimédia’, 4/05/2000, Toulouse.
  4. [AT94]
    Akutsu, A., Tonomura, Y., ‘Video Tomography: An Efficient Method for Camerawork Extraction and Motion Analysis’, Proc. ACM Multimedia′94, 349–356, 1994.Google Scholar
  5. [BER00]
    Bergholz A. ‘Extending your Mark-up: An XML Tutorial’, IEEE Internet Computing, July-August 2000, pp 74–79,Vol 4, No 4.Google Scholar
  6. [BOU00]
    Bouet, M. ‘Traitement de l’information multimédia: modélisation, indexation, traitement de la forme et recherche d’images dans un SGBD à objets’, Thèse de Doctorat, 2000, Université de Nantes.Google Scholar
  7. [BR94]
    Bohm, K., Rakow, C. ‘Metadata for Multimedia Documents’, ACM Sigmod Record, 23(4): 21–26, December 1994 — ISSN 0163–5808. Special issue: ‘Metadata for digital media’CrossRefGoogle Scholar
  8. [CCZ83]
    Chrisment, C., Crampes, J-B., Zurfluh, G. ‘The BIG project’, Proceedings of the 2nd International Conference on Databases, ICOD-2, Cambridge, 30 Août- 2 Septembre 1983, SM Deen, P. Hammersley Editeurs.Google Scholar
  9. [CLS00]
    Chrisment C., Le Maître J., Sedes F. ‘Bases d’objets documentaires’, Techniques de l’Ingénieur, 05/2000, Vol H7248, (14+4) pages.Google Scholar
  10. [CS00]
    Chrisment C., Sedes F. ‘Le document multimédia’, CIFED/CIDE 2000, Presses Polytechniques et universitaires romandes et INSA de Lyon, ISBN 2-88074-460-1, 10–20.Google Scholar
  11. [FRT+00]
    Ford, R. M., Robson, C., Temple, D., Gerlach, M. ‘Metrics for shot boundary detection in digital video sequences’, Multimedia Systems Vol 8, No 1, 2000, pp 37–46.CrossRefGoogle Scholar
  12. [GLA99]
    Gauvain, J-L., Lamel, L., Adda, G, ‘Audio Partitionning and Transcription for Broadcast Data Indexation’, CBMI′99 — Content-Based Multimedia Indexing, 25-27/10/99, Toulouse, pp 67–73.Google Scholar
  13. [GLA00]
    Gauvain, J-L., Lamel, L., Adda, G ‘Transcribing Broadcast News for Audio and Video Indexing’, CACM, February 2000, Vol 43, No 2, pp 64–70 —Special Issue: ‘News on Demand’.Google Scholar
  14. [GON99]
    Gong, Y. ‘Advancing content-based image retrieval by exploiting image colours and regions features’, Multimedia Systems Vol 7, No 6, November 1999, pp 449–457.CrossRefGoogle Scholar
  15. [HAM99]
    Hampapur, A. ‘Semantic Video Indexing: Approach and Issues’, SIGMOD Record, Vol. 28, No 1, March 1999, pp 32–39.CrossRefGoogle Scholar
  16. [HS99]
    Herrera, P., Serra, X. ‘A proposal for the description of audio in the context of MPEG-7’, CBMI′99 — Content-Based Multimedia Indexing, Oct. 1999, Toulouse, 81–88.Google Scholar
  17. [HSE+95]
    Hafner, J., Sawney, H., Equitz, W., Flickner M., Niblack W., ‘Efficient Colour Histogram Indexing for Quadratic Form Distance Functions’, IEEE Trans. Pattern Analysis and Machine Intelligence, 1(7), 729–736, Jul. 1995.CrossRefGoogle Scholar
  18. [HZ99]
    Hunter J., Zhan Z., ‘An indexing and Querying System for Online Images Based on the PNG Format and Embedded Metadata’, Proc. Arts Libraries Society-Australia and New Zealand-ARLIS/ANZ Conf., State library of Queensland, Brisbane, Australia, September 1999.Google Scholar
  19. [JOL96]
    Joly P. ‘Consultation et Analyse des Documents en Image Animée Numérique’ Thèse de Doctorat, Université de Toulouse3, 1996.Google Scholar
  20. [KC00]
    Kankanhalli, M. S., Chua, T-S., ‘Video modelling using Strata-Based Annotation’, IEEE Multimedia, Vol 7, No 1, January-March 2000, pp 68–74.CrossRefGoogle Scholar
  21. [LAS98]
    Lassila, O. ‘Web Metadata: A matter of Semantics’, IEEE Internet Computing, July-August 1998, Volume 2, No 4, pp 30–37.CrossRefGoogle Scholar
  22. [LE00]
    Lienhart, R., Effelsberg, W. ‘Automatic text segmentation and text recognition for vidéo indexing’, Multimedia Systems Vol 8, No 1, January 2000, pp 69–81.CrossRefGoogle Scholar
  23. [LM00]
    Lozano, R., Martin, H. ‘Intégration de données vidéo dans un SGBD à objets’, revue L’Objet, 6(3).Google Scholar
  24. [LPE97]
    Lienhart, R., Pfeiffer, S., Effelsberg, W., ‘Video Anstracting’, Comm. ACM, 40(12), 55–62, 10997.Google Scholar
  25. [LVQ+95]
    Lambollez, P-Y., Queille, J-P., Voidrot, J-F., Chrisment, C., ‘EXREP: un outil générique de réécriture pour l’extraction d’informations textuelles’, ISl Ingénierie des systèmes d’Information, Vol 3, No 4, 1995, pp 471–487.Google Scholar
  26. [MAHOO]
    Mahdi W., ‘Macro-segmentation Sémantique des Documents Audiovisuels à l’aide des Indices Spatio-temporels’, Doctorat Informatique, Ecole centrale de Lyon, 2000.Google Scholar
  27. [MM99a]
    Martinez, J., Mouaddib, N. ‘Multimedia and databases’, NIS (Networking and Information Systems), Volume 2, Nol, 1999, pp 89–123.Google Scholar
  28. [MM99b]
    Ma, W-Y., Majunath, B. S. ‘NeTra: A toolbox for navigating large image databases’, Multimedia Systems, Vol 7, No 3, May 1999, pp 184–198.CrossRefGoogle Scholar
  29. [NAC00]
    Nack, F. ‘All Contents Counts: The Future in Digital Média Computing is Meta’ IEEE Multimédia, July/September 2000, Vol 7, No 3, pp 10–13.Google Scholar
  30. [NL99]
    Nack, F., Lindsay, A. T. ‘Everything You Wanted to Know About MPEG-7’, IEEE Multimedia, July-September 1999, pp 65–77. Part 1, IEEE Multimedia, October-December 1999, pp 64–73. Part 2Google Scholar
  31. [PM95]
    Picard, R., Minka, T., ‘Vison Texture for Annotation’, Multimedia Systems, 3(3), “–14, 1995.Google Scholar
  32. [RHM99]
    Rui, Y., Huang, T. S., Mehrotra, S. ‘Constructing table-of-content for videos’, Multimedia systems, Vol 7, No 5, September 1999, pp 359–368.CrossRefGoogle Scholar
  33. [SKH+99]
    Sato, T., Kanade, T., Hughes, E.K., Smith, MA., Satoh, S. ‘Video OCR: indexing digital news libraries by recognition of superimposed captions’, Multimedia Systems Vol 7, No 5, September 1999, pp 385–395.CrossRefGoogle Scholar
  34. [SC99]
    Smith, J. R., Chang, S-F. ‘Integrated spatial and feature image query’, Multimedia Systems Vol 7, No 2, March 1999, pp 129–140.CrossRefGoogle Scholar
  35. [SI98]
    Sclaroff, S., Isidora, J., ‘Active Blobs’, Proc. Intl Conf. Computer Vision, 1998.Google Scholar
  36. [SJ97]
    Santini, S., Jain, R., ‘Similarity is a Geometer’, Multimedia Tools and Applications, 5(3), 277–306, 1997.CrossRefGoogle Scholar
  37. [W3C00a]
    ‘XML Schema Part 0, Primer, W3C Working Draft, 7 Apr. 2000,
  38. [W3C00b]
    ‘XML Schema Part 1, Structures, W3C Working Draft, 7 Apr. 2000, http://www.
  39. [W3C00c]
    ‘XML Schema Part 0, Datatypes, W3C Working Draft, 7 Apr. 2000,éma-2/.Google Scholar
  40. [WHH+99]
    Wen, X., Huffmire, T. D., Hu, H. H., Filkenstein, A. ‘Wavelet-based video indexing and querying’, Multimedia systems, Volume 7, No 5, pp 350–358, September 1999.CrossRefGoogle Scholar

Querying Languages and Retrieval

  1. [AL01]
    Ahanger, G., Little, T. D. C., ‘Data Semantics for Improving Retrieval Performance of Digital News Video Systems’, IEEE Transactions on Knowledge and Data Engineering, 13(3), May/June 2001, 352–360.CrossRefGoogle Scholar
  2. [BFJ+95]
    Brown, M.G., Foote, J.T., Jones G.J.F., Jones, K.S., Young, S.J., ‘Automatic Content-Based Retrieval of Broadcast News’, ACM Multimedia′95, 35–43, 1995.Google Scholar
  3. [CDF+99]
    Cluet, S., Deutsch, A., Florescu, D., Levy, A., Maier, D., McHugh, J., Robie, J., Suciu, D., Widom, J., ‘XML query languages: Experiences and exemplars’,
  4. [CFR+01]
    Chamberlin, D., Florescu, D., Robie, J., Simeon, J., Stefanescu, M., ‘Xquery: a query language for XML’,
  5. [CSB+97]
    Chang, S.F., Smith, J.R., Beigi, M., Benitez, A., ‘Visual Information retrieval from Large Distributed Online Repository’, Comm. ACM, 40(12), 63–72, 1997.CrossRefGoogle Scholar
  6. [DD94]
    DeRose S. J.,Durand D. D. ‘Making hypermedia work: a user’s guide to Hytime’, Kluwer Academic Publisher, 1994, ISBN 0-7923-9432-1.Google Scholar
  7. [DFF+98]
    Deutsch A., Fernandez M., Florescu D., Levy A., Suciu D. ‘XML-QL A query Language for XML’,
  8. [DPS01]
    Dubois, D., Prade, H., Sedes, F. ‘Fuzzy logic techniques in multimedia database querying’, IEEE Transaction on Data and Knowledge Engineering, Vol. 13(3), 2001.Google Scholar
  9. [FSW99]
    Fernandez M., Siméon J., Wadler P. ‘XML Languages: Experiences and exemplars’, html.
  10. [IBM99]
    IBM Digital Media Solutions Center, ‘Query by Audio Content’, IBM Germany, 1999,
  11. [JVW99]
    Jain, A. K., Vailaya, A., Wei, X. ‘Query by video clip’, Multimedia Systems Vol 7, 1999, pp 369–384.CrossRefGoogle Scholar
  12. [MSOO]
    E. Métais, F. Sèdes, ‘Entrepôts et documents semi-structurés: évolution textuelle’, in “Le Temps, l’Espace et l’Evolutif en Sciences du Traitement de l’Information” (H.Prade, R. Jeansoulin, C. Garbay, eds.), Cépaduès-Editions, 2000.Google Scholar
  13. [OS95]
    Ogle, V.E., Stonebraker, M., ‘Chabot: retrieval from a relational Database of Images’, Computer, 28(2), 49–56, 1995.Google Scholar
  14. [RLS98]
    Robie J., Lapp J., Schach D. ‘XML query language XQL’, QL′98, ed. Marchiori M. Scholar
  15. [SIGIROO]
    SIGIR: Working notes of the ACM Sigir Workshop on XML and Information Retrieval, July 28,2000, Athens, Greece, Editors D. Carmel, Y. Maarek, A. Soffer.Google Scholar
  16. [WKS+96]
    Wactlar, H., Kanade T., Smith, M.A., Stevens, S.M., ‘Intelligent Access to Digital Video: the Informedia Project’, Computer, 29(5), 46–52, 1996.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2003

Authors and Affiliations

  • CL. Chrisment
    • 1
  • FL. Sedes
    • 1
  1. 1.IRITUniversité Paul SabatierToulouseFrance

Personalised recommendations