Skip to main content
SpringerLink
Log in

Object-based MPEG-2 video indexing and retrieval in a collaborative environment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this paper, an object-based video retrieval methodology for search in large, heterogeneous video collections is presented. The proposed approach employs a real-time, compressed-domain, unsupervised algorithm for the segmentation of image sequences to spatiotemporal objects. For the resulting objects, MPEG-7 compliant low-level descriptors describing their color, shape, position and motion characteristics are extracted. These are automatically associated using a fuzzy C-means algorithm with appropriate intermediate-level descriptors, which are part of a simple vocabulary termed object ontology. Combined with a relevance feedback mechanism, this scheme allows the qualitative definition of the high-level concepts the user queries for (semantic objects, each represented by a keyword) and relations between them, facilitating the retrieval of relevant video segments. Furthermore, it allows the collaborative construction of a knowledge base by accumulating the information contributed to the system during feedback by different users. Thus, it enables faster and more accurate retrieval of commonly requested keywords or semantic objects. Experimental results in the context of a collaborative environment demonstrate the efficiency of the proposed video indexing and retrieval scheme.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Alberola C, Cardenes R, Martin M, Martin M, Rodriguez-Florido M, Ruiz-Alzola J (2000) diSNei: a collaborative environment for medical images analysis and visualization. In: Proc. Third Int. Conference on Medical Robotics, Imaging and Computer Assisted Surgery, Pittsburgh, Pennsylvania, pp 814–823

  2. Al-Khatib W, Day Y, Ghafoor A, Berra P (1999) Semantic modeling and knowledge representation in multimedia databases. IEEE Trans Knowl Data Eng 11(1):64–80

    Article  Google Scholar 

  3. Berlin B, Kay P (1969) Basic color terms: their universality and evolution. University of California, Berkeley

    Google Scholar 

  4. Bezdek J (1981) Pattern recognition with fuzzy objective function algorithms. Plenum, New York

    MATH  Google Scholar 

  5. Bezdek J, Keller J, Krishnapuram R, Pal N (1999) Fuzzy models and algorithms for pattern recognition and image processing. Kluwer, Norwell, Massachusetts

    MATH  Google Scholar 

  6. Bozsak E, Ehrig M, Handschuh S, Hotho A, Maedche A, Motik B, Oberle D, Schmitz C, Staab S, Stojanovic L, Stojanovic N, Studer R, Stumme G, Sure Y, Tane J, Volz R, Zacharias V (2002) KAON—towards a large scale Semantic Web. In: Proc. Third Int. Conf. on E-Commerce and Web Technologies (EC-Web 2002). Aix-en-Provence, France

  7. Chan S, Qing L, Wu Y, Zhuang Y (2002) Accommodating hybrid retrieval in a comprehensive video database management system. IEEE Trans Multimedia 4(2):146–159

    Article  Google Scholar 

  8. Chandrasekaran B, Josephso J, Benjamins V (1999) What are ontologies, and why do we need them? IEEE Intell Syst 14(1):20–26

    Article  Google Scholar 

  9. Chang S-F, Chen W, Meng H, Sundaram H, Zhong D (1998) A fully automated content-based video search engine supporting spatiotemporal queries. IEEE Trans Circuits Syst Video Technol 8(5):602–615

    Article  Google Scholar 

  10. Chang S-F, Sikora T, Puri A (2001) Overview of the MPEG-7 standard. IEEE Trans Circuits Syst Video Technol, special issue on MPEG-7 11(6):688–695

    Article  Google Scholar 

  11. Chen W, Chang S-F (2001) VISMap: an interactive image/video retrieval system using visualization and concept maps. In: Proc. IEEE Int. Conf. on Image Processing, Vol. 3, pp 588–591

  12. Day Y, Dagtas S, Iino M, Khokhar A, Ghafoor A (1995) Spatio-temporal modeling of video data for on-line object-oriented query processing. In: Proc. Int. Conf. on Multimedia Computing and Systems, pp 98–105

  13. Guo G-D, Jain A, Ma W-Y, Zhang H-J (2002) Learning similarity measure for natural image retrieval with relevance feedback. IEEE Trans Neural Netw 13(4):811–820

    Article  Google Scholar 

  14. Huijsmans D, Sebe N (2001) Extended performance graphs for cluster retrieval. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2001), Vol. 1. Kauai, Hawaii, pp 26–31

  15. Izquierdo E, Casas J, Leonardi R, Migliorati P, O’Connor N, Kompatsiaris I, Strintzis MG (2003) Advanced content-based semantic scene analysis and information retrieval: the schema project. In: Proc. Workshop on Image Analysis for Multimedia Interactive Services, London, UK

  16. Karp P, Chaudhri V, Paley S (1999) A collaborative environment for authoring large knowledge bases. Journal of Intelligent Information Systems 13(3):155–194

    Article  Google Scholar 

  17. Kim C, Hwang J-N (2002) Fast and automatic video object segmentation and tracking for content-based applications. IEEE Trans Circuits Syst Video Technol 12(2):122–129

    Article  Google Scholar 

  18. Kiranyaz S, Caglar K, Guldogan E, Guldogan O, Gabbouj M (2003) MUVIS: a content-based multimedia indexing and retrieval framework. In: Proc. Seventh Int. Symposium on Signal Processing and its Applications, ISSPA 2003, Paris, France, pp 1–8

  19. Kobla V, Doermann D, Lin K (1996) Archiving, indexing, and retrieval of video in the compressed domain. In: Proc. SPIE Conf. on Multimedia Storage and Archiving Systems, vol. 2916, pp 78–89

  20. Lammens J (1994) A computational model of color perception and color naming. Ph.D. thesis, University of Buffalo

  21. Manjunath B, Ohm J-R, Vasudevan V, Yamada A (2001) Color and texture descriptors. IEEE Trans Circuits Syst Video Technol, special issue on MPEG-7 11(6):703–715

    Article  Google Scholar 

  22. Martin P, Eklund P (2000) Knowledge retrieval and the World Wide Web. IEEE Intell Syst 15(3):18–25

    Article  Google Scholar 

  23. Mezaris V, Kompatsiaris I, Kokkinou E, Strintzis MG (2003) Real-time compressed-domain spatiotemporal video segmentation. In: Proc. Third Int. Workshop on Content-Based Multimedia Indexing (CBMI03)

  24. Mezaris V, Kompatsiaris I, Strintzis MG (2003) An Ontology Approach to Object-based Image Retrieval. In: Proc. IEEE Int. Conf. on Image Processing (ICIP03), Barcelona, Spain

  25. Mezaris V, Kompatsiaris I, Boulgouris N, Strintzis MG (2004) Real-time compressed-domain spatiotemporal segmentation and ontologies for video indexing and retrieval. IEEE Trans Circuits Syst Video Technol 14(5):606–621

    Article  Google Scholar 

  26. Mojsilovic A (2002) A method for color naming and description of color composition in images. In: Proc. IEEE Int. Conf. on Image Processing (ICIP02), New York, Rochester

  27. MPEG-2 (1996) Generic coding of moving pictures and associated audio information. Technical report, ISO/IEC 13818

  28. Naphade M, Huang T (2001) A probabilistic framework for semantic video indexing, filtering, and retrieval. IEEE Trans Multimedia 3(1):141–151

    Article  Google Scholar 

  29. Naphade M, Huang T (2002) Extracting semantics from audio-visual content: the final frontier in multimedia retrieval. IEEE Trans Neural Netw 13(4):793–810

    Article  Google Scholar 

  30. Naphade M, Yeung M, Yeo B (2000) A novel scheme for fast and efficient video sequence matching using compact signatures. In: Proc. SPIE Storage and Retrieval for Multimedia Databases, Vol. 3972, pp 564–572

  31. Naphade MR, Kozintsev I, Huang T (2002) A factor graph framework for semantic video indexing. IEEE Trans Circuits Syst Video Technol 12(1):40–52

    Article  Google Scholar 

  32. O’Connor N, Sav S, Adamek T, Mezaris V, Kompatsiaris I, Lui T, Izquierdo E, Bennstrom C, Casas J (2003) Region and object segmentation algorithms in the qimera segmentation platform. In: Proc. Third Int. Workshop on Content-Based Multimedia Indexing (CBMI03)

  33. Schreiber A, Dubbeldam B, Wielemaker J, Wielinga B (2001) Ontology-based photo annotation. IEEE Intell Syst 16(3):66–74

    Article  Google Scholar 

  34. Sikora T (2001) The MPEG-7 visual standard for content description—an overview. IEEE Trans Circuits Syst Video Technol, special issue on MPEG-7 11(6):696–702

    Article  MathSciNet  Google Scholar 

  35. TREC Video Track http://www-nlpir.nist.gov/projects/tv2004/

  36. Tsechpenakis G, Akrivas G, Andreou G, Stamou G, Kollias S (2002) Knowledge-assisted video analysis and object detection. In: Proc. European Symp. on Intelligent Technologies, Hybrid Systems and their implementation on Smart Adaptive Systems (Eunite02). Algarve, Portugal

  37. Vapnik V (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  38. Visser R, Sebe N, Lew M (2002) Detecting automobiles and people for semantic video retrieval. In: Proc. 16th Int. Conf. on Pattern Recognition, Vol. 2, pp 733–736

  39. Yoshitaka A, Ichikawa T (1999) A survey on content-based retrieval for multimedia databases. IEEE Trans Knowl Data Eng 11(1):81–93

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the EU projects SCHEMA “Network of Excellence in Content-Based Semantic Scene Analysis and Information Retrieval” (IST-2001-32795), aceMedia “Integrating knowledge, semantics and content for user centred intelligent media services” (FP6-001765), and the Hungary - Greece bilateral cooperation project CIRCE: Content-Based and Semantic Multimedia Indexing and Retrieval in Collaborative Environments. The assistance of COST211 quat and COST292 is also gratefully acknowledged.

Author information

Authors and Affiliations

  1. Information Processing Laboratory, Electrical and Computer Engineering Department, Aristotle University of Thessaloniki, Thessaloniki, 54124, Greece

    Vasileios Mezaris, Ioannis Kompatsiaris & Michael Gerassimos Strintzis

  2. Informatics and Telematics Institute (ITI)/Centre for Research and Technology Hellas (CERTH), Thessaloniki, 57001, Greece

    Vasileios Mezaris, Ioannis Kompatsiaris & Michael Gerassimos Strintzis

Authors
  1. Vasileios Mezaris
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ioannis Kompatsiaris
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Gerassimos Strintzis
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michael Gerassimos Strintzis.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mezaris, V., Kompatsiaris, I. & Strintzis, M.G. Object-based MPEG-2 video indexing and retrieval in a collaborative environment. Multimed Tools Appl 30, 255–272 (2006). https://doi.org/10.1007/s11042-006-0028-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-006-0028-0

Keywords

Search

Navigation