Multimedia Tools and Applications

, Volume 56, Issue 3, pp 439–467 | Cite as

Semantic web technologies for video surveillance metadata

  • Chris PoppeEmail author
  • Gaëtan Martens
  • Pieterjan De Potter
  • Rik Van de Walle


Video surveillance systems are growing in size and complexity. Such systems typically consist of integrated modules of different vendors to cope with the increasing demands on network and storage capacity, intelligent video analytics, picture quality, and enhanced visual interfaces. Within a surveillance system, relevant information (like technical details on the video sequences, or analysis results of the monitored environment) is described using metadata standards. However, different modules typically use different standards, resulting in metadata interoperability problems. In this paper, we introduce the application of Semantic Web Technologies to overcome such problems. We present a semantic, layered metadata model and integrate it within a video surveillance system. Besides dealing with the metadata interoperability problem, the advantages of using Semantic Web Technologies and the inherent rule support are shown. A practical use case scenario is presented to illustrate the benefits of our novel approach.


Video surveillance system Semantic web technologies Multimedia standards Reasoning Video analytics 



The research activities that have been described in this paper were funded by Ghent University, the Interdisciplinary Institute for Broadband Technology (IBBT), the Institute for the Promotion of Innovation by Science and Technology in Flanders (IWT-Flanders), the Fund for Scientific Research-Flanders (FWO-Flanders), and the European Union.


  1. 1.
    Annesley J, Colombo A, Orwell J, Velastin S (2007) A profile of MPEG-7 for visual surveillance. In: Proceedings of the IEEE conference on advanced video and signal based surveillance, pp 482–487Google Scholar
  2. 2.
    Arndt R, Troncy R, Staab S, Vacura M, Hardman L (2007) COMM: designing a well-founded multimedia ontology for the web. In: Lecture notes of computer science: the semantic web, vol 4825, pp 30–43Google Scholar
  3. 3.
    Bai L, Lao S, Zhang W, Jones GJF, Smeaton AF (2007) Video semantic content analysis framework based on ontology combined MPEG-7, pp 237–250Google Scholar
  4. 4.
    Battle S (2006) Gloze, XML to RDF and back again. In: Proceedings of first Jena user conferenceGoogle Scholar
  5. 5.
    Black J, Makris D, Ellis T (2005) Hierarchical database for a multi-camera surveillance system. Pattern Analysis and Applications (PAA) 7(4):430–446MathSciNetCrossRefGoogle Scholar
  6. 6.
    Brown LM, Senior AW, Tian Y, Connell J, Hampapur A, Shu C, Merkl H, Lu M (2005) Performance evaluation of surveillance systems under varying conditions. In: Proceedings of the IEEE international workshop on performance evaluation of tracking and surveillanceGoogle Scholar
  7. 7.
  8. 8.
    Clark J (1999) XSL transformations: XSLT (version 1.0). W3C recommendation, W3CGoogle Scholar
  9. 9.
    Fallside DC, Walmsley P (2004) XML schema part 0: primer, 2nd edn. W3C recommendation, W3CGoogle Scholar
  10. 10.
    Ferdinand M, Zirpins C, Trastour D (2004) Lifting XML schema to OWL. In: Proceedings web engineering—4th international conference, pp 354–358Google Scholar
  11. 11.
    Francois ARJ, Nevatia R, Hobbs J, Bolles R (2005) VERL: an ontology framework for representing and annotating video events. IEEE Multimed 12(4):76–78CrossRefGoogle Scholar
  12. 12.
    Fuentes LM, Vlastin SA (2006) People tracking in surveillance applications. Image Vis Comput 24:1165–1171CrossRefGoogle Scholar
  13. 13.
    Fullerton E (2006) Enabling video analytics, milestone white paper. Available on
  14. 14.
    Garcia R, Celma O (2006) Semantic integration and retrieval of multimedia metadata. In: Proc. 5th knowledge markup and semantic annotation workshop, pp 69–80Google Scholar
  15. 15.
    Hittema T (2010) Societal security—videosurveillance format for interoperability. Available on
  16. 16.
    Horrocks I, Patel-Schneider P, Boley H, Tabet S, Grosof B, Dean M (2004) SWRL: a semantic web rule language—combining OWL and RuleML, W3C member submission, 21 May 2004. Available on
  17. 17.
    Hunter J (2001) Adding multimedia to the semantic web—building an MPEG-7 ontology. In: Proceedings of the first semantic web working symposium (SWWS), pp 261–281Google Scholar
  18. 18.
    Intelligent Systems for Security and Safety (2010)
  19. 19.
    Klein M (2002) Interpreting XML documents via an RDF schema ontology. In: Proceedings DEXA workshop, pp 889–894Google Scholar
  20. 20.
    List T, Fisher RB (2004) CVML—An XML-based computer vision markup language. In: Proceedings of the 17th international conference on pattern recognition, pp 789–792Google Scholar
  21. 21.
    Ma Y, Yu Q, Cohen I (2009) Target tracking with incomplete detection. Comput Vis Image Underst 113:580–587CrossRefGoogle Scholar
  22. 22.
    Manola F, Miller E (2004) RDF primer. W3C recommendation, W3CGoogle Scholar
  23. 23.
    Mariano VY, Min J, Park J-H, Kasturi R, Mihalcik D, Doermann D, Drayer T (2002) Performance evaluation of object detection algorithms. In: Proceedings of the international conference on pattern recognition, pp 965–969Google Scholar
  24. 24.
    MPEG-7 overview. International Organization for Standardisation, Klagenfurt ISO/IEC JTC1/SC29/WG11, July 2002Google Scholar
  25. 25.
    Nevatia R, Hobbs J, Bolles R (2004) An ontology for video event representation. In: Proceedings of computer vision and pattern recognition workshop, pp 119–129Google Scholar
  26. 26.
    Open Network Video Interface Forum Core SpecifIcation (2009) Available on
  27. 27.
    OVReady, ObjectVideo’s interoperability program (2010) Available on
  28. 28.
    Patel-Schneider P, Hayes P, Horrocks I (2004) OWL web ontology language semantics and abstract syntax, W3C recommendation, 10 February 2004. Available on
  29. 29.
    Poppe C, De Bruyne S, Martens G, Lambert P, Van de Walle R (2008) Intelligent preprocessing for fast moving object detection. In: Proceedings of SPIE security and defense, vol 6978, p 69780SGoogle Scholar
  30. 30.
    Poppe C, De Bruyne S, Paridaens T, Lambert P, Van de Walle R (2009) Moving object detection in the H.264/AVC compressed domain for video surveillance applications. Vis Commun Image Represent 20(6):428–437CrossRefGoogle Scholar
  31. 31.
    San Miguel JC, Martinez JM (2007) On the effect of motion segmentation techniques in description based adaptive video transmission. In: Proceedings of the advanced video and signal based surveillance, pp 255–261Google Scholar
  32. 32.
    San Miguel JC, Martínez JM, García Á (2009) An ontology for event detection and its application in surveillance video. In: Proceedings of the 2009 advanced video and signal based surveillance, pp 220–225Google Scholar
  33. 33.
    Senior A (2009) An introduction to automatic video surveillance, pp 1–9Google Scholar
  34. 34.
    Shan Y, Wang R (2006) Improved algorithms for motion detection and tracking. Opt Eng 45(6):067201CrossRefGoogle Scholar
  35. 35.
  36. 36.
    SPARQL query language for RDF, W3C recommendation, 15 January 2008. Available on
  37. 37.
    Stauffer C, Grimson WEL (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22(8):747–757. Available on Google Scholar
  38. 38.
    Tian YL, Brown L, Hampapur A, Lu M, Senior A, Shu CF (2008) IBM Smart Surveillance System (S3): event based video surveillance system with an open and extensible framework. Mach Vis Appl 19:315–327zbMATHCrossRefGoogle Scholar
  39. 39.
    Troncy R, Bailer W, Hausenblas M, Hofmair P, Schlatte R (2006) Enabling multimedia metadata interoperability by defining formal semantics of MPEG-7 profiles. In: Lecture notes in computer science, vol 4306, pp 41–55Google Scholar
  40. 40.
    Van Deursen D, Poppe C, Martens G, Mannens E, Van de Walle R (2008) XML to RDF conversion: a generic approach. In: Proceedings the 4th international conference on automated solutions for cross media content and multi-channel distributionGoogle Scholar
  41. 41.
    Vezzani R, Calderara S, Piccinini P, Cucchiara R (2009) Video surveillance online repository (VISOR): an integrated framework. Multimedia Tools and Applications (MTAP) 50(2):359–380CrossRefGoogle Scholar
  42. 42.
    Wijnhoven RGJ, Jaspersand EGT, de With PHN (2006) Flexible surveillance system architecture for prototyping video content analysis algorithms. In: Proc. SPIE electronic imaging, vol 6073Google Scholar
  43. 43.
    W3C Multimedia Semantics Incubator Group (2006) Available on
  44. 44.
    Xu LQ (2007) Issues in video analytics and surveillance systems: research/prototyping vs. applications/user requirements, pp 10–14Google Scholar
  45. 45.
    Yilmaz A, Javez O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38(4):1–45CrossRefGoogle Scholar
  46. 46.
    Young DP, Ferryman JM (2005) PETS metrics: on-line performance evaluation service. In: Proceedings of visual surveillance and performance evaluation of tracking and surveillance, pp 317–324Google Scholar
  47. 47.
    Yu T, Zhou B, Li Q, Wang W, Chang C (2009) The design of distributed real-time video analytic system, pp 49–52Google Scholar
  48. 48.
    Zerzour K, Frazier G, Marir F (2000) VIGILANT: a semantic model for content and event based indexing and retrieval of surveillance video. In: Proceedings of international workshop on knowledge representation meets databases, pp 143–154Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2010

Authors and Affiliations

  • Chris Poppe
    • 1
    Email author
  • Gaëtan Martens
    • 1
  • Pieterjan De Potter
    • 1
  • Rik Van de Walle
    • 1
  1. 1.Department of Electronics and Information Systems—Multimedia LabGhent University—IBBTLedeberg-GhentBelgium

Personalised recommendations