Abstract
Video surveillance systems are growing in size and complexity. Such systems typically consist of integrated modules of different vendors to cope with the increasing demands on network and storage capacity, intelligent video analytics, picture quality, and enhanced visual interfaces. Within a surveillance system, relevant information (like technical details on the video sequences, or analysis results of the monitored environment) is described using metadata standards. However, different modules typically use different standards, resulting in metadata interoperability problems. In this paper, we introduce the application of Semantic Web Technologies to overcome such problems. We present a semantic, layered metadata model and integrate it within a video surveillance system. Besides dealing with the metadata interoperability problem, the advantages of using Semantic Web Technologies and the inherent rule support are shown. A practical use case scenario is presented to illustrate the benefits of our novel approach.
Similar content being viewed by others
References
Annesley J, Colombo A, Orwell J, Velastin S (2007) A profile of MPEG-7 for visual surveillance. In: Proceedings of the IEEE conference on advanced video and signal based surveillance, pp 482–487
Arndt R, Troncy R, Staab S, Vacura M, Hardman L (2007) COMM: designing a well-founded multimedia ontology for the web. In: Lecture notes of computer science: the semantic web, vol 4825, pp 30–43
Bai L, Lao S, Zhang W, Jones GJF, Smeaton AF (2007) Video semantic content analysis framework based on ontology combined MPEG-7, pp 237–250
Battle S (2006) Gloze, XML to RDF and back again. In: Proceedings of first Jena user conference
Black J, Makris D, Ellis T (2005) Hierarchical database for a multi-camera surveillance system. Pattern Analysis and Applications (PAA) 7(4):430–446
Brown LM, Senior AW, Tian Y, Connell J, Hampapur A, Shu C, Merkl H, Lu M (2005) Performance evaluation of surveillance systems under varying conditions. In: Proceedings of the IEEE international workshop on performance evaluation of tracking and surveillance
Cagle K (2003) Good schema management helps to maintain xml namespace. Available on http://www.builderau.com.au/strategy/architecture/soa/Good-schema-management-helps-to-maintain-XML-namespace/0,339028264,320273067,00.htm
Clark J (1999) XSL transformations: XSLT (version 1.0). W3C recommendation, W3C
Fallside DC, Walmsley P (2004) XML schema part 0: primer, 2nd edn. W3C recommendation, W3C
Ferdinand M, Zirpins C, Trastour D (2004) Lifting XML schema to OWL. In: Proceedings web engineering—4th international conference, pp 354–358
Francois ARJ, Nevatia R, Hobbs J, Bolles R (2005) VERL: an ontology framework for representing and annotating video events. IEEE Multimed 12(4):76–78
Fuentes LM, Vlastin SA (2006) People tracking in surveillance applications. Image Vis Comput 24:1165–1171
Fullerton E (2006) Enabling video analytics, milestone white paper. Available on http://www.milestonesys.com/files/UserFiles/Docs/whitepapers/Enabling_Video_Analytics-A_Milestone_White_Paper.pdf
Garcia R, Celma O (2006) Semantic integration and retrieval of multimedia metadata. In: Proc. 5th knowledge markup and semantic annotation workshop, pp 69–80
Hittema T (2010) Societal security—videosurveillance format for interoperability. Available on http://www.iso.org/iso/isotc223_n112_isonp_video_surveillance.pdf
Horrocks I, Patel-Schneider P, Boley H, Tabet S, Grosof B, Dean M (2004) SWRL: a semantic web rule language—combining OWL and RuleML, W3C member submission, 21 May 2004. Available on http://www.w3.org/Submission/SWRL/
Hunter J (2001) Adding multimedia to the semantic web—building an MPEG-7 ontology. In: Proceedings of the first semantic web working symposium (SWWS), pp 261–281
Intelligent Systems for Security and Safety (2010) http://www.ibbt.be/en/project/isyss
Klein M (2002) Interpreting XML documents via an RDF schema ontology. In: Proceedings DEXA workshop, pp 889–894
List T, Fisher RB (2004) CVML—An XML-based computer vision markup language. In: Proceedings of the 17th international conference on pattern recognition, pp 789–792
Ma Y, Yu Q, Cohen I (2009) Target tracking with incomplete detection. Comput Vis Image Underst 113:580–587
Manola F, Miller E (2004) RDF primer. W3C recommendation, W3C
Mariano VY, Min J, Park J-H, Kasturi R, Mihalcik D, Doermann D, Drayer T (2002) Performance evaluation of object detection algorithms. In: Proceedings of the international conference on pattern recognition, pp 965–969
MPEG-7 overview. International Organization for Standardisation, Klagenfurt ISO/IEC JTC1/SC29/WG11, July 2002
Nevatia R, Hobbs J, Bolles R (2004) An ontology for video event representation. In: Proceedings of computer vision and pattern recognition workshop, pp 119–129
Open Network Video Interface Forum Core SpecifIcation (2009) Available on http://www.onvif.org/imwp/download.asp?ContentID=16154
OVReady, ObjectVideo’s interoperability program (2010) Available on http://www.objectvideo.com/programs/ovready/
Patel-Schneider P, Hayes P, Horrocks I (2004) OWL web ontology language semantics and abstract syntax, W3C recommendation, 10 February 2004. Available on http://www.w3.org/TR/2004/REC-owl-semantics-20040210/
Poppe C, De Bruyne S, Martens G, Lambert P, Van de Walle R (2008) Intelligent preprocessing for fast moving object detection. In: Proceedings of SPIE security and defense, vol 6978, p 69780S
Poppe C, De Bruyne S, Paridaens T, Lambert P, Van de Walle R (2009) Moving object detection in the H.264/AVC compressed domain for video surveillance applications. Vis Commun Image Represent 20(6):428–437
San Miguel JC, Martinez JM (2007) On the effect of motion segmentation techniques in description based adaptive video transmission. In: Proceedings of the advanced video and signal based surveillance, pp 255–261
San Miguel JC, Martínez JM, García Á (2009) An ontology for event detection and its application in surveillance video. In: Proceedings of the 2009 advanced video and signal based surveillance, pp 220–225
Senior A (2009) An introduction to automatic video surveillance, pp 1–9
Shan Y, Wang R (2006) Improved algorithms for motion detection and tracking. Opt Eng 45(6):067201
Societal Security (tc 223) (2001) More information available on http://www.iso.org/iso/standards_development/technical_committees/list_of_iso_technical-_committees/iso_technical_committee.htm?commid=295786
SPARQL query language for RDF, W3C recommendation, 15 January 2008. Available on http://www.w3.org/TR/rdf-sparql-query/
Stauffer C, Grimson WEL (2000) Learning patterns of activity using real-time tracking. IEEE Trans Pattern Anal Mach Intell 22(8):747–757. Available on citeseer.ist.psu.edu/article/stauffer00learning.html
Tian YL, Brown L, Hampapur A, Lu M, Senior A, Shu CF (2008) IBM Smart Surveillance System (S3): event based video surveillance system with an open and extensible framework. Mach Vis Appl 19:315–327
Troncy R, Bailer W, Hausenblas M, Hofmair P, Schlatte R (2006) Enabling multimedia metadata interoperability by defining formal semantics of MPEG-7 profiles. In: Lecture notes in computer science, vol 4306, pp 41–55
Van Deursen D, Poppe C, Martens G, Mannens E, Van de Walle R (2008) XML to RDF conversion: a generic approach. In: Proceedings the 4th international conference on automated solutions for cross media content and multi-channel distribution
Vezzani R, Calderara S, Piccinini P, Cucchiara R (2009) Video surveillance online repository (VISOR): an integrated framework. Multimedia Tools and Applications (MTAP) 50(2):359–380
Wijnhoven RGJ, Jaspersand EGT, de With PHN (2006) Flexible surveillance system architecture for prototyping video content analysis algorithms. In: Proc. SPIE electronic imaging, vol 6073
W3C Multimedia Semantics Incubator Group (2006) Available on http://www.w3.org/2005/Incubator/mmsem/
Xu LQ (2007) Issues in video analytics and surveillance systems: research/prototyping vs. applications/user requirements, pp 10–14
Yilmaz A, Javez O, Shah M (2006) Object tracking: a survey. ACM Comput Surv 38(4):1–45
Young DP, Ferryman JM (2005) PETS metrics: on-line performance evaluation service. In: Proceedings of visual surveillance and performance evaluation of tracking and surveillance, pp 317–324
Yu T, Zhou B, Li Q, Wang W, Chang C (2009) The design of distributed real-time video analytic system, pp 49–52
Zerzour K, Frazier G, Marir F (2000) VIGILANT: a semantic model for content and event based indexing and retrieval of surveillance video. In: Proceedings of international workshop on knowledge representation meets databases, pp 143–154
Acknowledgements
The research activities that have been described in this paper were funded by Ghent University, the Interdisciplinary Institute for Broadband Technology (IBBT), the Institute for the Promotion of Innovation by Science and Technology in Flanders (IWT-Flanders), the Fund for Scientific Research-Flanders (FWO-Flanders), and the European Union.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Poppe, C., Martens, G., De Potter, P. et al. Semantic web technologies for video surveillance metadata. Multimed Tools Appl 56, 439–467 (2012). https://doi.org/10.1007/s11042-010-0600-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-010-0600-5