Multimedia Tools and Applications

, Volume 50, Issue 1, pp 227–248 | Cite as

A new spatio-temporal method for event detection and personalized retrieval of sports video



In this paper, a new spatio-temporal method for adaptively detecting events based on Allen temporal algebra and external information support is presented. The temporal information is captured by presenting events as the temporal sequences using a lexicon of non-ambiguous temporal patterns. These sequences are then exploited to mine undiscovered sequences with external text information supports by using class associate rules mining technique. By modeling each pattern with linguistic part and perceptual part those work independently and connect together via transformer, it is easy to deploy this method to any new domain (e.g baseball, basketball, tennis, etc.) with a few changes in perceptual part and transformer. Thus the proposed method not only can work well in unwell structured environments but also can be able to adapt itself to new domains without the need (or with a few modification) for external re-programming, re-configuring and re-adjusting. Results of automatic event detection progress are tailored to personalized retrieval via click-and-see style using either conceptual or conceptual-visual query scheme. Experimental results carried on more than 30 hours of soccer video corpus captured at different broadcasters and conditions as well as compared with well-known related methods, demonstrated the efficiency, effectiveness, and robustness of the proposed method in both offline and online processes.


Event detection Data-mining Temporal sequential patterns Web-casting text Personalized retrieval 


  1. 1.
    Allen J (1983) Maintaining knowledge about temporal intervals. Commun ACM 26(11):832–843MATHCrossRefGoogle Scholar
  2. 2.
    Calic J, Campbell N, Dasiopoulou S, Kompatsiaris Y (2005) An overview of multimodal video representation for semantic analysis. In: European workshop on the integration of knowledge, semantics and digital media technologies, conference proceedings, pp 39–45Google Scholar
  3. 3.
    Chen M, Chen S, Shyu M, Wickramaratna K (2006) Semantic event detection via multimodal data mining. IEEE Signal Process Mag 23(2):38–46CrossRefGoogle Scholar
  4. 4.
    Chen M, Chen S, Shyu M (2007) Hierarchical temporal association mining for video event detection in video databases. In: MDDM’06 conference proceedings. IEEE, Piscataway, pp 137–145Google Scholar
  5. 5.
    Duan L, Xu M, Tian Q, Xu C, Jin J (2005) A unified framework for semantic shot representation of sports video. IEEE Trans Multimedia 7(6):1066–1083CrossRefGoogle Scholar
  6. 6.
    Ekin A, Tekalp A, Mehrotta R (2003) Automatic soccer video analysis and summarization. IEEE Trans Image Process 12(7):796–807CrossRefGoogle Scholar
  7. 7.
    Fleischman M, Roy D (2007) Unsupervised content-based indexing of sports video. In: MIR’07 conference proceedings. ACM, New York, pp 87–94Google Scholar
  8. 8.
    Fleischman M, Decamp P, Roy D (2006) Mining temporal patterns of movement for video content classification. In: MIR’06 conference proceedings. ACM, New York, pp 183–191Google Scholar
  9. 9.
    Jiang S, Huang Q, Gao W (2007) Mining information of attack-defense status from soccer video based on scene analysis. In: ICME07 conference proceedings. IEEE, Piscataway, pp 1095–1098Google Scholar
  10. 10.
    Liu Y, Jiang S, Ye Q, Gao W, Huang Q (2005) Playfield detection using adaptive gmm and its application. In: ICASSP’05 conference proceedings. ACM, New York, pp 421–424Google Scholar
  11. 11.
    Missaoui R, Palenichka R (2005) Effective image and video mining: an overview of model-based approaches. In: MDM’05 conference proceedings. ACM, New York, pp 43–52Google Scholar
  12. 12.
    Pei J, Han J, Mortazavi-Asl B, Pinto H, Chen Q, Dayal U, Hsu M (2004) Mining sequential patterns by pattern-growth: the prefixspan approach. IEEE Trans Knowl Data Eng 16(11):1424–1440CrossRefGoogle Scholar
  13. 13.
    Sadlier D, O’Connor N (2005) Event detection in field sports video using audio-visual features and a support vector machine. IEEE Trans Circuits Syst Video Technol 15(10):1225–1233CrossRefGoogle Scholar
  14. 14.
    Sebe N, Tian Q (2007) Personalized multimedia retrieval: the new trend? In: ACM MIR 07 conference proceedings. ACM, New York, pp 229–306Google Scholar
  15. 15.
    Snoek C, Worring M (2005) Multimedia event-based video indexing using time intervals. IEEE Trans Multimedia 7(4):638–647CrossRefGoogle Scholar
  16. 16.
    Snoek C, Worring M (2005) Multimodal video indexing: a review of the state-of-the-art. J Multimedia Tools Appl 35(5):5–34CrossRefGoogle Scholar
  17. 17.
    Tong X, Lu H, Liu Q, Jian H (2004) Replay detection in broadcasting sports video. In: MIR’05 conference proceedings. ACM, New York, pp 337–340Google Scholar
  18. 18.
    Wang F, Sun L, Yang B, Yang S (2006) Fast arc detection algorithm for play field registration in soccer video mining. In: Systems, man, and cybernetics conference proceedings. IEEE, Piscataway, pp 4932–4936Google Scholar
  19. 19.
    Wu S, Chen Y (2007) Mining nonambiguous temporal patterns for interval-based events. IEEE Trans Knowl Data Eng 19(6):742–758CrossRefGoogle Scholar
  20. 20.
    Xiong Z, Zhou X, Tian Q, Rui Y, Huang T (2006) Semantic retrieval of video. IEEE Signal Process Mag 23(2):18–27CrossRefGoogle Scholar
  21. 21.
    Xu C, Wang J, Kwan K, Li Y, Duan L (2006) Live sports event detection based on broadcast video and web-casting text. In: MM’06 conference proceedings. ACM, New York, pp 221–230Google Scholar
  22. 22.
    Zhao Q, Bhowmick S (2003) Sequential pattern mining: a survey. ITechnical report, CAIS Nayang Technological University, Singapore, pp 1–26Google Scholar
  23. 23.
    Zhu X, Wu X, Elmagarmid A, Feng Z, Wu L (2005) Video data mining: semantic indexing and event detection from the association perspective. IEEE Trans Knowl Data Eng 7(5):665–677Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Media Integrated Communication Lab. (MICL), Department of Information and Communications Technology, Division of Electrical, Electronic and Information Engineering, Graduate School of EngineeringOsaka UniversityOsakaJapan

Personalised recommendations