Multimedia Tools and Applications

, Volume 47, Issue 3, pp 545–579 | Cite as

Semantic concept mining in cricket videos for automated highlight generation

  • Maheshkumar H. Kolekar
  • Somnath Sengupta


This paper presents a novel approach towards automated highlight generation of broadcast sports video sequences from its extracted events and semantic concepts. A sports video is hierarchically divided into temporal partitions namely, megaslots, slots, and semantic entities, namely concepts, and events. The proposed method extracts event sequence from video and classifies each sequence into a concept by sequential association mining. The extracted concepts and events within the concepts are selected according to their degree of importance to include those in the highlights. A parameter degree of abstraction is proposed, which gives a choice to the user about how concisely the extracted concepts should be produced for a specified highlight duration. We have successfully extracted highlights from recorded video of cricket match and compared our results with the manually-generated highlights by sports television channel.


Event detection Association rule Semantic concept Cricket Sports video highlights 


  1. 1.
    Aigrain P, Zhang H, Petkovic D (1996) Representation and retrieval of visual media: a state-of-the-art review. Int J Multimedia Tools Appl 3:179–182CrossRefGoogle Scholar
  2. 2.
    Assfalg J, Bertini M, Colombo C, Bimbo AD (2002) Semantic annotation of sports videos. IEEE Multimedia 9(2):52–60CrossRefGoogle Scholar
  3. 3.
    Babaguchi N, Kawai Y, Ogura T, Kitahashi T (2004) Personalized abstraction of broadcasted american football video by highlight selection. IEEE Trans Multimedia 6(4):107–109CrossRefGoogle Scholar
  4. 4.
    Baillie M, Jose JM (2003) Audio-based event detection for sports video. In: Lecture notes on computer science, vol 2728, pp 61–66Google Scholar
  5. 5.
    Bao P, Zhang L, Wu X (2005) Canny edge detection enhancement by scale multiplication. In: IEEE trans. on pattern recognition and machine intelligence, vol 27, pp 1485–1490Google Scholar
  6. 6.
    Baoxin L, Pan H, Sezan I (2003) A general framework for sports video summarization with its application to soccer. In: Proc. of int conf on acoustics, speech and signal processing, vol 3, no 169–172Google Scholar
  7. 7.
    Bertini M, Cucchiara R, Bimbo AD, Prati A (2005) An integrated framework for semantic annotation and adaptation. Int J Multimed Tools Appl 26:345–363CrossRefGoogle Scholar
  8. 8.
    Cheng C, Hsu C (2006) Fusion of audio and motion information on HMM-based highlight extraction for baseball games. IEEE Trans Multimedia 8(3):585–599CrossRefGoogle Scholar
  9. 9.
    Chang P, Han M, Gong Y (2002) Extract highlights from baseball game video with hidden markov models. Proc Int Conf Image Proc 1:609–612Google Scholar
  10. 10.
    Christel M, Stevens S, Kanade T, Mauldin M, Reddy R, Wactlar H (1995) Techniques for the creation and exploration of digital video libraries. Multimed Tools Appl 2Google Scholar
  11. 11.
    Dimitrova N, Zhang HJ, Shahraray B, Sezan I, Huang T, Zakhor A (2002) Applications of video-content analysis and retrieval. IEEE Multimedia 9(3):42–55CrossRefGoogle Scholar
  12. 12.
    Duan L, Xu M, Tian Q, Xu C, Jin J (2005) A unified framework for semantic shot classification in sports video. IEEE Trans Multimedia 7(6):1066–1083CrossRefGoogle Scholar
  13. 13.
    Ekin A, Tekalp AM, Mehrotra R (2003) Automatic soccer video analysis and summarization. IEEE Trans Image Process 12(7):796–807CrossRefGoogle Scholar
  14. 14.
    Gauch JM, Shivadas A (2005) Identification of new commercials using repeated video sequence detection. IEEE Int Conf Image Proc 3:1252–1255Google Scholar
  15. 15.
    Hanjalic A (2005) Adaptive extraction of highlights from a sport video based on excitement modeling. IEEE Trans Multimedia 7(6):1114–1122CrossRefGoogle Scholar
  16. 16.
    Hua W, Han M, Gong Y (2002) Baseball scene classification using multimedia features. In: Proc of IEEE int. conf. on multimedia and expo, vol 1, pp 821–824Google Scholar
  17. 17.
    Hua XS, Lu L, Zhang HJ (2005) Robust learning-based TV commercial detection. In: IEEE int. conf. multimedia and expo, pp 149–152Google Scholar
  18. 18.
    Huang J, Liu Z, Wang Y (2005) Joint scene classification and segmentation based on hidden markov model. IEEE Trans Multimedia 7(3):538–550CrossRefGoogle Scholar
  19. 19.
    Hauptmann AG, Smith M (1995) Text, speech and vision for video segmentation: the informedia project. writing notes of ijcai workshop on intelligent multimedia information retrieval, pp 17–22Google Scholar
  20. 20.
    Kijak E, Gravier G, Gros P, Oisel L, Bimbot F (2003) HMM based structuring of tennis videos using visual and audio cue. In: Proc. of int. conf. on multimedia and expo, vol 3, pp 309–312Google Scholar
  21. 21.
    Kokaram A, Rea N, Dahyot R, Tekalp M, Bouthemy P, Gros P, Sezan I (2006) Browsing sports video: trends in sports-related indexing and retrieval work. IEEE Signal Process Mag 23(2):47–58CrossRefGoogle Scholar
  22. 22.
    Kolekar MH, Sengupta S (2004) Hidden markov model based video indexing with discrete cosine transform as a likelihood function. In: IEEE INDICON conference, IIT Kharagpur, India, pp 157–159Google Scholar
  23. 23.
    Kolekar MH, Sengupta S (2005) Semantic indexing of news video sequences: a multimodal hierarchical approach based on hidden markov model. In: Proc of IEEE int. region 10 conference (TENCON), Melbourne, pp 1–5Google Scholar
  24. 24.
    Kolekar MH, Sengupta S (2006) A hierarchical framework for generic sports video classification. In: Lecture notes on computer science, vol 3852. Springer, Berlin, pp 633–642Google Scholar
  25. 25.
    Kolekar MH, Sengupta S (2006) Event-importance based customized and automatic cricket highlight generation. In: IEEE int. conf. on multimedia and expo, pp 1617–1620Google Scholar
  26. 26.
    Kolekar MH, Sengupta S (2006) Semantic concept extraction from sports video for highlight generation. In: Proc. of ACM int. conf. on mobile multimedia communication, vol 324Google Scholar
  27. 27.
    Kolekar MH, Talbar SN, Sontakke TR (2000) Texture segmentation using fractal signature. IETE J Research 46(5):319–323Google Scholar
  28. 28.
    Leonardi R, Migliorati P, Prandini M (2004) Semantic indexing of soccer audio-visual sequences: a multimodal approach based on controlled markov chains. IEEE Trans Circuits Syst Video Technol 14(5)Google Scholar
  29. 29.
    Li B, Sezan MI (2003) Semantic sports video analysis: approaches and new applications. Proc IEEE Int Conf Image Proc 1:17–20Google Scholar
  30. 30.
    Mei T, Ma YF, Zhou HQ, Ma WY, Zhang HJ (2005) Sports video mining with mosaic. In: IEEE—Multimedia Modeling Conference, pp 107–114Google Scholar
  31. 31.
    Naphade MR, Smith JR (2004) On the detection of semantic concepts at trecvid. In: 12th annual ACM int. conf. on multimedia, pp 660–667Google Scholar
  32. 32.
    Otsuka I, Nakane K, Divakaran A, Hatanaka K, Ogawa M (2005) A highlight scene detection and video summarization system using audio feature for a personal video recorder. IEEE Trans Consum Electron 51(1):112–116CrossRefGoogle Scholar
  33. 33.
    Peker K, Cabasson R, Divakaran A (2002) Rapid generation of sports video highlights using the mpeg-7 motion activity descriptor. In: Proc SPIE storage and retrieval for media databases, vol 4676, pp 318–323Google Scholar
  34. 34.
    Rui Y, Gupta A, Acero A (2000) Automatically extracting highlights for tv baseball programs. In: Proc. ACM multimedia, pp 105–115Google Scholar
  35. 35.
    Sankar KP, Pandey S, Jawahar CV (2006) Text driven temporal segmentation of cricket videos. Int Conf Pattern Recognit 4338:433–444Google Scholar
  36. 36.
    Takahashi Y, Nitta N, Babaguchi N (2005) Video summarization for large sports video archives. In: Proc IEEE int. conf. multimedia and expo, pp 1170–1173Google Scholar
  37. 37.
    Utsumi O, Miura K, Ide I, Sakai S, Tanaka H (2002) An object detection method for describing soccer games from video. In: Proc of IEEE int. conf. on multimedia and expo, vol 1, pp 45–48Google Scholar
  38. 38.
    Wan K, Xu C (2004) Efficient multimodal features for automatic soccer highlight generation. Int Conf Pattern Recognit 3:973–976Google Scholar
  39. 39.
    Wang J, Chng E, Xu C, Hanqinq L, Tian Q (2007) Generation of personalized music sports video using multimodal cues. IEEE Trans Multimedia 9(3):576–588CrossRefGoogle Scholar
  40. 40.
    Xiong Z, Radhakrishnan R, Divakaran A, Huang TS (2003) Audio events detection based highlights extraction from baseball, golf, soccer games in a unified framework. In: Proc. int. conf. on acoustics, speech and signal processing, vol 5, pp 632–635Google Scholar
  41. 41.
    Xu H, Chau T (2004) The fusion of audio-visual features and external knowledge for event detection in team sports video. In: ACM SIGMM int. multimedia workshop on multimedia information retrieval, pp 127–134Google Scholar
  42. 42.
    Xu P, Xie L, Chang S, Divakaran A, Vetro A, Sun H (2001) Algorithms and system for segmentation and structure analysis in soccer video. In: IEEE int. conf. on multimedia and expoGoogle Scholar
  43. 43.
    Zhang Z, Masseglia F, Jain R, Bimbo AD (2008) Editorial: introduction to the special issue on multimedia data mining. IEEE Trans Multimedia 10(2):165–166CrossRefGoogle Scholar
  44. 44.
    Zhou W, Vellaikal A, Kuo CCJ (2000) Rule-based video classification system for basketball video indexing. In: Proc. ACM workshop on multimedia, pp 213–216Google Scholar
  45. 45.
    Zhu X, Wu X, Elmagarmid AK, Feng Z, Wu L (2005) Video data mining: semantic indexing and event detection from the association perspective. IEEE Trans Knowl Data Eng 17(5):665–677CrossRefGoogle Scholar
  46. 46.
    Zhu G, Huang Q, Xu C, Xing L, Gao W, Yao H (2007) Human behavior analysis for highlight ranking in broadcast racket sports video. IEEE Trans Multimedia 9(6):1167–1182CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of MissouriColumbiaUSA
  2. 2.Electronics and Electrical Communication EnggIndian Institute of TechnologyKharagpurIndia

Personalised recommendations