Skip to main content
SpringerLink
Log in

Incorporating frequent pattern analysis into multimodal HMM event classification for baseball videos

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Data mining and frequent pattern analysis have recently become a popular way of discovering new knowledge from a data set. However, it is rarely applied to video semantic analysis. Therefore, this paper introduces two methods: frequent-pattern trained HMM and frequent-pattern tailored HMM to incorporate frequent pattern analysis into multimodal HMM event classification for baseball videos. Besides, different symbol coding methods including temporal sequence coding and co-occurrence symbol coding for multimodal HMM classification are compared. The results of our experiments on baseball video event classification demonstrate that integration of frequent pattern analysis could help to improve event classification performances.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Allen JF (1983) Maintaining knowledge about temporal intervals. ACM Commun 26(11):832–843

    Article  MATH  Google Scholar 

  2. Assari SM, Zamir AR, Shah M (2014) Video classification using semantic concept co-occurrences. In: Proc. IEEE Int. Conf. on Computer Vision and Pattern Recognition

  3. Bae TM, Kim CS, Jin SH, Kim KH, Ro YM (2005) Semantic event detection in structured video using hybrid HMM/SVM. In: Leow WK, Lew M, Chua TS, Ma WY, Chaisorn L, Bakker EM (eds.) CIVR 2005. LNCS, vol. 3568, 113–122

  4. Ballan L, Bertini M, Bimbo AD, Seidenari L, Serra G (2011) Event detection and recognition for semantic annotation of video. Multimedia Tools Appl 51(1):279–302

    Article  Google Scholar 

  5. Ballan L, Bertini M, Bimbo AD, Serra G (2010) Video event classification using string kernels. Multimedia Tools Appl 48(1):69–87

    Article  Google Scholar 

  6. Ballan L, Bertini M, Bimbo AD, Serra G (2011) Semantic annotation of soccer videos by visual instance clustering and spatial/temporal reasoning in ontologies. Multimedia Tools Appl 48(2):313–337

    Article  Google Scholar 

  7. Bossard L, Guillaumin M, Van L (2013) Event recognition in photo collections with a stopwatch HMM, In: Proc. IEEE Int. Conf. on Computer Vision. 2013, 1193–1200

  8. Bouqata B (2006) Vogue: a novel variable order-gap state machine for modeling sequences. Ph.D. Thesis. Rensselaer Polytechnic Institute

  9. Bouthemy P, Gelgon M, Ganansia F (1999) A unified approach to shot change detection and camera motion characterization. IEEE Trans Circ Syst Video Technol 9(7):1030–1044

    Article  Google Scholar 

  10. Brendel W, Fern A, Todorovic S (2011) Probabilistic event logic for interval-based event recognition. In: Proc. IEEE Int. Conf. on Computer Vision and Pattern Recognition 2011, 3329–3336

  11. Chai W, Vercoe B (2001) Folk music classification using hidden Markov models, In: Proc. of International Conference on Artificial Intelligence

  12. Chang P, Han M, Gong YH (2002) Extract highlights from baseball game video with hidden Markov models. In: Proc. IEEE Int. Conf. on Image Processing 609–612

  13. Chen M, Chen SC, Shyu ML, Wickramaratna K (2006) Semantic event detection via multimodal data mining. IEEE Signal Process Mag IEEE 23(2):38–46

    Article  Google Scholar 

  14. Chen M, Chen SC, Shyu ML (2007) Hierarchical temporal association mining for video event detection in video databases. In: Proceedings of IEEE 23rd International Conference on Data Engineering Workshop, 137–145

  15. Chen HT, Chou CL, Tsai WC, Lee SY, Lin BSP (2012) HMM-based ball hitting event exploration system for broadcast baseball video. J Vis Commun Image Represent 23(5):767–781

    Article  Google Scholar 

  16. Chen HS, Tsai WJ (2014) A framework for video event classification by modeling temporal context of multimodal features using HMM. J Vis Commun Image Represent 25(2):285–295

    Article  MathSciNet  Google Scholar 

  17. Dao MS, Babaguchi N (2010) A new spatio-temporal method for event detection and personalized retrieval of sports video. Multimedia Tools Appl 50(1):227–248

    Article  Google Scholar 

  18. Du Y, Chen F, Xu W, Qian X (2013) Video content categorization using the double decomposition. Multimedia Tools Appl 66(3):545–572

    Article  Google Scholar 

  19. Fleischman M, Decamp P, Roy D (2006) Mining temporal patterns of movement for video content classification. In: 8th ACM international workshop on Multimedia information retrieval, 183–192

  20. Fleischman M, Roy D (2007) Unsupervised content-based indexing of sports video. In: Proceedings of the international workshop on Workshop on multimedia information retrieval, 87–94

  21. Fleischman M, Roy B, Roy D (2007) Temporal feature induction for baseball highlight classification. In: 15th ACM international conference on Multimedia, 333–336

  22. Gong Y, Han M, Wei H, Xu W (2004) Maximum entropy model-based baseball highlight detection and classification. Comput Vis Image Underst 96(2):181–199

    Article  Google Scholar 

  23. Hasan T, Bořil H, Sangwan A, Hansen JHL (2013) Multi-modal highlight generation for sports videos using an information-theoretic excitability measure. EURASIP J Adv Signal Process

  24. Jiang YG, Bhattacharya S, Chang SF, Shah M (2013) High-level event recognition in unconstrained videos. Intl J Multimedia Inf Retr 2(2):73–101

    Article  Google Scholar 

  25. Karaman S, Benois-Pineau J, Dovgalecs V, Mégret R, Pinquier J, André-Obrecht R, Gaëstel Y, Jean-François D (2014) Hierarchical hidden Markov model in detecting activities of daily living in wearable videos for studies of dementia. Multimedia Tools Appl 69(3):743–771

    Article  Google Scholar 

  26. Lee H, Morariu VI, Davis LS (2015) Clauselets: leveraging temporally related actions for video event analysis. In: Proc. IEEE Winter Conference on Applications of Computer Vision (WACV)

  27. Lexing X, Sundaram H, Campbell M (2008) Event mining in multimedia streams. Proc IEEE 96(4):623–647

    Article  Google Scholar 

  28. Lien CC, Chiang CL, Lee CH (2007) Scene-based event detection for baseball videos. J Vis Commun Image Represent 18(1):1–14

    Article  Google Scholar 

  29. Liu J, Yu Q, Javed O, Ali S, Tamrakar A, Divakaran A, Cheng H, Sawhney HS (2013) Video event recognition using concept attributes. In: Proc. IEEE workshop on applications of computer vision (WACV) 339–346

  30. Merler M, Huang B, Xie L, Hua G, Natsev A (2012) Semantic model vectors for complex video event recognition. IEEE Trans Multimedia 14(1):88–101

    Article  Google Scholar 

  31. Mouret M, Solnon C, Wolf C (2009) Classification of images based on hidden Markov models. In: 7th international workshop on content-based multimedia indexing, pp 169–174

  32. Oskouie P, Alipour S, Eftekhari-Moghadam AM (2014) Multimodal feature extraction and fusion for semantic mining of soccer video: a survey. Artif Intell Rev 42(2):173–210

    Article  Google Scholar 

  33. Qian X, Wang H, Liu G, Hou X (2012) HMM based soccer video event detection using enhanced mid-level semantic. Multimedia Tools Appl 60(1):233–255

    Article  Google Scholar 

  34. Rabiner LR (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286

    Article  Google Scholar 

  35. Rehman A, Saba T (2014) Features extraction for soccer video semantic analysis: current achievements and remaining issues. Artif Intell Rev 41(3):451–461

    Article  Google Scholar 

  36. Snoek CGM, Worring M (2002) A review on multimodal video indexing. In: Proceedings of International Conference on Multimedia and Expo, 21–24

  37. Snoek CGM, Worring M (2005) Multimedia event-based video indexing using time intervals. IEEE Trans Multimedia 7(4):638–647

    Article  Google Scholar 

  38. Talha AM, Junejo IN (2014) Dynamic scene understanding using temporal association rules. Image Vis Comput 32(12):1102–1116

    Article  Google Scholar 

  39. Wang Z, Yu J, He Y, Guan T (2014) Affection arousal based highlight extraction for soccer video. Multimedia Tools Appl 73(1):519–546

    Article  Google Scholar 

  40. Wang XF, Zhang XP (2012) Ice hockey shooting event modeling with mixture hidden Markov model. Multimedia Tools Appl 57(1):131–144

    Article  Google Scholar 

  41. Wu SY, Chen YL (2007) Mining nonambiguous temporal patterns for interval-based events. IEEE Trans Knowl Data Eng 19(6):742–758

    Article  Google Scholar 

  42. Yan WQ, Kieran DF, Rafatirad S, Jain R (2011) A comprehensive study of visual event computing. Multimedia Tools Appl 55(3):443–481

    Article  Google Scholar 

  43. Zaki MJ, Carothers CD, Szymanski BK (2010) VOGUE: a variable order hidden Markov model with duration based on frequent sequence mining. ACM Trans Knowl Discov Data 4(1):1–31

    Article  Google Scholar 

  44. Zhan Y, Sun J, Niu D, Mao Q, Fan J (2014) A semi-supervised incremental learning method based on adaptive probabilistic hypergraph for video semantic detection. Multimedia Tools Appl

  45. Zhang Y, Swears E, Larios N, Wang Z, Ji Q (2013) Modeling temporal interactions with interval temporal Bayesian networks for complex activity recognition. IEEE Trans Pattern Anal Mach Intell 35(10):2468–2483

    Article  Google Scholar 

  46. Zhu XQ, Wu XD, Elmagarmid AK, Feng Z, Wu L (2005) Video data mining: semantic indexing and event detection from the association perspective. IEEE Trans Knowl Data Eng 17(5):665–677

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, National Chiao-Tung University, Taiwan, 1001 University Road, Hsinchu, Taiwan, 300, Republic of China

    Hsuan-Sheng Chen & Wen-Jiin Tsai

Authors
  1. Hsuan-Sheng Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen-Jiin Tsai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hsuan-Sheng Chen.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, HS., Tsai, WJ. Incorporating frequent pattern analysis into multimodal HMM event classification for baseball videos. Multimed Tools Appl 75, 4913–4932 (2016). https://doi.org/10.1007/s11042-015-2447-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-2447-2

Keywords

Search

Navigation