Multimedia Tools and Applications

, Volume 62, Issue 3, pp 847–877 | Cite as

A design-of-experiment based statistical technique for detection of key-frames

  • Snehasis MukherjeeEmail author
  • Dipti Prasad Mukherjee


In this paper decision variables for the key-frame detection problem in a video are evaluated using statistical tools derived from the theory of design of experiments. The pixel-by-pixel intensity difference of consecutive video frames is used as the factor or decision variable for designing an experiment for key-frame detection. The determination of a key-frame is correlated with the different values of the factor. A novel concept of meaningfulness of a video key-frame is also introduced to select the representative key-frame from a set of possible key-frames. The use of the concepts of design of experiments and the meaningfulness property to summarize a video is tested using a number of videos taken from MUSCLE-VCD-2007 dataset. The performance of the proposed approach in detecting key-frames is found to be superior in comparison to the competing approaches like PME based method (Liu et al., IEEE Trans Circuits Syst Video Technol 13(10):1006–1013, 2003; Mukherjee et al., IEEE Trans Circuits Syst Video Technol 17(5):612–620, 2007; Panagiotakis et al., IEEE Trans Circuits Syst Video Technol 19(3):447–451, 2009).


Key-frame Video summarization Design of experiment Helmholtz principle Meaningfulness Gestalt 


  1. 1.
    Adjeroh D, Lee MC, Banda N, Kandaswamy U (2009) Adaptive edge-oriented shot boundary detection. J Image Video Process 2009(5):5:1–5:13CrossRefGoogle Scholar
  2. 2.
    Calic J, Izquierdo E (2002) Efficient key-frame extraction and video analysis. In: Proc. IEEE international conference on information technology: coding and computing, Washington, DC, USA, pp 28–33Google Scholar
  3. 3.
    Chasanis VT, Likas AC, Galatsanos NP (2009) Scene detection in videos using shot clustering and sequence alignment. IEEE Trans Multimedia 11(1):89–100CrossRefGoogle Scholar
  4. 4.
    Desolneux A, Moisan L, Morel J (2003) A grouping principle and four applications. IEEE Trans Pattern Anal Mach Intell 25(4):508–513CrossRefGoogle Scholar
  5. 5.
    Desolneux A, Moisan L, Morel J (2008) From gestalt theory to image analysis: a probabilistic approach. In: Interdisciplinary applied mathematics, vol 34. Springer, New YorkGoogle Scholar
  6. 6.
    Gao Y, Tang J, Xie X (2009) Key frame vector and its application to shot retrieval. In: Proc. 1st international workshop on interactive multimedia for consumer electronics, Beijing, China, pp 27–34Google Scholar
  7. 7.
    Hoeffding W (1963) Probability inequalities for sum of bounded random variables. J Am Stat Assoc 58(301):13–30MathSciNetzbMATHCrossRefGoogle Scholar
  8. 8.
    Law-To J, Joly A, Boujemaa N (2007) Muscle-VCD-2007: a live benchmark for video copy detection. Accessed May 2010
  9. 9.
    Lienhart R (2001) Reliable transition detection in videos: a survey and practitioner’s guide. Int J Image Graph 1(3):469–486CrossRefGoogle Scholar
  10. 10.
    Liu TM, Zhang HJ, Qi FH (2003) A novel key-frame extraction algorithm based on perceived motion energy model. IEEE Trans Circuits Syst Video Technol 13(10):1006–1013CrossRefGoogle Scholar
  11. 11.
    Mills M (1992) A magnifier tool for video data. In: Proc. ACM conference on human factors in computing systems, Monterey, California, USA, pp 93–98Google Scholar
  12. 12.
    Mukherjee DP, Das SK, Saha S (2007) Key-frame estimation in video using randomness measure of feature point pattern. IEEE Trans Circuits Syst Video Technol 17(5):612–620CrossRefGoogle Scholar
  13. 13.
    Ouyang J, Li J, Tang H (2006) Interactive key frame selection model. J Vis Commun Image Represent 17(6):1145–1163CrossRefGoogle Scholar
  14. 14.
    Panagiotakis C, Doulamis A, Tziritas G (2009) Equivalent key frames selection based on iso-content principles. IEEE Trans Circuits Syst Video Technol 19(3):447–451CrossRefGoogle Scholar
  15. 15.
    Park SH (1996) Robust design and analysis for quality engineering. Chapman & Hall, LondonGoogle Scholar
  16. 16.
    Pickering MJ, Rüger SM, Sinclair D (2002) Video retrieval by feature learning in key frames. In: Proc. International Conference on Image and Video Retrieval, pp 309–317Google Scholar
  17. 17.
    Pye D, Hollinghurst NJ, Mills TJ, Wood KR (1998) Audio-visual segmentation for content-based retrieval. In: Proc. international conference on spoken language processingGoogle Scholar
  18. 18.
    Rasheed Z, Shah M (2005) Detection and Representation of scenes in videos. IEEE Trans Multimedia 7(6):1097–1105CrossRefGoogle Scholar
  19. 19.
    Richard GL (2007) Statistical concepts: a second course. Lawrence Erlbaum Associates, MahwahGoogle Scholar
  20. 20.
    Roy RK (2001) Design of experiments using the Taguchi approach. Wile, New YorkGoogle Scholar
  21. 21.
    Smeaton AF, Over P, Doherty AR (2010) Video shot boundary detection: seven years of TRECVid activity. Comput Vis Image Underst 114(4):411–418CrossRefGoogle Scholar
  22. 22.
    Song X, Fan G (2005) Joint key-frame extraction and object-based video segmentation. In: Proc. IEEE workshop on motion and video computing, vol 2. Breckenridge, Colorado, pp 126–131Google Scholar
  23. 23.
    Spyrou E, Tolias G, Mylonas P, Avrithis Y (2009) Concept detection and keyframe extraction using a visual thesaurus. Multimedia Tools Appl 41(3):337–373CrossRefGoogle Scholar
  24. 24.
    Valdes V, Martinez JM (2010) A framework for video abstraction systems analysis and modelling from an operational point of view. Multimedia Tools Appl 49(1):7–35CrossRefGoogle Scholar
  25. 25.
    Wolf W (1996) Key frame selection by motion analysis. In: Proc. IEEE international conference on acoustics, speech and signal processing, vol 2, Washington, DC, USA pp 1228–1231Google Scholar
  26. 26.
    Yeung MM, Yeo BL (1997) Video visualization for compact presentation and fast browsing of pictorial content. IEEE Trans Circuits Syst Video Technol 7(5):771–785CrossRefGoogle Scholar
  27. 27.
    Zhuang Y, Rui Y, Huang TS, Mehrotra S (1998) Adaptive key frame extraction using unsupervised clustering. In: Proc. IEEE international conference on image processing, Chicago, USA, pp 866–870Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  1. 1.Electronics and Communication Sciences UnitIndian Statistical InstituteKolkataIndia

Personalised recommendations