Skip to main content
SpringerLink
Log in

Dynamic video summarization using two-level redundancy detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The mushroom growth of video information, consequently, necessitates the progress of content-based video analysis techniques. Video summarization, aiming to provide a short video summary of the original video document, has drawn much attention these years. In this paper, we propose an algorithm for video summarization with a two-level redundancy detection procedure. By video segmentation and cast indexing, the algorithm first constructs story boards to let users know main scenes and cast (when this is a video with cast) in the video. Then it removes redundant video content using hierarchical agglomerative clustering in the key frame level. The impact factors of scenes and key frames are defined, and parts of key frames are selected to generate the initial video summary. Finally, a repetitive frame segment detection procedure is designed to remove redundant information in the initial video summary. Results of experimental applications on TV series, movies and cartoons are given to illustrate the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Benini S, Bianchetti A, Leonardi R, Migliorati P (2006) Extraction of significant video summaries by dendrogram analysis. In: Proceeding of IEEE international conference on image processing. IEEE, Piscataway, pp 133–136

    Google Scholar 

  2. Calic J, Gibson D, Campbell N (2007) Efficient layout of comic-like video summaries. IEEE Trans Circuits Syst Video Technol 17(7):931–936

    Article  Google Scholar 

  3. Cernekova Z, Nikou C (2006) Information theory-based shot cut/fade detection and video summarization. IEEE Trans Circuits Syst Video Technol 16(1):82–91

    Article  Google Scholar 

  4. Cheng W, Xu D (2003) An approach to generating two-level video abstraction. In: Proceeding of international conference on machine learning and cybernetics, vol 5. IEEE, Piscataway, pp 2896–2900

    Google Scholar 

  5. Ciocca G, Schettini R (2006) Supervised and unsupervised classification post-processing for visual video summaries. IEEE Trans Consum Electron 52(2):630–638

    Article  Google Scholar 

  6. Ferman A, Tekalp A (2003) Two-stage hierarchical video summary extraction to match low-level user browsing preferences. IEEE Trans Multimedia 5(2):244–256

    Article  Google Scholar 

  7. Gao Y, Wang T, Li J (2007) Cast indexing for videos by ncuts and page ranking. In: Proceeding of ACM international conference on image and video retrieval. ACM, New York, pp 441–447

    Google Scholar 

  8. Gong Y, Liu X (2001) Video summarization with minimal visual content redundancies. In: Proceeding of IEEE international conference on image processing. IEEE, Piscataway, pp 362–265

    Google Scholar 

  9. Jain A, Vailaya A, Wei X (1999) Query by video clip. Multimedia Syst 7(5):369–384

    Article  Google Scholar 

  10. Kim S, Park R (2002) An efficient algorithm for video sequence matching using the modified hausdorff distance and the directed divergence. IEEE Trans Circuits Syst Video Technol 12(7):592–596

    Article  Google Scholar 

  11. Koprinska I, Carrato S (2001) Temporal video segmentation: a survey. Signal Process Image Commun 16(5):477–500

    Article  Google Scholar 

  12. Lee J, Oh J, Hwang S (2005) Scenario based dynamic video abstractions using graph matching. In: Proceeding of ACM international conference on multimedia. ACM, New York, pp 810–819

    Google Scholar 

  13. Li Y, Ai H, Huang C (2006) Robust head tracking with particles based on multiple cues fusion. In: Proceeding of European conference on computer vision. Springer, Heidelberg, pp 29–39

    Google Scholar 

  14. Li Z, Schuster G, Katsaggelos A (2005) Rate-distortion optimal video summary generation. IEEE Trans Image Process 14(10):1550–1560

    Article  Google Scholar 

  15. Liu T, Katpelly R (2006) Content-adaptive video summarization combining queueing and clustering. In: Proceeding of IEEE international conference on image processing. IEEE, Piscataway, pp 145–148

    Google Scholar 

  16. Lu S, Lyu M, King I (2004) Video summarization by spatial-temporal graph optimization. In: Proceedings of the 2004 international symposium on circuits and systems. IEEE, Piscataway, pp 197–200

    Google Scholar 

  17. Ngo C, Ma Y, Zhang HJ (2003) Automatic video summarization by graph modeling. In: Proceeding of IEEE international conference on computer vision, vol 1. IEEE, Piscataway, pp 104–109

    Google Scholar 

  18. Otsuka I, Nakane K, Divakaran A (2005) A highlight scene detection and video summarization system using audio feature for a personal video recorder. IEEE Trans Consum Electron 51(1):112–116

    Article  Google Scholar 

  19. Peker K, Otsuka I, Divakaran A (2006) Broadcast video program summarization using face tracks. In: Proceedings of international conference on multimedia and exp. IEEE, Piscataway, pp 1053–1056

    Chapter  Google Scholar 

  20. Peng Y, Ngo C (2006) Clip-based similarity measure for query-dependent clip retrieval and video summarization. IEEE Trans Circuits Syst Video Technol 16(5):612–627

    Article  Google Scholar 

  21. Porter S, Mirmehdi M, Thomas B (2003) A shortest path representation for video summarization. In: Proceeding of IEEE international conference on image analysis and processing. IEEE, Piscataway, pp 460–465

    Google Scholar 

  22. Rasheed Z, Shah M (2003) Scene detection in Hollywood movies and TV shows. In: Proceeding IEEE international conference on computer vision and pattern recognition. IEEE, Piscataway, pp 343–348

    Google Scholar 

  23. Rasheed Z, Shah M (2005) Detection and representation of scenes in videos. IEEE Trans Multimedia 7(6):1097–1105

    Article  Google Scholar 

  24. Scharcanski J, Gaviao W (2006) Hierarchical summarization of diagnostic hysteroscopy videos. In: Proceeding of IEEE international conference on image processing. IEEE, Piscataway, pp 129–132

    Google Scholar 

  25. Shipman S, Radhakrishan R, Divakaran A (2006) Architecture for video summarization services over home networks and the Internet. In: Proceedings of international conference on consumer electronics. IEEE, Piscataway, pp 201–202

    Google Scholar 

  26. Smith T, Waterman M (1981) Identification of common molecular subsequences. J Mol Biol 147:195–197

    Article  Google Scholar 

  27. Song B, Vaswani N, Roy-Chowdhury A (2006) Summarization and indexing of human activity sequences. In: Proceeding of IEEE international conference on image processing. IEEE, Piscataway, pp 2925–2928

    Google Scholar 

  28. Sze K, Lam K, Qiu G (2005) A new key frame representation for video segment retrieval. IEEE Trans Circuits Syst Video Technol 15(9):1148–1155

    Article  Google Scholar 

  29. Uchihashi S, Foote J, Girgensohn A (1999) Video manga: generating semantically meaningful video summaries. In: Proceeding of ACM conference on multimedia. ACM, New York, pp 383–392

    Google Scholar 

  30. You J, Liu G, Sun L (2007) A multiple visual models based perceptive analysis framework for multilevel video summarization. IEEE Trans Circuits Syst Video Technol 17(3):273–285

    Article  Google Scholar 

  31. Zabih R, Miller J, Mai K (1999) A feature-based algorithm for detecting and classifying production effects. Multimedia Syst 7:119–128

    Article  Google Scholar 

  32. Zhao Y, Wang T, Wang P (2007) Scene segmentation and categorization using ncuts. In: Proceeding of IEEE workshop of computer vision and pattern recognition. IEEE, Piscataway, pp 1–7

    Google Scholar 

  33. Zhu X, Wu X (2003) Sequential association mining for video summarization. In: Proceeding of IEEE international conference on multimedia and expo. IEEE, Piscataway, pp 333–336

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank the reviewers and editors whose comments and suggestions have greatly improved this paper.

Author information

Authors and Affiliations

  1. Department of Automation, Tsinghua University, Beijing, China

    Yue Gao

  2. School of Software, Tsinghua University, Beijing, China

    Wei-Bo Wang & Jun-Hai Yong

  3. Jiangxi Academy of Sciences, Nanchang, China

    He-Jin Gu

Authors
  1. Yue Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei-Bo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jun-Hai Yong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. He-Jin Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yue Gao.

Additional information

Authors of Tsinghua University were supported by Chinese 973 Program(2004CB719400) and the National Science Foundation of China (60533070,90715043).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Gao, Y., Wang, WB., Yong, JH. et al. Dynamic video summarization using two-level redundancy detection. Multimed Tools Appl 42, 233–250 (2009). https://doi.org/10.1007/s11042-008-0236-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-008-0236-x

Keywords

Search

Navigation