Video Summarization: Techniques and Classification

  • Muhammad Ajmal
  • Muhammad Husnain Ashraf
  • Muhammad Shakir
  • Yasir Abbas
  • Faiz Ali Shah
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7594)


A large number of cameras record video around the clock, producing huge volumes. Processing these huge chunks of videos demands plenty of resources like time, man power, and hardware storage etc. Video summarization plays an important role in this context. It helps in efficient storage, quick browsing, and retrieval of large collection of video data without losing important aspects. In this paper, we categorize video summariztion methods on the basis of methodology used, provide detailed description of leading methods in each category, and discuss their advantages and disadvantages. Moreover, we discuss the situation in which each method is most suitable to use. The advantage of this research is that one can quickly learn different video summarization techniques, and select the method that is the most suitable according to one’s requirements.


Spectral Cluster Music Video Video Summarization Spectral Cluster Algorithm Video Summary 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Divakaran, A., Peker, K.A., Sun, H.: Video Summarization Using Motion Descriptors. In: Conf. on Storage and Retrieval from Multimedia Databases (2001)Google Scholar
  2. 2.
    Ju, S.X., Black, M.J., Minneman, S., Kimber, D.: Summarization of Video-Taped Presentations: Automatic Analysis of Motion and Gestures. IEEE Transactions on CSVT (1998)Google Scholar
  3. 3.
    Fujimur, K., Honda, K., Uehara, K.: Automatic Video Summarization by Using Color and Utterance Information. In: Proceedings 2002 IEEE International (2002)Google Scholar
  4. 4.
    Zhang, H.J., Low, C.Y., Smoliar, S.W.: Video parsing and browsing using compressed data. Multimedia Tools and Applications 1, 89–111 (1995)CrossRefGoogle Scholar
  5. 5.
    DeManthon, D., Kobla, V., Doermann, D.: Video Summarization by Curve Simplification. In: Proceedings of the Sixth ACM International Conference on Multimedia (1998)Google Scholar
  6. 6.
    Koskela, M., Sjberg, M., Laaksonen, J., Viitaniemi, V., Muurinen, H.: Rushes Summarization with Self-Organizing Maps. In: Proceedings of the International Workshop on TRECVID Video Summarization (2007)Google Scholar
  7. 7.
    Truong, B.T., Venkatesh, S.: Video Abstraction: a Systematic Review and Classification. ACM Transactions on Multimedia Computing, Communications, and Applications 3(1) (2007)Google Scholar
  8. 8.
    Li, Y., Zhang, T., Tretter, D.: An Overview of Video Aabstraction Techniques. Technical Report HPL (2001)Google Scholar
  9. 9.
    Barbieri, M., Agnihotri, L., Dimitrova, N.: Video summarization: methods and landscape. In: Proceedings of SPIE, vol. 5242, p. 1 (2003)Google Scholar
  10. 10.
    Adami, N., Benini, S., Leonardi, R.: An Overview of Video Shot Clustering and Summarization Techniques for Mobile Applications. In: Proceedings of the 2nd International Conference on Mobile Multimedia Communications (2006)Google Scholar
  11. 11.
    Wolf, W.: Key frame selection by motion analysis. In: ICASSP, vol. 2, pp. 1228–1231 (1996)Google Scholar
  12. 12.
    Wang, F., Ngo, C.W.: Summarizing rushes videos by motion, object and event understanding. IEEE Transactions on Multimedia 14 (2012)Google Scholar
  13. 13.
    Zhang, H.J., Wu, J., Zhong, D., Smoliar, S.W.: An integrated system for content based video retrieval and browsing. Pattern Recognition 30, 643–658 (1997)CrossRefGoogle Scholar
  14. 14.
    Chheng, T.: Video Summarization Using Clustering. Department of Computer Science University of California, Irvine (2007)Google Scholar
  15. 15.
    Damnjanovic, U., Fernandez, V., Izquierdo, E.: Event Detection and Clustering for Surveillance Video Summarization. In: Proceedings of the Ninth International Workshop on Image Analysis for Multimedia Interactive Services. IEEE Computer Society, Washington, USA (2008)Google Scholar
  16. 16.
    Hanjalic, A., Zhang, H.: An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis. IEEE Transactionson Circuits and Systems for Video Technology 9, 1280–1289 (1999)CrossRefGoogle Scholar
  17. 17.
    Vctor Valdes, J.M.M.: On-Line Video Skimming Based on Histogram Similarity. In: Proceedings of the International Workshop on TRECVID Video Summarization (2007)Google Scholar
  18. 18.
    Li, B., Sezan, M.I.: Event Detection and Summarization in Sports Video. In: Content-Based Access of Image and Video Libraries, CBAIVL IEEE Workshop (2001)Google Scholar
  19. 19.
    Uchihachi, S., Foote, J., Wilcox, L.: Automatic Video Summarization Using a Measure of Shot Importance and a Frame Packing Method. United States Patent 6, 535,639, March 18 (2003)Google Scholar
  20. 20.
    Evangelopoulos, G., Rapantzikos, K., Potamianos, A., Maragos, P., Zlatintsi, A., Avrithis, Y.: Movie Summarization Based on Audio-Visual Valiency Detection. In: IEEE Intl Conf. Image Processing (ICIP), San Diego, CA (2008)Google Scholar
  21. 21.
    Wang, J., Adelson, E.: Representing moving images with layers. IEEE Transactions on Image Processing 3 (1994)Google Scholar
  22. 22.
    Pope, A., Kumar, R., Sawhney, H., Wan, C.: Video abstraction: Summarizing video content for retrieval and visualization (1998)Google Scholar
  23. 23.
    Aner, A., Kender, J.R.: Video Summaries through Mosaic-Based Shot and Scene Clustering. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part IV. LNCS, vol. 2353, pp. 388–402. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  24. 24.
    Sawhney, H., Ayer, S.: Compact representation of video through dominent and multiple motion estimation. IEEE Trans. on Pattern. Analysis and Machine Intelligence 18 (1996)Google Scholar
  25. 25.
    Lee, C.H., Varshney, A., Jacob, D.W.: Mesh saliency. ACM Transaction on Graphics, 659–666 (2005)Google Scholar
  26. 26.
    Ngo, C.W., Ma, Y.F., Zhang, H.J.: Automatic Video Summarization by Graph Modeling. In: Proceedings of the 9th IEEE International Conference on Computer Vision (2003)Google Scholar
  27. 27.
    Qiu, X., Jiang, S., Liu, H., Huang, Q., Cao, L.: Spatial temporal attention analysis for home video. In: IEEE International Multimedia and Expo, vol. 23 (2008)Google Scholar
  28. 28.
    Bulut, E., Capin, T.: Key Frame Extraction from Motion Capture Data by Curve Saliency. In: Proceedings of 20th Annual Conference on Computer Animation and Social Agents, Belgium (2007)Google Scholar
  29. 29.
    Peyrard, N., Bouthemy, P.: Motion-Based Selection of Relevant Video Segments for Video Summarization 26(3) (2005)Google Scholar
  30. 30.
    Li, C., Wu, Y.T., Yu, S.S., Chen, T.: Motion-focusing key frame extraction and video summarization for lane surveillance system. In: 16th IEEE International Conference on Image Processing (ICIP), pp. 7–10 (2009)Google Scholar
  31. 31.
    Chen, F., Cooper, M., Adcock, J.: Video Summarization Preserving Dynamic Content. In: Proceedings of the International Workshop on TRECVID Video Summarization (2007)Google Scholar
  32. 32.
    Adnan, H., Mufti, M.: Video Summarization Based Handout Generation from Video Lectures: A Gesture Recognition Framework. In: 5th WSEAS International Conference on Signal Processing, Computational Geometry and Artificial Vision (2005)Google Scholar
  33. 33.
    Kosmopoulos, D.I., Doulamis, A., Doulamis, N.: Gesture-based video summarization. In: ICIP IEEE International Image Processing, pp. 11–14 (2005)Google Scholar
  34. 34.
    Furini, M., Ghini, V.: An Audio-Video Summarization Scheme Based on Audio and Video Analysis. In: IEEE CCNC (2006)Google Scholar
  35. 35.
    Divakaran, A., Peker, K., Radhakrishnan, R., Xiong, Z., Cabasson, R.: Video summarization using mpeg7 motion activity and audio descriptors. In: Video Mining, vol. 91 (2003)Google Scholar
  36. 36.
    Evangelopoulos, G., Rapantzikos, K., Potamianos, A., Maragos, P., Zlatintsi, A., Avrithis, Y.: Movie Summarization Based on Audiovisual Saliency Detection. In: ICIP (2008)Google Scholar
  37. 37.
    Shao, X., Xu, C., Maddage, N.C., Kankanhalli, M.S., Jin, J.S., Tian, Q.: Automatic summarization of music videos. ACM Transactions on Multimedia Computing,Communications and Applications (TOMCCAP) 2 (2006)Google Scholar
  38. 38.
    Taskiran, C.M., Amir, A., Ponceleon, D., Delp, E.J.: Auto-mated video summarization using speech transcripts. In: Proceedings of SPIE Conference on Storage and Retrieval for Media Databases volume, San Jose, CA, pp. 20–25 (2002)Google Scholar
  39. 39.
    Taskiran, C.M., Pizlo, Z., Amir, A., Ponceleon, D., Delp, E.J.: Automated video program summarization using speech transcripts. IEEE Transactions on Multimedia (2006)Google Scholar
  40. 40.
    Bahl, L.R., Aiyer, S.B., Bellegarda, J.R., Franz, M., Gopalakrisnan, P.S., Nahamoo, D., Novak, M., Padmanabhan, M., Picheny, M.A., Roukos, S.: Performance of the IBM Large Vocabulary Continuous Speech Recognition System on the ARPA Wall Street Journal Task. In: Proceedings of IEEE International Conference on Acoustic, Speech and Signal Processing, Detroit, MI (1995)Google Scholar
  41. 41.
    Dunning, T.E.: Accurate methods for the statistics of surprise and coincidence. Computational Linguistics 19(1), 61–74 (1993)Google Scholar
  42. 42.
    Liu, D., Chen, T., Hua, G.: A hierarchical visual model for video object summarization. IEEE Transactions on Pattern Analysis and Machine Intelligence 32 (2010)Google Scholar
  43. 43.
    Kim, C., Hwang, J.N.: An Integrated Scheme for Object-Based Video Abstraction. In: Proceedings of the 8th ACM International Conference on Multimedia (2000)Google Scholar
  44. 44.
    Lee, Y.J., Ghosh, J., Grauman, K.: Discovering Important People and Objects for Egocentric Video Summarization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR (2012)Google Scholar
  45. 45.
    Ferman, A.M., Gunsel, B., Tekalp, A.M.: Object-Based Indexing of MPEG-4 Compressed Video. In: Proceedings of IS&T/SPIE Symp. on Electronic Imaging (1997)Google Scholar
  46. 46.
    Pritch, Y., Ratovitch, S., Hendel, A., Peleg, S.: Clustered synopsis of surveillance video. In: 6th IEEE Int Conf. on Advance Video and Signal Base Selection (AVSS 2009), Genoa, Italy, pp. 2–4 (2009)Google Scholar
  47. 47.
    Ali Amiri, M.F.: Hierarchical key frame-based video summarization using qr-decomposition and modified k-means clustering. EURASIP Journal on Advaces in Signal Processing (February 2010)Google Scholar
  48. 48.
    Farin, D., Effelsberg, W., Peter, H.N.: Robust Clustering Based Video Summarization with Integration of Domain Knowledge. In: Proceedings 2002 IEEE International Conference (2002)Google Scholar
  49. 49.
    Jain, A.K., Dubes, R.C.: Algorithms for Clustering Data. Prentice-Hall, Englewood Cliffs (1988)zbMATHGoogle Scholar
  50. 50.
    Peker, K.A., Bashir, F.I.: Content-Based Video Summarization using Spectral Clustering. Mitsubishi Electric Research Laboratories Cambridge, MA. University of Illinois at Chicago, Chicago, IL (2009)Google Scholar
  51. 51.
    Girgensohn, A., Foote, J.: Video Frame Classification Using Transform Coefficients. In: ICASSP 1999 (1999)Google Scholar
  52. 52.
    Stefanidis, A., Partsinevelos, P., Peggy Agouris, P.D.: Summarizing Video Datasets in the Spatiotemporal Domain (2000)Google Scholar
  53. 53.
    Massey, M., Bender, W.: Salient stills: Process and practice. IBM Systems Journal 35 (1996)Google Scholar
  54. 54.
    Lee, M., Chen, W., Lin, C., Gu, C., Markoc, T., Zabinsky, S., Szeliski, R.: A layered video object coding system using sprite and affine motion model. IEEE Transactions on Circuits and Systems for Video Technology (1997)Google Scholar
  55. 55.
    Vasconcelos, N., Lippman, A.: A Spatio Temporal Motion Model for Video Summarization. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (1998)Google Scholar
  56. 56.
    Iran, M., Anandan, P.: Video indexing based on mosaic representation. IEEE Computer Society (1998)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Muhammad Ajmal
    • 1
  • Muhammad Husnain Ashraf
    • 1
  • Muhammad Shakir
    • 1
  • Yasir Abbas
    • 1
  • Faiz Ali Shah
    • 1
  1. 1.COMSATS Institute of Information TechnologyLahorePakistan

Personalised recommendations