Abstract
The need for watching movies is in perpetual increase due to the widespread of the internet and the increasing popularity of the video on demand service. The important mass of movies stored in the Internet or in VOD servers need to be structured to accelerate the browsing operation. In this paper, we propose a new system called "The Scene Pathfinder" that aims at segmenting the movies into scenes to give users the opportunity to have a non- sequential access and to watch particular scenes of the movie. This helps them to judge quickly the movie and decide if they have to buy or to download it and avoiding waste of time and money. The proposed approach is multimodal. We use both of visual and auditory information to accomplish the segmentation. We base on the assumption that every movie scene is either action or non- action scene. Non-action scenes are generally characterized by static backgrounds and occur in the same place. For this reason, we base on the content information and on the Kohonen map to extract these kinds of scenes (shots agglomerations). Action scenes are characterized by high tempo and motion. For this reason, we base on tempo features and on the Fuzzy CMeans to classify shots and to localize the action zones. The two processes are complementary. Indeed, the over segmentation that may occur in the extraction of action scenes by basing on the content information is repaired by the Fuzzy clustering. Our system is tested on a varied database and obtained results show the merit of our approach and that our assumptions are well-founded.
Similar content being viewed by others
References
Arijon D (1991) Grammar of the Film Language. Silman James Press, Los Angeles
Bezdek JC (1981) Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum, New York
Bordwell, D, Thompson K (1997) Film Art: An Introduction, 5th edn. McGraw-Hill
Boujemaa N, Fauqueur J, Ferecatu M, Fleuret F, Gouet V, Saux BL, Sahbi H (2001) Ikona: Interactive generic and specific image retrieval. In: Proceedings of the International workshop on Multimedia
Brunelli R, Mich O, Modena CM (1999) A survey on the automatic indexing of video data. Journal of Visual Communication Image Represent 10:78–112
Chen L, Ozsu MT (2002) Rule-based scene extraction from video. In International Conference on Image Processing, pp 737-740
Chen SC, Shyu ML, Zhang CC, Kashyap RL (2001) Video Scene change detection method using unsupervised segmentation and object tracking, In Proceedings of IEEE International Conference on Multimedia and Expo, pp 56-59
Chen HW, Kuo JH, Chu WT, Wu JL (2004) Action Movies Segmentation and Summarization Based on Tempo Analysis, In Proceedings of the ACM SIGMM International Workshop on Multimedia Information Retrieval, pp 251-258
Chen LH, Lai YC, Liao HYM (2008) Movie scene segmentation using background information. Pattern Recognition 41:1056–1065
Cotsaces C, Nikolaidis N, Pitas I (2006) Video Shot Detection and Condensed Representation, A review. IEEE Signal Processing Magazine 23, pp 28–37
Ellouze M, Karray H, Alimi AM (2006) Genetic Algorithm For Summarizing News Stories. In Proceedings of international conference on computer vision theory and applications, pp 303-308
Ellouze M, Karray H, Alimi AM (2008) REGIM, Research Group on Intelligent Machines, Tunisia, at TRECVID 2008, BBC Rushes Summarization, In Proceedings of international conference ACM Multimedia, TRECVID BBC Rushes Summarization Workshop
Ellouze M, Karray H, Soltana WB, Alimi AM (2007) Utilisation de la carte de Kohonen pour la détection des plans présentateur d’un journal télévisé, In Proceedings of international conference TAIMA 2007, cinquième édition des ateliers de travail sur le traitement et l'analyse de l’information, pp 271-276
Geng Y, Xu D, Wu A (2005) Effective Video Scene Detection Approach Based on Cinematic Rules. In Proceedings 9th International Conference on Knowledge-Based Intelligent Information and Engineering Systems, pp 1197-1203
Hanjalic A, Lagendijk RL, Biemond J (1999) Automated high-level movie segmentation for advanced video-retrieval systems. IEEE Transaction Circuits and Systems for Video Technology 9:580–588
Hanjalic A (2002) Shot-boundary detection: unraveled and resolved? IEEE Transactions on Circuits and Systems for Video Technology 12:90–105
Huang J, Liu Z, Wang Y (1998) Integration of Audio and Visual Information for Content-based Video Segmentation. In Proceedings of IEEE International Conference on Image Processing, pp 526–529
IMDB (2008) http://www.imdb.com/, Last viewed July 2008
Karray H, Ellouze M, Alimi AM (2008) KKQ: K-frames and K-words extraction for quick news story browsing. International Journal of Information and Communication Technology 1, pp. 69–76
Karray H, Ellouze M, Alimi AM (2008) Indexing video summaries for quick video browsing. Chapter in Computer Communications and Networks published by Springer Verlag, Germany. In Press
Kender JR, Yeo BL (1998) Video Scene Segmentation Via Continuous Video Coherence, In Proceedings of the conference of Computer Vision and Pattern Recognition, pp 367–373
Kherallah M, Karray H, Ellouze M, Alimi AM (2008) Toward an Interactive Device for Quick News Story Browsing. In Proceedings of international conference on pattern recognition. Accepted
Kohonen T (1990) The Self-Organizing Map. In Proceedings of the IEEE, pp 1464-1480
Lehane B, O’Connor NE (2006) Movie Indexing via Event Detection. In Proceedings of the Workshop on image analysis for multimedia interactive services, pp 1-4
Lin T, Zhang HJ (2000) Automatic Video Scene Extraction by Shot Grouping. In proceedings of the International Conference of Pattern Recognition 6:39–42
Lin T, Zhang HJ, Shi QY (2001) Video scene extraction by force competition, In the proceedings of IEEE International Conference on Multimedia and Expo, pp 753-756
Lu L, Zhang HJ, Jiang H (2002) Content Analysis for Audio Classification and Segmentation. IEEE Transactions on Speech and Audio Processing 10:504–516
Lukas B, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In Proceedings of the International Joint Conference on Artificial Intelligence, pp 674–679
Nagasaka A, Tanaka Y (1991) Automatic scene-change detection method for video works. In 2ndWorking Conference on Visual Database Systems, pp 119–133
Ngo CW, Pong TC, Zhang HJ (2002) Motion-Based Video Representation for Scene Change Detection. International Journal of Computer Vision 2:127–142
Oh J, Hua KA, Liang N (2000) A content-based scene change detection and classification technique using background tracking, In Proceedings of the conference on Multimedia Computing and Networking, pp 254-265
Rasheed Z, Shah M (2005) Detection and Representation of Scenes in Videos. IEEE Transaction on Multimedia 7:1097–1105
Rui Y, Huang TS, Mehrotra S (1998) Constructing table of contents for videos. ACM J. Multimedia Systems, pp 359–368
Smeaton AF, Lehane B, O'Connor NE, Brady C, Craig G (2006) Automatically selecting shots for action movie trailers. In Proceedings of the ACM international workshop on Multimedia information , pp 231-238
Snoek CGM, Worring M, Geusebroek JM, Koelma DC, Seinstra FJ, Smeulders AWM (2006) The Semantic Pathfinder: Using an Authoring Metaphor for Generic Multimedia Indexing. IEEE Transactions on Pattern Analysis and Machine Intelligence 28:1678–1689
Studio4networks (2008) http://www.studio4networks.com/, Last viewed July 2008
Sundaram H, Chang SF (2000) Video Scene Segmentation Using Video and Audio Features. In Proceedings of the International Conference on Multimedia and Expo, pp1145-1148
Tavanapong W, Zhou J (2004) Shot clustering techniques for story browsing. IEEE Transactions on Multimedia 6:517–527
TRECVID (2008) http://www-nlpir.nist.gov/projects/trecvid/, Last viewed July 2008
Truong BT, Dorai C, Venkatesh S (2003) Automatic scene extraction in motion pictures. IEEE Transactions in Circuits and Systems for Video Technology 1:5–10
Yale film studies, 2008, http://classes.yale.edu/film-analysis/index.htm, Last viewed July 2008
Yeung M, Yeo BL, Liu B (1998) Segmentation of video by clustering and graph analysis, Computer Vision and Image Understanding 71, pp 94-109
Zhao L, Yang SQ, Feng B (2001) Video scene detection using slide windows method based on temporal constrain shot similarity. In Proceedings of international conference on Multimedia and Expo, pp 1171–1174
Acknowledgments
The authors would like to thank several individuals and groups for making the implementation of this system possible. The authors would like to acknowledge the financial support of this work by grants from the General Direction of Scientific Research and Technological Renovation (DGRSRT), Tunisia, under the ARUB program 01/UR/11/02. We are also grateful, to EGIDE and INRIA, France, for sponsoring this work and the three-month research placement of Mehdi Ellouze from 1/11/2007 to 31/1/2008 in INRIA IMEDIA Team in which parts of this work were done.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Ellouze, M., Boujemaa, N. & Alimi, A.M. Scene pathfinder: unsupervised clustering techniques for movie scenes extraction. Multimed Tools Appl 47, 325–346 (2010). https://doi.org/10.1007/s11042-009-0325-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-009-0325-5