Shot Weave: A Shot Clustering Technique for Story Browsing for Large Video Databases

Abstract

Automatic video segmentation is the first and necessary step for organizing a long video file into several smaller units for subsequent browsing and retrieval. The smallest basic unit is shot. Since users of a video database management system are more likely to recall important events or stories rather than a particular frame or shot, relevant shots are typically grouped into a high-level unit called scene. Each scene is part of a story. Browsing these scenes unfolds the entire story of the film, allowing the users to locate their desired video segments quickly and efficiently. Existing scene definitions are rather broad, making it difficult to evaluate the scene results and compare existing techniques. This paper first gives a stricter scene definition and presents ShotWeave, a novel technique for clustering relevant shots into a scene for narrative films. The crux of ShotWeave is its feature extraction and comparison. Features are extracted from carefully selected regions of representative frames of shots. These regions capture essential information needed to maintain viewers’ thought in presence of shot breaks guided by common continuity-editing techniques used in film making. The experimental results show that ShotWeave performs well, and is more robust than a recent shot clustering technique on full-length films consisting of a wide range of camera motions and a complex composition of related shots.