Combining Hierarchical Classifiers with Video Semantic Indexing Systems

Abstract

This paper proposes a mechanism to integrate hierarchical video classification into video indexing systems seamlessly using video mixed media cues. Our approach centers on novel techniques that semiautomatically generate a media concept hierarchy using hierarchical classifiers to represent relevant video, audio and closed-caption text features for each video concept. Video classification functions, which directly connect low-level features with high-level semantic meanings for various applications, are first learned from training data using supervised learning algorithms for the hierarchical video concepts. The text classifier and video/audio classifier are constructed using independent learning algorithms and independent media streams of video. The joint classification fusion strategy is derived from Bayesian Theory and provides consistent and optimized classification results.