Very Low Bit Rate Video Coding Based on Statistical Spatio-Temporal Prediction of Motion, Segmentation and Intensity Fields

Abstract

There are a large number of applications requiring the compression of video at Very Low Bit Rates (VLBR). Such applications include wireless video conferencing, video over the internet, multimedia database retrieval and remote sensing and monitoring. Recently, the MPEG-4 standardization effort has been a motivating factor to find a solution to this challenging problem. The existing approaches to this problem can generally be grouped into block-based, model-based, and object-oriented. Block-based approaches follow the traditional strategy of decoupling the image sequence into blocks, model-based approaches rely on complex 3-D models for specific objects that are encoded, and object-oriented approaches rely on analyzing the scene into differently moving objects. All three approaches exhibit potential problems. Block-based approaches tend to generate artifacts at the boundaries of the blocks, as well as to limit the minimum achievable bit rate due to the fixed analysis structure of the scene. Model-based codecs are limited by the complex 3-D models of the objects to be encoded. On the other hand, object-oriented codecs can generate a significant overhead due to the analysis of the scene which needs to be transmitted, which in turn can be the limiting factor in achieving the target bit rates. In this chapter, we propose a hybrid object-oriented codec in which the correlations among the three information fields, e.g., motion, segmentation and intensity fields, are exploited both spatially and temporally. In the proposed method, additional intelligence is given to the decoder, resulting in a reduction of the required bandwidth. The residual information is analyzed in three different categories, i.e., occlusion, model failures, and global refinement. With other side information, it is encoded and transmitted across the channel. Experimental results are presented which demonstrate the effectiveness of the proposed approach.