Video Processing Using Compressed Data

Abstract

The processing described in the preceding Chapter is all concerned with operations on pixel representations of individual image frames. However, if a video source is provided in compressed form, these operations cannot be performed until that representation has been decompressed. If decompression can be provided by hardware, this will not constitute a significant computational overhead; but software decompression is far less efficient. If special-purpose hardware is not available, it is worth asking to what extent these operations may be performed directly on compressed representations. We shall now present three difference metrics for frame comparison, two of which employ the DCT coefficients used in both JPEG and MPEG representations, while the third uses MPEG motion vectors. After this, we discuss how to combine the power of the two approaches, incorporating multi-pass, twin-comparison strategies and motion analysis which enable the detection of gradual transitions and camera operations.