We propose a motion-compensated video coding system employing dense motion fields. The dense motion field is calculated at the transmitter, and the motion information is efficiently encoded and transmitted along with the residual frame. The motion estimation is performed by existing techniques in the literature, while we focus on the coding of the motion field and the displaced frame difference (DFD) frame. The dense motion field formulation leads to several novel and distinct advantages. The motion field is encoded in a lossy manner to make the motion rate manageable. The more accurate and precise motion description allows us to predict where the DFD energy will be significant, thus leading to a more efficient DFD encoder compared to applying traditional still-image coding techniques. Furthermore, the dense motion field framework allows us to refine and tailor the motion estimation process such that the resulting DFD frame is easier to encode. Simulations demonstrate superior performance against standard block-based coders, with greater advantages for sequences with more complex motion.

We presents a method for encoding the displaced frame difference (DFD) that rounds out our continuing study on video compression using dense motion fields. The main idea of the DFD coding strategy is to utilize the more dense and thus accurate motion field to predict where the sparse but significant DFD energy is located. This leads to a more efficient DFD encoder compared to applying traditional still-image coding techniques. Furthermore, the dense motion field framework allows us to refine and tailor the motion estimation process such that the resulting DFD frame is easier to encode. Simulations demonstrate superior performance against standard block-based coders, with greater advantages for sequences with more complex motion.

A dense motion field provides a number of advantages over the traditional block-based motion models in interpreting the temporal correlation between video frames. However, until now, its application to motion-compensated video coding has been limited due to the inhibitive rate needed to encode the motion field. We present an efficient scheme for encoding a dense motion field which makes it possible to apply it in a forward motion-compensated video coding framework. A multi-scale motion field is first estimated to represent the motion at different scales. This full multi-scale motion field is then reduced adaptively by a tree pruning algorithm, resulting in a motion field that is represented at different scales from region to region. The pruned motion vector components are coded by a backward context-based arithmetic coding scheme. The motion vector encoding scheme is then incorporated into a complete video codec in which a much greater percentage of the overall bit-rate is allocated to the motion information (+30%). Simulations show that such a system provides substantial improvements over standard block-based coders.

This paper describes an object-based video coding system with new ideas in both the motion analysis and source encoding procedures. The moving objects in a video are extracted by means of a joint motion estimation and segmentation algorithm based on the Markov random field (MRF) model. The two important features of the presented technique are the temporal linking of the objects, and the guidance of the motion segmentation with spatial color information. This facilitates several aspects of an object-based coder. First, a new temporal updating scheme greatly reduces the bit rate to code the object boundaries without resorting to crude lossy approximations. Next, the uncovered regions can be extracted and encoded in an efficient manner by observing their revealed contents. The objects are classified adaptively as P objects or I objects and encoded accordingly. Subband/wavelet coding is applied in encoding the object interiors. Simulations at very low bit rates yielded comparable performance in terms of reconstructed PSNR to the H.263 coder. The object-based coder produced visually more pleasing video with less blurriness and devoid of block artifacts, thus confirming the advantages of object-based coding at very low bit-rates.

We present a motion-compensated interpolation scheme that allows recovery of skipped frames at the receiver. The interpolation is performed based on an object-based interpretation of the video. The main advantage of the scheme is that the decoded motion and segmentation information is used without refinement. This is attributable to the fact that the object-based representation is true to the "real" motion. We present simulation results in a very low bit-rate coding environment, where multiple frames need to be skipped. The interpolated frames show good reconstruction quality, whereas standard block-based methods lead to unacceptable artifacts.

We present a novel object-based spatiotemporal (3-D) subband/wavelet coding scheme for very low bit-rate video compression. First, the image sequence is segmented into objects based on motion and color. The objects are then independently decomposed by subband/wavelet analysis, both in the temporal and spatial domain. The temporal analysis is greatly facilitated over existing methods by the object-based representation. Each object's subband/wavelet coefficients are then quantized according to a bit allocation scheme based on the generalised BFOS algorithm. Simulations at very low bit-rates indicate competitive performance to classical motion-compensated predictive (MCP) coding schemes, while allowing the flexibility of frame-rate scalability.