Video coding standards, such as H.264/MPEG-4 AVC, have adopted the hybrid codec paradigm that consists of intra and inter prediction, followed by transform and quantization of the prediction residual. Inter prediction uses block-based motion compensation (BMC) to predict samples in the current picture. BMC, however, suffers from blocking artifacts and discontinuities. Along with deblocking, overlapped block motion compensation (OBMC) has been proposed in the past, primarily to suppress blocking artifacts. However, issues that include decoding complexity and implementation have been impediments to practical applications of OBMC. Here1, we propose a new decoder- and encoder-friendly architecture for inter prediction with OBMC. Furthermore, we show preliminary experimental results that point to coding efficiency gains over the BMC architecture of H.264/MPEG-4 AVC.

Weighted prediction in the H.264/MPEG-4 AVC video coding standard is a coding tool used primarily for improving coding efficiency during illumination changes, such as fading transitions. In this paper, we extend the usage of weighted prediction within the context of motion compensation in H.264, to further improve coding efficiency even for scenes with constant illumination characteristics. This is done by better exploiting coding tools already available in H.264 such as reference list reordering and multiple reference prediction.

The combination of intra prediction, transform, and quantization in the H.264/AVC video coding standard offers high compression performance in picture areas where inter prediction is unavailable or fails. Motivated from coding experiments with the reference software that resulted in significant visual distortion with certain types of content, we investigated intra prediction further. Intra prediction may propagate distortion to subsequent blocks or macroblocks. As a result, rate-distortion optimized mode decision needs to account for this distortion. Furthermore, rounding operations in H.264/AVC accumulate small but non-trivial errors. This is especially true for the encoding process of the 16 times 16 intra prediction mode, which involves a combination of DCT and Hadamard transforms. We investigate, theoretically and experimentally, the problem of visual distortion caused by sub-optimal intra prediction coding decisions by modeling distortion propagation across subsequent blocks. We propose two algorithms that address the problem.

Rate control is an integral part of a video coding system as it ensures the generation of compressed bit streams that satisfy bandwidth and buffering constraints. Traditional rate control algorithms maintain a rate control model for a single picture type that is applied on pictures of this type. The remaining pictures are allocated a quantization parameter with no consideration of bit rate constraints. Video sequences may be coded with complex prediction structures that affect coding requirements. In this paper, we improve rate control for I- and B-coded pictures, especially in the context of hierarchical coding. We also describe a new rate control paradigm for coding video sequences with multiple slice types that achieves the bit target for all pictures. Both contributions improve rate-distortion performance and rate control accuracy.

Flickering is a temporal visual artifact that affects compressed video. It is prominent in intra-frame video coders and is largely the result of content variations and quantization. We concentrate on flickering due to quantization. JPEG2000 uses post-compression quantization which is applied through the EBCOT algorithm. EBCOT has been found, however, to cause significant flickering in the reconstructed video. In this work, we evaluate existing flicker metrics, investigate the causes of flicker, and propose a new rate-distortion optimal algorithm that suppresses flicker. The proposed algorithm suppresses temporal flicker at a negligible cost in spatial image quality.

Real-time video applications require tight bounds on end-to-end delay. Hierarchical bidirectional prediction requires buffering frames in the encoder input buffer, thereby contributing to encoder input delay. Long-term frame prediction with pulsed quality requires buffering at the encoder output, increasing the output buffer delay. Both hierarchical B-pictures and pulsed-quality coders involve uneven bit-rate allocation. Both the encoder and decoder buffering requirements depend on the rate allocation. We derive an efficient rate allocation for hierarchical B-pictures using the power spectral density of a wide-sense stationary process. In addition, we discuss important aspects of hierarchical predictive coding, such as the effect of the temporal prediction distance and delay tradeoffs for prediction branch truncation. Finally, we investigate experimentally the tradeoff between delay and compression efficiency.

Little attention has been paid to an impairment common in motion-compensated video compression: the addition of high-frequency (HF) energy as motion compensation displaces blocking artifacts off block boundaries. In this paper, we employ an energy-based approach to measure this motion-compensated edge artifact, using both compressed bitstream information and decoded pixels. We evaluate the performance of our proposed metric, along with several blocking and blurring metrics, on compressed video in two ways. First, ordinal scales are evaluated through a series of expectations that a good quality metric should satisfy: the objective evaluation. Then, the best performing metrics are subjectively evaluated. The same subjective data set is finally used to obtain interval scales to gain more insight. Experimental results show that we accurately estimate the percentage of the added HF energy in compressed video.

Video sequences encoded with JPEG2000 exhibit flicker artifact. This artifact is not perceivable in still images. In contrast, it is perceived in the temporal domain and is the result of two factors: (a) the image content, and (b) the EBCOT post-compression rate allocation. In this work, we address the second factor. First, we propose an intra-frame rate-allocation scheme that ensures uniform error energy distribution and improves perceptual quality. We then extend this scheme to the temporal domain and further modify it to address flicker. Experimental results show that our algorithm suppresses the flicker artifact without sacrificing good PSNR performance

Real-time video applications require tight bounds on end-to-end delay. Hierarchical bi-directional prediction requires buffering frames in the encoder input buffer, thereby contributing to encoder input delay. Long-term frame prediction with pulsed quality requires buffering at the encoder output, increasing the output buffer delay. We compare the end-to-end delay of these two approaches using simulations to determine the delay vs. compression efficiency trade-off.

We address the problem of enhancement layer drift estimation for fine granular scalable video. An optimal per-pixel drift estimation algorithm is introduced. The encoder assumes that there is some truncation of the enhancement layer, which does not allow the enhancement layer reference to be properly reconstructed, and the encoder recursively estimates the associated drift and chooses coding modes accordingly. The approach yields performance gains of about 1 dB across low to medium rates. In addition, we investigate dual frame prediction, for both base and enhancement layer, with pulsed-quality allocation in the base

We evaluate the performance on compressed video of a number of available similarity, blocking, and blurring quality metrics. Using a systematic, objective framework based on simple subjective comparisons, we evaluate the ability of each metric to rank images correctly according to the subjective impact of differences in spatial content, quantization parameters, amounts of filtering, distances from the most recent I-frame, and long-term frame prediction strategies. The indicated weaknesses of available metrics can be used as guides in the development of future quality metrics.

A major focus of video quality assessment research has been to quantify the amount of blocking, blurring, and ringing impairments. However, little attention has been paid to another impairment common in motion-compensated video compression systems: the addition of high frequency (HF) energy as motion compensation moves blocking artifacts off block boundaries. In this paper, we employ an energy-based approach to measure this motion-compensated edge artifact (MCEA) impairment, using both compressed bitstream information and decoded pixels. Experimental results show that we can accurately estimate the percentage of this energy in compressed video.

A dual-frame video coder employs two past reference frames for motion compensated prediction. Compared to conventional single frame prediction, the dual-frame encoder can have advantages both in distortion-rate performance and in error resilience. In previous work, it was shown that optimal mode selection can enhance the performance of a dual-frame encoder. In another strand of previous work, it was shown that uneven assignment of quality to frames, to create high-quality (HQ) long-term reference frames, can enhance the performance of a dual-frame encoder. In this letter, we combine these two strands and demonstrate the performance advantages of optimal mode selection among HQ frames for video transmission over noisy channels.

Video codecs that use motion compensation benefit greatly from the development of algorithms for near-optimal intra/inter mode switching within a rate-distortion framework. A separate development has involved the use of multiple-frame prediction, in which more than one past reference frame is available for motion estimation. In this paper, we show that using a dual-frame buffer (one short-term frame and one long-term frame available for prediction) together with intra/inter mode switching improves the compression performance of the coder. We improve the mode-switching algorithm with the use of half-pel motion vectors. In addition, we investigate the effect of feedback in making more informed and effective mode-switching decisions. Feedback information is used to limit drift errors due to packet losses by synchronizing the long-term frame buffers of both the encoder and the decoder.

Video codecs that use motion compensation have achieved performance improvements from the use of intra/inter mode switching decisions within a rate-distortion framework. A separate development has involved the use of multiple frame prediction, in which more than one past reference frame is available for motion estimation. It is shown that using a dual frame buffer, together with intra/inter mode switching improves the compression performance of the coder. Also, the mode switching algorithm is improved with the use of half-pel motion vectors.

A novel framework for multiple description coding of video is presented. The proposed scheme is based on an efficient methodology for the generation of multiple descriptions which eliminates drift at the decoder. An arbitrary number of descriptions can be obtained from the same original video sequence. Experimental results for the transmission of video using two descriptions demonstrate the efficiency of the proposed methodology.

In this paper we investigate the coding of 3D medical images using three-dimensional wavelet decomposition and adaptive arithmetic coding. A highly efficient context arithmetic coding scheme is used in conjunction with classical lifting-based wavelet decompositions. The resulting coder is shown to attain superior performance in comparison to other compression schemes for 3D medical image coding.