Video distortion metrics based on models of the human visual system have traditionally used comparisons between the distorted signal and a reference signal to calculate distortions objectively. In video coding applications, this is not prohibitive. In quality monitoring applications, however, access to the reference signal is often limited. This paper presents a computationally efficient video distortion metric that can operate in full- or reduced-reference mode as required. The metric is based on a model of the human visual system implemented using the wavelet transform and separable filters. The visual model is parameterized using a set of video frames and the associated quality scores. The visual model's hierarchical structure, as well as the limited impact of fine scale distortions on quality judgments of severely impaired video, are exploited to build a framework for scaling the bitrate required to represent the reference signal. Two applications of the metric are also presented. In the first, the metric is used as the distortion measure in a rate-distortion optimized rate control algorithm for MPEG-2 video compression. The resulting compressed video sequences demonstrate significant improvements in visual quality over compressed sequences with allocations determined by the TM5 rate control algorithm operating with MPEG-2 at the same rate. In the second, the metric is used to estimate time series of objective quality scores for distorted video sequences using reference bitrates as low as 10 kb/s. The resulting quality scores more accurately model subjective quality recordings than do those estimated using the mean squared error as a distortion metric, while requiring a fraction of the bitrate used to represent the reference signal. The reduced-reference metric's performance is comparable to that of the full-reference metrics tested in the first Video Quality Experts Group evaluation.

A rate control technique for video encoding under complex transmission scenarios is presented. A typical application for this method is the transmission of video over variable bit rate channels while accounting for restrictions on the end-to-end delay and decoder buffer size. That the resulting multiple constraints on the source and channel rates may be relaxed without loss of optimality into a set of linear inequality constraints-though they are usually expressed in nonlinear form-is a key insight of this paper. This allows for a systematic treatment of a large class of rate constraints and leads to a convex programming (CP) formulation for rate control. Approximation of the frame distortion-rate data by piecewise linear functions further facilitates an efficient solution based on linear programming (LP), a special case of CP. The LP method provides bounds for the deviation from optimality. Results for a standard video test set show that the proposed method provides solutions with mean square error (MSE) distortion value within 2% of the global minimum across a range of rates. The proposed technique is also applied in conjunction with a perceived distortion measure. Results exhibit significant reduction in blocking artifacts and flicker compared to the use of MSE

This paper presents a Rate-Distortion based optimization procedure for video coding that explicitly incorporates a perceptual distortion measure. A flexible MINMAX (minimizing the maximum distortion) optimization procedure based on Linear Programming (LP) is used to minimize the variation in distortion across frames. The distortion measure utilizes a multi-channel human vision model to generate a map of the perceived distortion at each pixel in a frame. This map is then collapsed into a single frame-level distortion. Lagrangian optimization is employed within each frame to assign macroblock-level quantization step-sizes that minimize this frame-level distortion. Coding results with MPEG-2 show a large reduction in peak perceived distortion with respect to TM5 as well as lower variation in distortion across the frames in the video sequence. Although results are presented for a specific metric, the LP based optimization procedure easily extends to other distortion measures.

A double stimulus subjective evaluation was performed to determine preferred frame rates at a fixed bit rate for low bit rate video. Stimuli consisted of eight reference color video sequences of size 352/spl times/240 pixels. These were compressed at rates of 100, 200 and 300 kbps for low, medium, and high motion sequences, respectively, using three encoders and frame rates of 10, 15 and 30 frames per second. Twenty-two viewers ranked their frame rate preferences using an adjectival categorical scale. Their preferences were analyzed across sequence content, motion type, and encoder. Viewers preferred a frame rate of 15 frames per second across all categories, with several notable content-based exceptions.

The paper presents an objective metric for the perceived quality of low bit rate compressed video. Frames are decomposed into several frequency bands at multiple scales and a motion detection algorithm is used to segment moving and nonmoving regions. The distance between the decompositions of the reference and distorted video sequences is used to estimate a frame distortion. The pooled error across several frames and the correlation between successive estimates of perceived quality are used to estimate the current perceived quality at a rate of 2 samples per second. The results of a subjective evaluation of perceived quality are used to determine the model parameters. The variety of content and coding conditions used in the subjective evaluation permits a wide range of training conditions. The results indicate that the metric can capture many of the features of the measured perceived quality over time.

A subjective quality evaluation was performed to quantify viewer responses to various visual defects that appear in low bit rate video, both at full and reduced frame rates. The stimuli were eight video sequences compressed by three encoders operating at five bit rate/frame rate combinations. The sequences covered a broad range of typical streaming video content. Viewers continuously registered their responses to the stimuli as per the SSCQE method of ITU-R Recommendation BT.500-8. Data from 19 viewers was collected. An analysis of particularly disturbing artifacts is performed. The effects of changes in frame rate at a fixed bitrate on perceived quality are studied, demonstrating a preference for lower frame rates. A comparison of PSNR to the perceived quality ratings is made, revealing several notable trends.

With the proliferation of readily available image content, data hiding/watermarking has become a topic of considerable importance. In many applications, it is desirable to embed identifying data strings for authentication, or to embed contextual data that may be used for other purposes. It is also desirable that the image into which the data has been embedded be visually indistinguishable from the original. This paper presents a method in which psychovisual metrics are used in a wavelet based algorithm that can embed a set of bits into an image. The resulting image remains visually identical to the original. The algorithm can be used to embed very robust authentication data, or less robust data to be used for other purposes, where the robustness is a function of the number of bits to be embedded.