Visual quality of compressed video sequences depends on factors including spatial texture content and cognition-based factors such as prior knowledge and task in hand. The MOSp metric is a full reference objective quality metric which predicts perceived quality of sequences with video compression-induced impairments based on the spatial texture content and the mean squared error between original and compressed video sequences. In this paper, we extend the MOSp metric to incorporate cognition-based factors to identify regions in a video scene that attract human attention. The proposed metric has been tested on a variety of multimedia sequences of common intermediate format resolution compressed at a wide range of bitrates using the H.264/AVC coding standard. This metric shows a higher correlation with mean opinion score (MOS) than popular metrics, such as peak signal-to noise ratio, National Telecommunications and Information Administration/Institute for Telecommunication Sciences video quality metric, PSNRplus, and the Yonsei University metric. Results also show that by extending the MOSp metric to incorporate cognition-based factors such as skin information, its correlation with subjective scores (MOS) can be significantly improved in video content where humans are present. This algorithm is particularly useful for real-time quality estimation of multimedia sequences with block-based video compression-induced impairments because all the parameters of the metric can be calculated automatically with a modest amount of processing overhead.

Fully configurable video coding is a novel approach to video compression in which a video decoder can be dynamically constructed based on a decoder description. This paper presents a new syntax for describing video decoder functionality and structure, namely, the decoder description syntax (DDS). The DDS is a platform-independent syntax that can define all aspects of a video decoder in terms of basic processing instructions. A DDS description of a video decoder may be coded and transmitted, and then executed by a generic processing platform, a “universal video decoder (UVD).” Any new or modified video decoding function may be described, communicated, and instantiated using the DDS, which makes it possible to rapidly implement new coding algorithms, to dynamically adapt the coding algorithm to suit the video data, and to efficiently implement multiple coding formats on one platform. We present examples of video decoding functions implemented in the DDS and show how these may be executed on a UVD. We demonstrate that flexible configuration, re-configuration, and decoding of video can be achieved using a real-time prototype UVD with functions transmitted and instantiated on-the-fly using the DDS.

This paper presents a new video quality metric for automatically estimating the perceptual quality of compressed video sequences. Distortion measures such as the mean squared error (MSE) and the peak signal to noise ratio (PSNR) have been found to poorly correlate with visual quality at lower bit-rates. The proposed quality metric (MOSp) predicts perceptual quality of compressed video using sequence characteristics and the mean squared error (MSE) between the original and compressed video sequences. The metric has been tested on various video sequences compressed using the H.264 video compression standard at different bit-rates. Results show that the proposed metric has better correlation with subjective quality compared to popular metrics such as PSNR, SSIM and PSNRplus. The new metric is simple to compute and hence suitable for incorporation into real-time applications such as the standard video compression codecs inorder to improve the visual quality of compressed video sequences. Index Terms — quality metrics, video quality, mean squared error, perceptual quality, compressed video 1.

Fully configurable video coding is a novel approach to video compression in which a video decoder can be dynamically constructed based on a decoder description. This paper presents a new syntax for describing video decoder functionality and structure, namely, the decoder description syntax (DDS). The DDS is a platform-independent syntax that can define all aspects of a video decoder in terms of basic processing instructions. We present examples of video decoding functions implemented in the DDS and show that flexible configuration, reconfiguration, and decoding of video can be achieved using a real-time prototype universal video decoder.

A codec that can be re-configured dynamically has the potential to offer greater rate-distortion performance than fixed codecs by adapting video coding algorithms depending on local statistics and varying system resources. In addition, complete, dynamic configuration of a video decoder makes it possible to rapidly implement new algorithms and efficiently support multiple coding formats on a single platform. This paper describes a framework for dynamic re-configuration of video codecs, fully configurable video coding, in which new or modified decoder designs are generated, transmitted, and then implemented by a universal video decoder. We present rate-distortion results generated by a real-time prototype of this new approach.

Modern video compression systems make optimum coding decisions based on rate-distortion performance. Typically the distortion is evaluated as a mathematical error measurement, such as mean squared error (MSE), between the original and compressed video sequences. However, simple automatic error measurements such as MSE do not correlate well with perceptual quality, leading to sub-optimal coding decisions. In this paper we present a new subjective video quality metric that predicts subjective quality of compressed video using temporal and spatial masking information and the MSE between the original and compressed video sequences. The results show that this metric can predict perceptual quality with significantly higher correlation compared to popular metrics such as PSNR, VSSIM and PSNRplus. This algorithm is particularly useful for real-time, accurate perceptual video quality estimation in video applications because all the parameters of the metric can be calculated with a minimal amount of processing overhead.

Video coding solutions based on fixed, pre-defined video coding standards have certain limitations. Dynamic reconfiguration has the potential to offer greater rate-distortion performance than fixed codecs, making it possible to rapidly implement new algorithms and efficiently support multiple coding formats on a single platform. In this paper we describe a framework for dynamic re-configuration of video codecs, fully configurable video coding, in which new or modified decoder designs are generated, transmitted and implemented by a Universal Video Decoder. We present initial results of a real-time software prototype.

A computational complexity control algorithm is proposed for an H.264 encoder running on a processor/power constrained platform. This new computational complexity control algorithm is based on a macroblock mode prediction algorithm that employs a Bayesian framework for accurate early skip decision. Complexity control is achieved by relaxing the Bayesian maximum-likelihood (ML) criterion in order to match the mode decision threshold to a target complexity level. A feedback algorithm is used to maintain the performance of the algorithm with respect to achieving an average target complexity level, reducing frame by frame complexity variance and optimizing rate-distortion performance. Experimental results show that this algorithm can effectively control the encoding computational complexity while maintaining a good rate-distortion performance at a range of target complexity levels.

This paper examines the computational and rate-distortion performance of a reconfigurable video codec. Dynamic configuration of a video codec has the potential for increased flexibility and rate distortion performance, through enabling adaptation of the coding algorithm to suit sequence statistics or operating conditions. This adaptability may be achieved by sending configuration information to the decoder during a communications session as part of the compressed video signal. This paper presents an example of dynamic re-configurability by sending descriptions of new transforms during coding. A decoder is reconfigured on-the- fly in order to incorporate a new transform, previously unknown to the decoder. We show that a reconfigurable video decoder may be implemented without significant computational penalties and we demonstrate the rate-distortion benefits of dynamic re-configuration.

The current paradigm of fixed, standardized video coding algorithms limits the performance and adaptability of video codecs. Enabling dynamic adaptation of a video coding algorithm has the potential to offer improved rate distortion performance and the ability to rapidly implement changes in coding tools and techniques. We present an example of dynamic adaptation based on the H.264/AVC framework in which new transforms may be communicated to, and implemented by, a video decoder, during a coding session. Experimental results demonstrate that dynamic reconfiguration can give improved rate-distortion performance for certain combinations of video sequence and bitrate. We conclude that this is a promising avenue for further research, with significant potential benefits compared with fixed coding strategies.

The H.264 video coding standard supports efficient coding of video at the expense of high computational complexity. This work addresses the problem of maintaining acceptable video coding performance in a computation-constrained application scenario. A complexity management approach is proposed for an H.264 encoder running in a processor/power-constrained environment. We hypothesize that, in a power-constrained application such as mobile video telephony, good perceptual quality requires a balance between a high frame rate and acceptable image quality. Therefore, the objective of the complexity management approach is to maintain a smooth video frame rate whilst ensuring that the frame quality is not degraded unacceptably. A frame-level algorithm calculates a target coding time for each frame and drops frames when necessary to maintain acceptable image quality. A per-frame algorithm controls the coding complexity of each frame in order to achieve the target coding time. The performance of the approach is evaluated by carrying out subjective tests and comparing the managed complexity encoder with a reference encoder in a computation-constrained scenario. Subjective results show that the managed complexity encoder consistently achieves superior perceptual video quality ratings compared to the reference encoder.

While the H.264 standard offers improved compression efficiency compared with prior video coding standards, this efficiency arises at the cost of significant complexity. We present a reduced complexity coding algorithm which estimates, prior to coding each macroblock, whether the coder would choose the skip or code mode for the macroblock. Computation savings are achieved, since pre-coding of skipped macroblocks is avoided. The decision to skip a macroblock is based on estimating and modelling mode cost differences, and employing these models in a MAP framework. Results are shown which indicate that for low-activity sequences savings of over 70% in computation time can be achieved with little or no decrease in video quality.

A complexity reduction algorithm for an H.264 encoder is proposed. Computational savings are achieved by identifying, prior to motion estimation, macroblocks (MBs) that are likely to be skipped and hence saving further computational processing of these MBs. This early prediction is made by estimating a Lagrangian rate-distortion cost function which incorporates an adaptive model for the Lagrange multiplier parameter based on local sequence statistics. Simulation results demonstrate that the algorithm can achieve computational savings of 19%-67% (depending on the source sequence) with no significant loss of rate-distortion performance.

DCT-based CODECs such as MPEG-4 Visual and H.263 "skip" many macroblocks during encoding of typical video sequences (i.e. no coded data is transmitted for these macroblocks). This paper describes an algorithm that predicts macroblocks that are likely to be skipped by the encoder. The algorithm estimates the increase in distortion due to not coding (skipping) each macroblock. Macroblocks that can be skipped with little or no increase in distortion are not coded, resulting in substantial computational savings without significantly affecting rate-distortion performance.

Summary form only given. The performance of software-only video codecs is often constrained by available processing power. Existing fast motion estimation algorithms are not designed to provide flexible, predictable control of computational complexity. We propose an adaptive algorithm, which maintains the computational complexity of the motion estimation function at various target levels by controlling the motion estimation search pattern.

The multimedia capability of video telephony and video conferencing systems has many applications and benefits. This paper describes research and development that aims to optimise video compression systems for a specific application - personal communication at a distance for deaf people. Results of eye movement tracking experiments and proposals for image content prioritisation based on these results are presented. The requirement for an appropriate quality assessment methodology is also addressed.

Computational complexity is an important performance constraint for software-only video CODECs. The aim of this research is to develop a video coding system with variable, controllable computational complexity. Adaptive algorithms for DCT and motion estimation are proposed separately to reduce complexity of each function and maintain it at target level. An integrated approach to video CODEC complexity management is also addressed. This work will have potential benefit for a wide range of computation-constrained or power-constrained multimedia applications.

Variable-complexity algorithms provide a means of managing the computational complexity of a software video CODEC. The reduction in computational complexity provided by existing variable-complexity algorithms depends on the video scene characteristics and is difficult to predict. A new approach to variable-complexity encoding is proposed. A variable-complexity DCT algorithm is adaptively updated in order to maintain a near-constant computational complexity. The adaptive update algorithm is shown to be capable of providing a significant, predictable, reduction in computational complexity with only a small loss of video quality. The proposed approach may be particularly useful for software-only video encoding, in applications where processing resources are limited.