Fully configurable video coding is a novel approach to video compression in which a video decoder can be dynamically constructed based on a decoder description. This paper presents a new syntax for describing video decoder functionality and structure, namely, the decoder description syntax (DDS). The DDS is a platform-independent syntax that can define all aspects of a video decoder in terms of basic processing instructions. A DDS description of a video decoder may be coded and transmitted, and then executed by a generic processing platform, a “universal video decoder (UVD).” Any new or modified video decoding function may be described, communicated, and instantiated using the DDS, which makes it possible to rapidly implement new coding algorithms, to dynamically adapt the coding algorithm to suit the video data, and to efficiently implement multiple coding formats on one platform. We present examples of video decoding functions implemented in the DDS and show how these may be executed on a UVD. We demonstrate that flexible configuration, re-configuration, and decoding of video can be achieved using a real-time prototype UVD with functions transmitted and instantiated on-the-fly using the DDS.

Fully configurable video coding is a novel approach to video compression in which a video decoder can be dynamically constructed based on a decoder description. This paper presents a new syntax for describing video decoder functionality and structure, namely, the decoder description syntax (DDS). The DDS is a platform-independent syntax that can define all aspects of a video decoder in terms of basic processing instructions. We present examples of video decoding functions implemented in the DDS and show that flexible configuration, reconfiguration, and decoding of video can be achieved using a real-time prototype universal video decoder.

A codec that can be re-configured dynamically has the potential to offer greater rate-distortion performance than fixed codecs by adapting video coding algorithms depending on local statistics and varying system resources. In addition, complete, dynamic configuration of a video decoder makes it possible to rapidly implement new algorithms and efficiently support multiple coding formats on a single platform. This paper describes a framework for dynamic re-configuration of video codecs, fully configurable video coding, in which new or modified decoder designs are generated, transmitted, and then implemented by a universal video decoder. We present rate-distortion results generated by a real-time prototype of this new approach.

Video coding solutions based on fixed, pre-defined video coding standards have certain limitations. Dynamic reconfiguration has the potential to offer greater rate-distortion performance than fixed codecs, making it possible to rapidly implement new algorithms and efficiently support multiple coding formats on a single platform. In this paper we describe a framework for dynamic re-configuration of video codecs, fully configurable video coding, in which new or modified decoder designs are generated, transmitted and implemented by a Universal Video Decoder. We present initial results of a real-time software prototype.

A computational complexity control algorithm is proposed for an H.264 encoder running on a processor/power constrained platform. This new computational complexity control algorithm is based on a macroblock mode prediction algorithm that employs a Bayesian framework for accurate early skip decision. Complexity control is achieved by relaxing the Bayesian maximum-likelihood (ML) criterion in order to match the mode decision threshold to a target complexity level. A feedback algorithm is used to maintain the performance of the algorithm with respect to achieving an average target complexity level, reducing frame by frame complexity variance and optimizing rate-distortion performance. Experimental results show that this algorithm can effectively control the encoding computational complexity while maintaining a good rate-distortion performance at a range of target complexity levels.

This paper examines the computational and rate-distortion performance of a reconfigurable video codec. Dynamic configuration of a video codec has the potential for increased flexibility and rate distortion performance, through enabling adaptation of the coding algorithm to suit sequence statistics or operating conditions. This adaptability may be achieved by sending configuration information to the decoder during a communications session as part of the compressed video signal. This paper presents an example of dynamic re-configurability by sending descriptions of new transforms during coding. A decoder is reconfigured on-the- fly in order to incorporate a new transform, previously unknown to the decoder. We show that a reconfigurable video decoder may be implemented without significant computational penalties and we demonstrate the rate-distortion benefits of dynamic re-configuration.

The current paradigm of fixed, standardized video coding algorithms limits the performance and adaptability of video codecs. Enabling dynamic adaptation of a video coding algorithm has the potential to offer improved rate distortion performance and the ability to rapidly implement changes in coding tools and techniques. We present an example of dynamic adaptation based on the H.264/AVC framework in which new transforms may be communicated to, and implemented by, a video decoder, during a coding session. Experimental results demonstrate that dynamic reconfiguration can give improved rate-distortion performance for certain combinations of video sequence and bitrate. We conclude that this is a promising avenue for further research, with significant potential benefits compared with fixed coding strategies.

In this paper, two unsupervised methods for multi-band change detection are presented. Both methods model the multi-band difference image histogram in order to characterize different degrees of observed spectral changes. In the first approach, we extend the single-band change detection algorithm proposed by Prieto and Bruzzone in which a two-component mixture density is fit to the observed difference image histogram, where the components correspond to the changed and unchanged populations. The second approach employs the hierarchical modal associative clustering algorithm proposed by Li et al., in which a hierarchy of kernel densities at different bandwidths is employed to model the multi-band difference image histogram. The kernel density modes correspond to different scales of changes and are analyzed with respect to increasing kernel bandwidth so that changes occurring at different scales may be identified. Experiments, carried out on ASTER data; are conducted to display the changes captured by each method as well as to illustrate how the degrees of detected changes can be interpreted with respect to model complexity or scale.

We propose a method for object-level change interpretation which we apply to remotely-sensed multispectral imagery. The method employs a model-based change description determined by a previously-developed change understanding system [3] in which object change is expressed by estimated model parameters. In this work, object change is interpreted using a physical description of the model parameter changes and the evaluation of object change model probabilities and error objects. Experiments are performed on multispectral ASTER imagery to analyze the proposed change interpretation mechanisms.

As adoption of the newly standardized H.264 becomes increasingly more widespread, efforts must be made to transcode video from earlier standards, such as MPEG-2, to the H.264 format. One aspect of transcoding is the conversion from the discrete cosine transform (DCT) used in MPEG-2 to the integer-based DCT of H.264, the integer cosine transform. By investigating the structure of the transform kernels, two efficient schemes are introduced to convert an 8 times 8 DCT block into its corresponding four 4times4 integer cosine transform block8 times 8 DCT block into its corresponding four 4times4 integer cosine transform blockss. An analysis of computational complexity shows a reduction of up to 42% in multiplications over existing conversion schemes, while simulations of the conversion system with pre- and post-quantization show that the proposed methods yield high fidelity in terms of peak signal-to-noise ratio. Further approximation of matrix coefficients in the second proposed method yields additional conversion schemes which require no multiplications. These also provide comparable quality with fewer operations than existing schemes. Thus, the proposed transform conversion methods can potentially contribute to efficient transcoding systems.

Variants of the discrete cosine transform (DCT) have been selected for different image/video compression standards, and fast conversion algorithms between these transforms are needed. In this paper, conversion between an 8×8 DCT block, used in MPEG-2, and its corresponding 4×4 integer DCT blocks, used in H.264, is studied. By investigating the structure of the two transform kernels, a DCT decomposition scheme is adopted in development of the fast conversion algorithm. Analysis of the performance of this conversion scheme shows that the proposed scheme provides over 45% computation savings on multiplications, as compared to an existing scheme, while maintaining high fidelity in terms of PSNR.

While the H.264 standard offers improved compression efficiency compared with prior video coding standards, this efficiency arises at the cost of significant complexity. We present a reduced complexity coding algorithm which estimates, prior to coding each macroblock, whether the coder would choose the skip or code mode for the macroblock. Computation savings are achieved, since pre-coding of skipped macroblocks is avoided. The decision to skip a macroblock is based on estimating and modelling mode cost differences, and employing these models in a MAP framework. Results are shown which indicate that for low-activity sequences savings of over 70% in computation time can be achieved with little or no decrease in video quality.

Correct reception of synchronization patterns aperiodically embedded in random data is vital for robust data transmission. In this letter, we derive general metrics and high-signal-to-noise ratio approximations to the metrics for random-length frames transmitted over Rayleigh fading channels. Simulation results are presented for commonly used modulation schemes with coherent and noncoherent demodulation

A complexity reduction algorithm for an H.264 encoder is proposed. Computational savings are achieved by identifying, prior to motion estimation, macroblocks (MBs) that are likely to be skipped and hence saving further computational processing of these MBs. This early prediction is made by estimating a Lagrangian rate-distortion cost function which incorporates an adaptive model for the Lagrange multiplier parameter based on local sequence statistics. Simulation results demonstrate that the algorithm can achieve computational savings of 19%-67% (depending on the source sequence) with no significant loss of rate-distortion performance.

For sketch-based retrieval of images, characterization and comparison of the geometrical structure (shape) information is of great importance. In this paper, a directional-projection-based sketch image retrieval system (DPSIR) is proposed. In the DPSIR system, efficient filter banks are adopted to decompose the image into directional subbands in order to capture edges at different orientations. Through projection of each subband into a pair of orthogonal profiles, the shape information is compactly captured by identifying the structure of each one-dimensional projection profile. By comparing the projection profiles of the sketched image with those in an image database, a set of images with shapes similar to the sketched images are retrieved. Experiments using the DPSIR system on an MPEG-7 shape database yield high retrieval rates.

Two statistical models for color texture retrieval based on a hidden Markov model (HMM) in the contourlet domain are described in this paper. Through a contourlet transformation, each color component of an image is decomposed into a set of directional subbands with texture details captured in different orientations. By exploiting inter-scale dependencies and in-band spatial dependencies, the distribution of the coefficients in each subband triplet (subbands of three color components at the same scale with the same orientation) can be estimated using a vector hidden Markov model. The Kullback-Leibler distance (KLD) is used to measure the difference between the distributions of query texture images and those of images in the database. The experimental results show the proposed retrieval systems yield high retrieval rates and better visual quality as compared with previous methods employing hidden Markov models for luminance component alone.

While serial concatenated codes were designed to provide good overall performance with reasonable system complexity, they may arise naturally in certain cases, such as the interface between two networks. In this work we consider the problem of constrained rate allocation between nonsystematic block codes in a serial concatenated coding system with either ideal or no interleaving between the codes. Given constraints on system parameters, such as a limit on the overall rate, analytic guidelines for the selection of good inner code rates are found by using an upper bound on the average system block error rate.

In this work we compare nonscalable video coding with data partitioning using H.264/AVC under similar application and channel constraints for conversational applications over mobile channels. For both systems optimized rate allocation and network feedback has been applied. From the experimental results it is observed that based on the average PSNR the nonscalable system outperforms the data partitioning system. However, with the data partitioning system the percentage of entirely lost frames can be lowered, and the probability of poor quality decoded video can be reduced.

A general joint source-channel rate allocation scheme for a dependent video coding environment is presented. It is assumed that the source coders rely on motion-compensated prediction, and thus error propagation will contribute significantly to the degradation of the reconstructed video. The rate allocation methodology is based upon generation of operational distortion-rate characteristics. In order to reduce computational complexity, these surfaces and the channel code performance are modeled. An analytic method for computing optimal rate allocation across frames in a video sequence is then introduced. Comparisons are made between equal error protection and unequal error protection over video frames. Results are shown for H.263 compressed video used in conjunction with rate-compatible punctured convolutional codes.

Synchronization patterns aperiodically embedded in data may result from applications such as transmission of framed data and compression of video. In this letter, a metric for detection of these patterns is derived. It is assumed that the distribution of the synchronization patterns in the symbol stream is known. By taking advantage of this knowledge, improved synchronization can be realized over traditional correlation with correction. Transmission using binary phase shift keying (BPSK) on an additive white Gaussian noise channel is considered first. This derivation is then extended to higher-order modulation schemes, and high-signal-to-noise ratio approximations to the exact metrics are derived. Experimental results for BPSK and quarternary phase shift keying signaling are shown. Finally, the effects of inexact knowledge of the channel and a priori probabilities are examined, and it is shown that the aperiodic metric is not sensitive to small deviations in the estimates of the channel or the synchronization pattern probabilities.

A form of combined source-channel coding is investigated in which error resilient source codes replace the typical concatenation of compression with channel coding. When very low bit rate transmission over poor quality channels, low coding complexity, or partial decoding of the source is required, these codes can outperform the traditional concatenated schemes. A simple, although suboptimal, technique is given for constructing the codes and a code-specific bound on performance is given. Simulation results are shown for a number of error resilient codes.

Typically, spectral efficiency, the throughput in bits/second/hertz, has been evaluated for fixed values of the bit error probability. We introduce a cutoff-rate-based approach to evaluation of the spectral efficiency of direct-sequence spread-spectrum schemes so that the results are independent of specific bit error rates or channel codes. Results are shown for M-ary phase-shift keyed modulation on the slow-fading Rician channel.

Motivated by previous results in joint source-channel coding and decoding, we consider the problem of decoding of variable-length codes using soft channel values. We present results of decoding of selected codes using the maximum a posteriori (MAP) decoder and the sequential decoder, and show the performance gains over decoding using hard decisions alone. The objective behind this work is to provide motivation for decoding of data compressed by standard source coding schemes, that is, to view the compressed bitstreams as being the output of variable-length coders and to make use of the redundancy in the bitstreams to assist in decoding. In order to illustrate the performance achievable by soft decoding, we provide results for decoding of MPEG-4 reversible variable-length codes as well as for decoding of MPEG-4 overhead information, under the assumption that this information is transmitted without channel coding over an additive white Gaussian noise channel. Finally, we present a method of unequal error protection for an MPEG-4 bitstream using the MAP and sequential source decoders, and show results comparable to those achievable by serial application of source and channel coding.

We present the results of two methods for soft decoding of variable-length codes. We first show that maximum likelihood (ML) sequential decoding and maximum a posteriori (MAP) sequence estimation gives significant decoding improvements over hard decisions alone, then we show that further improvements can be gained by additional transmission of the symbol length. Finally, we show that it is possible to make use of the inherent meaning of the codewords without additional transmission of side information which results in a further gain.

We consider the problem of locating a known periodically embedded synchronization pattern in random data, which is subject to flat fading and additive white Gaussian noise. We derive the maximum likelihood rule and its high-SNR approximation for the case of noncoherent demodulation and show experimental results for the slow-fading Rayleigh channel. It is shown that the well-known correlation rule performs substantially worse than maximum-likelihood rule derived here.

We present a method for modeling the distortion-rate characteristics of compressed video sequences. It is assumed that the source coders rely on motion-compensated prediction, and thus error propagation will contribute significantly to the degradation of the reconstructed video. It is shown that through the use of the modeled characteristics source and channel coding rate can be allocated across frames in a video sequence in order to minimize the effects of error propagation.

Often compressed video can be separated into high- and low-priority data on the basis of the contribution of the data to the quality of the reconstructed video. There has been significant interest in unequal error protection (UEP) for robust video transmission. Traditional UEP methods protect the high-priority data more highly than the low-priority data, that is, they employ less powerful channel codes on the low-priority data. In this work a method of utilizing soft decoding of variable-length source codes for unequal error protection is presented. No channel coding is used on the high-priority data, instead, soft-decision decoding is employed to recover from channel errors. Results are shown using an MPEG-4 codec. These results are subjectively comparable to those obtained by using a traditional UEP method.

There has been an increased interest in the transmission of digital video over real-world transmission media, such as the direct broadcast satellite (DBS) channel. Video transmitted over such a channel is subject to degradation due, in part, to additive white Gaussian noise (AWGN). Some form of forward error-control (FEC) coding may be applied in order to reduce the effect of the noise on the transmitted bitstream; however, determination of the appropriate level of FEC coding is generally an unwieldy and computationally intensive problem, as it may depend upon a variety of parameters such as the type of video, the available bandwidth, and the channel SNR. More specifically, a combined source-channel coding approach is necessary in optimally allocating rate between source and channel coding subject to a fixed constraint on overall transmission bandwidth. In this paper we develop a method of optimal bit allocation under the assumption that the distortion is additive and independent on a frame-by-frame basis. A set of universal operational distortion-rate characteristics is developed which balances the tradeoff between source coding accuracy and channel error protection for a fixed overall transmission rate and provides the basis for the optimal bit allocation approach. The results for specific source and channel coding schemes show marked improvement over suboptimum choices of channel error protection. In addition, we show that our results approach information-theoretic performance bounds which are developed in this work.

In the original paper (see ibid., vol.9, no.6, p.868-81, Sept. 1999) some of the figures did not appear as intended by the authors. The correct versions of the figures are given here.

Two hybrid error concealment schemes based on application of forward error control (FEC) coding in combination with passive error concealment techniques to different layers of coded data in a video transmission system are evaluated. It is shown that use of passive error concealment schemes alone generally does not provide sufficient protection against channel errors, especially in the case of video sequences with significant motion or scene changes. Motivated by these observations, a third hybrid scheme is introduced; this scheme involves adaptive application of FEC codes to high-priority information depending on the level of motion in a sequence. It is shown that through application of this adaptive motion-based scheme, significant savings in bandwidth may be obtained at the expense of only a negligible decrease in reconstructed image quality. Implementation of adaptive motion-based FEC coding in a multiresolution asynchronous transfer mode framework is discussed, and metrics for computing motion in both a subband and an MPEG-2 coding system are introduced.

In this work we consider the problem of combined source-channel coding for H.263 coded video transmitted over a wireless channel which is modeled as a slow-fading Rician channel. As wireless channels are generally bandwidth limited, the specific channel coding scheme employed here is non-binary bandwidth-efficient trellis-coded modulation (TCM) employing appropriate phase-shift keyed modulation. We describe a methodology for determining the optimal tradeoff between source and channel coding as a function of channel characteristics. In particular, if feedback on channel parameters is available, the video information can be optimally coded. In the case where feedback is not available, we illustrate the effects of using mismatched codes designed for other than the existing channel conditions.