Binocular rivalry, a challenging visual phenomenon occurs frequently for dissimilar monocular stimuli in 3D video communication particularly in error concealment. The quality assessment of distorted 3D video is a crucial element in designing and arranging advanced immersive media distribution platforms. In this paper we propose a quality assessment (QA) criterion that can be measured by considering the correlation of binocular rivalry impact. At first, subjective assessment is done for different frame freezing stereoscopic error concealment techniques and is compared with 2D to approximate the binocular rivalry characteristics. Secondly, a QA approach is proposed based on the perceptual characteristics of the Human Visual System (HVS). Finally, by incorporating the approximation with the perceptual measure, an assessment metric is generated for the transmitted 3D videos. Experimental analysis with common video datasets and comparison with different algorithms shows the efficiency of the proposed algorithm for video that is affected by asymmetric distortion and so is subject to binocular rivalry for stereoscopic views.

Broadcasting of high definition stereoscopic 3D videos is growing rapidly because of increased demand in the mass consumer market. In spite of increasing consumer interest, poor quality, crosstalk or side effects and lack of defined broadcast standards has hampered the advancement of 3D displays. Real time transmission of 3DTV sequences over packet based networks may result in visual quality degradation due to packet loss and delay. In conventional 2D videos, different extrapolation and directional interpolation strategies have been used for concealing the missing blocks but in 3D, this is still an emerging field of research. Also, subjective testing is the most direct way to evaluate 3D quality and human perception on the concealed videos. This paper reviews state-of-the-art error concealment strategies and proposes a low complexity frame loss concealment method for a video decoder. Subjective testing on common 3D video sequences and its statistical comparison with existing concealment methods shows that the proposed method is efficient for the concealment of error frames of stereoscopic videos in terms of visual comfort and 3D quality.

This paper introduces a new approach to motion compensated prediction for video compression. A two-way motion compensation approach is utilized in which the motion compensated prediction is generated either from a super resolution mosaic accumulated using a number of previously decoded frames or from a low-resolution reference frame. The super resolution mosaic is constructed using our improved global motion estimation technique. Experimental results show that the two-way motion compensation provides a better coding gain compared with a standard motion compensation approach and moreover, in the process of super resolution mosaic construction, our robust global motion estimation process outperforms other recent motion estimation techniques when utilizing the same compression framework.

Motion-compensated prediction is the key to high-performance video coding. Previous works have explored alternatives to the classical translational motion model in video coding, but the cumulative rate-distortion performance has not been significant enough to see such approaches adopted in mainstream standards. In this paper, we propose a new extended prediction strategy that incorporates non-translational motion prediction. This method uses an elastic motion model with 2-D cosine basis functions to estimate non-translational motion between the blocks. To achieve superior performance, the proposed scheme takes advantage of larger blocks with multi-level partitioning. Experimental results show that this combined framework outperforms the existing techniques, including those available in the recent H.264 standard.

In the most common form of wavelet video coding, a three-dimensional discrete wavelet transform (DWT) is applied on a group of video frames (GOF). This approach suffers in compression performance because no motion compensation is used. Other approaches to motion compensated wavelet video coding fails to achieve good video quality in packet loss network due to the absence of a resilient transmission method for the motion data. This paper describes a resilient transmission method for the motion parameters. A multiple description (MD) codec is used to encode the video for lossy network. Motion coding and transmission are incorporated in the MD framework. The resulting codec achieves good compression performance in lossless network as well as good resilience in unreliable network. Experimental results also show encouraging performance of the proposed codec compared to state-of-the-art video codec.

Resilient delivery of the motion parameters is critical in video transmission over unreliable networks. Several previous approaches to motion compensated wavelet video coding have failed to achieve good video quality due to the absence of a resilient transmission method for the motion vectors (MV). A codec employing a wavelet transform with no motion compensation is also not attractive, since it suffers from poor compression performance. This paper therefore investigates packetization methods for the motion parameters. A multiple description (MD) codec is used to encode the video for lossy networks. The motion packetization techniques are incorporated in the MD framework. We compare the performance of the investigated techniques in both lossless and lossy networks and conclude by proposing the best packet structure for motion transmission. Our proposed approach provides better performance than a recently proposed MD variant of H.264.

We investigate the problem of robust video transmission over lossy packet networks. A resilient video coding framework is important to ensure quality video over these networks. We propose a combination of a rate-distortion optimized multiple description codec and an integrated packetization method to constitute the error resilient codec. Multiple description (MD) coding techniques utilize path-diversity (multiple paths between sender and receiver) in networks by sending the descriptions along different paths. Two different optimization controls for the MD codec are proposed that are suited to variable rates of packet loss for multipath transmission of the MD coded video. An optimized strategy for packetizing the descriptions is also proposed which guarantees that each packet is self-contained and efficient. Simulations done under various packet loss scenarios show the need for two different optimization strategies and also that the developed MD codec achieves significantly improved video quality when compared with similar techniques.

Many video compression algorithms require decisions to be made to select between different coding modes. In the case of H.264, this includes decisions about whether or not motion compensation is used, and the block size to be used for motion compensation. It has been proposed that constrained optimization techniques, such as the method of Lagrange multipliers, can be used to trade off between the quality of the compressed video and the bit rate generated. In this paper, we show that in many cases of practical interest, very similar results can be achieved with much simpler optimizations. Mode selection by simply minimizing the distortion with motion vectors and header information produces very similar performance to the full constrained optimization, while it reduces the mode selection and over all encoding time by 31% and 12%, respectively. The proposed approach can be applied together with fast motion search algorithms and the mode filtering algorithms for further speed up.

We investigate the problem of robust video transmission over packet loss networks. A resilient video coding framework is important to ensure quality video over these networks. We propose a combination of multiple description coding and an optimized packetization method to constitute the error resilient codec. Multiple description coding technique utilizes path-diversity (multiple paths between sender and receiver) in networks by sending the descriptions along different paths. An optimized strategy for packetizing the descriptions is also proposed which guarantees that each packet is self-contained and efficient. Experimental results show that the proposed method achieves improved video quality over lossy networks.

The problem of resilient video transmission over lossy packet networks is addressed in this paper. We propose a rate-distortion optimized multiple description (MD) codec. Two different optimization controls of the codec are described that are suited to rates of packet loss, including the case where packets can travel over multiple paths through the network, with each path-dependent packet-loss probabilities. A packetization method optimized to work seamlessly with the proposed MD codec is also proposed. Simulations performed under various packet loss scenarios show the importance of the two optimizations and also that the proposed framework achieves significantly improved video quality when compared with similar techniques.

Motion estimation and compensation contribute greatly to video compression efficiency. Previous work has explored the effects of higher-order motion models with standard block sizes on video coding, but the impact on compression efficiency has not been sufficient to see such approaches adopted in mainstream video-compression standards. In this paper, we put forward a new strategy to augment the performance of extended motion models in the perspective of classical block-matching algorithms. To accomplish superior compression, the proposed scheme takes advantage of larger blocks with multi-level partitioning and a higher order elastic motion model. Performance evaluations show that these two features, when combined under a common framework, can significantly outperform the existing techniques including those available in the recent H.264 standard.

The efficiency of real-time digital image processing operations has an important impact on the cost and realizability of complex algorithms. Global motion estimation is an example of such a complex algorithm. Most digital image processing is carried out with a precision of 8 bits per pixel, however there has always been interest in low-complexity algorithms. One way of achieving low complexity is through low precision, such as might be achieved by quantization of each pixel to a single bit. Previous approaches to one-bit motion estimation have achieved quantization through a combination of spatial filtering/averaging and threshold setting. In this paper we present a generalized framework for precision reduction. Motivated by this framework, we show that bit-plane selection provides higher performance, with lower complexity, than conventional approaches to quantization.

We describe a multiple description (MD) video coding technique using 3D-set partitioning in hierarchical tree algorithm (3D-SPIHT). Multiple description coding (MDC) is a way of coding video information into multiple bitstreams and is useful in a number of scenarios including peer-to-peer video streaming and video transmission over error-prone networks. We propose a modified tree structure for 3D-SPIHT that is more efficient for MD coding. We also propose a branch-pruning technique to generate multiple descriptions. We then show how rate-distortion optimization can be incorporated into the MDC framework to develop an efficient MD video coding system. Experimental results show that the proposed MD system achieves significant improvement in performance compared to existing methods.

Shape masks are used in object-based video coding, such as MPEG-4 part 2, to specify the boundaries of a video object. The term ldquobinary shaperdquo is used to describe shape masks where each pixel is either completely inside the object or completely outside it, i.e., there is no blending of pixels at object boundaries. In this paper, we present a new lossless block-based coding algorithm for binary shapes that combines a quad-tree structure with context-based arithmetic coding. Experimental results demonstrate that this new approach provides a saving in bits generated up to approximately 66% compared to the MPEG-4 binary shape coding algorithm.

In this paper we propose a new texture segmentation technique that produces segmentation results which more closely match the manual segmentation that would be performed by a human operator. To perform this type of segmentation, we propose a new texture feature based on the double dyadic dual-tree complex wavelet transform (D3T-CWT) which provides the ability to analyse a signal at and between dyadic scales. This new texture feature is invariant to shift, rotation and scale and hence can group the texture features in a single object (which may have different sizes and orientations) into a single more meaningful segment. When compared with other texture segmentation approaches, the proposed approach provides segmentation results which more closely match the semantically meaningful objects in the scene.

Multiple description coding (MDC) of video is a useful technique for robust video transmission over unreliable networks. MDC not only provides at least acceptable quality of video in error-prone network but it also efficiently utilizes the multipath nature of Internet and wireless networks. In this paper, multiple description video coding with 3-D set partitioning in hierarchical tree (3D-SPIHT) codec has been proposed. In particular, we propose branch-pruning technique to generate multiple descriptions. We then propose a new tree structure for 3D-SPIHT which is particularly efficient for MDC. Experimental results prove the effectiveness of the proposed method.

In object-based video coding, shape masks are used to specify the shape of a video object. The binary shape means each pixel is either background or object. In this paper, we present a new lossless block-based coding algorithm for binary shapes that combines a quad-tree structure with block-based arithmetic coding; an optimal pruning quadtree structure searching algorithm is proposed to further reduce the spatial redundancy of quad-tree blocks. Experimental results demonstrate that this new approach provides a saving in bits generated between approximately 40-75% compared to the MPEG-4 binary shape coding algorithm

In this paper we propose a method for extending the standard motion estimation algorithms used in video compression algorithms, to incorporate motion parameters that describe non-translational motion. This method uses an elastic image registration algorithm with two-dimensional discrete cosine basis functions to estimate the motion between block partitions in two video frames. A rate and a distortion are then determined for the elastic motion compensation and these are included in the mode decision calculations of the encoder algorithm. This allows the encoder to trade off bits to describe a more complex motion with bits to describe a larger residual error from a simpler translational motion model. Experimental results show an improvement in compression efficiency of 10-20% can be obtained with this enhanced motion compensation algorithm.

We investigate streaming video over Differentiated Services (Diffserv) networks that can provide a number of aggregated traffic classes with increasing quality guarantee. We propose a method to measure the loss impact of a video packet on the quality of the decoded video images. We show how the optimal Quality-of-Service (QoS) mapping from the video packets into a set of traffic classes depends on the loss rates of the classes and the pricing model, and we develop an algorithm to accurately find the optimal QoS mapping. The performance of our algorithm is evaluated through computer simulations and compares favorably to an existing algorithm.

In this paper, we propose a robust approach to super-resolution static sprite generation from multiple low-resolution images. Considering both short-term and long-term motion influences, a hybrid global motion estimation technique is first presented for sprite generation. An iterative super-resolution reconstruction algorithm is then proposed for the super-resolution sprite construction. This algorithm is robust to outliers and results in a sprite image with high visual quality. In addition, the generated super-resolution sprite can provide better reconstructed images than a conventional low-resolution sprite. Experimental results demonstrate the effectiveness of the proposed methods.

In this paper, we propose some extensions of an efficient gradient-based image registration method called the inverse compositional algorithm. Specifically, these extensions include cumulative multi-image registration and incorporations of illumination change and lens distortion correction. By combining these extensions, we propose efficient cumulative multi-image registration methods with illumination and lens distortion correction. It is shown that high efficiency can still be achieved for multiple images using the proposed methods. Some experimental results show the efficacy of the proposed methods.

In many scenarios where it is desirable to provide a number of different descriptions of a video sequence, the simulcasting of single-layer bitstreams is impractical due to the amount of data involved. The new technique "stream morphing" introduced in this paper exploits the redundancy that exists between different compressed descriptions of the same video sequence at different quality levels. This works to dramatically reduce the data rates required to achieve the same effect as simulcast. The results shown here indicate that an SNR scalable video system constructed using this principle outperforms the structures defined in MPEG-2 and the MPEG-4 Streaming Video Profile. Stream morphing has a number of additional advantages: no loss of subjective quality compared to single-layer bitstreams at the same peak SNR regardless of the number of layers involved (unlike the MPEG-2 approach), the ability to generate a scalable representation of a video sequence from a set of single-layer descriptions in a computationally inexpensive way, and compatibility with existing single-layer decoders via a simple preprocessor without the need for transcoding. Stream morphing is demonstrated here in the context of MPEG-4; however, it is compatible with other similar video coding standards.

It is well known that tile-boundary artifacts occur in wavelet-based lossy image coding. However, until now, their cause has not been understood well. In this paper, we show that boundary artifacts are an inescapable consequence of the usual methods used to choose tile size and the type of symmetric extension employed in a wavelet-based image decomposition system. This paper presents a novel method for reducing these tile-boundary artifacts. The method employs odd tile sizes (2N + 1 samples) rather than the conventional even tile sizes (2N samples). It is shown that, for the same bit rate, an image compressed using an odd tile length low-pass first (OTLPF) convention has significantly less boundary artifacts than an image compressed using even tile sizes. The OTLPF convention can also be incorporated into the JPEG 2000 image compression algorithm using extensions defined in Part 2 of this standard.

Differentiated services networks can only function effectively if an appropriate mix of traffic can be maintained in all service classes using a practical pricing model. Previous work has noted that the same video sequence being distributed to clients who are prepared to pay different amounts for the service results in traffic distributions that vary considerably between those clients. It is shown that this variation can be reduced by trading off coding distortion and the amount of protection given to packets. As a result, a simple pricing model provides stable traffic distributions over a wide range of network conditions and client budgets.

Image segmentation can be viewed as the process of classifying regions in a picture into groups with common properties (e.g., texture). A difficulty arising is that a common texture can be classified differently when viewed at different scales and rotated viewpoints. The paper presents a feature vector based on the DT-CWT (dual-tree complex wavelet transform) (Kingsbury, N., Applied and Computational Harmonic Anal., vol.10, p.234-53, 2001) that is invariant to scale and rotation. The promising image segmentation results (without cleaning misclassified regions) demonstrate the suitability of this feature vector in representing texture.

Stream morphing is a technique that allows for two or more video bitstreams coded from the same original material with different quality settings to be simulcast in an efficient way. This paper describes our second generation stream morphing system that is based on the H.264 standard rather than MPEG-4 as used in our original implementation. While the basic operation of stream morphing remains largely unchanged for H.264, a number of enhancements have been made. Firstly, enhancement layers are coded using the same low-complexity, adaptive arithmetic coder utilized by H.264 itself. Furthermore, a scalable motion model has been added which allows for the use of very low bit rates in the base layer compared to the previous approach that used the same motion vectors and macroblock partitioning modes in all layers.

The reliance of many video processing techniques on motion estimation requires a good motion confidence measure, in order to ensure that unreliable estimations are not accepted into a system. This paper proposes a new measure of motion confidence based on phase information from Fourier transform of a video frame. Using the measure, blocks associated with unreliable motion estimation are detected efficiently and accurately. The merit of the technique is a much improved sensitivity over circumstances where other measures show ambiguity. This improvement is demonstrated through our experimental and analytical results.

We investigate streaming packet video over relative Differentiated Services (DiffServ) networks which can provide a number of aggregated traffic classes, ordered in a way such that class q+l is better or at least no worse than class q in terms of packet loss. We propose an algorithm for optimal quality of service (QoS) mapping from the video packets to a set of available DiffServ classes. The performance of our algorithm is evaluated through experimental tests and compares favorably to previous works.

Stream morphing has been previously introduced as an efficient way of simulcasting a hierarchy of single-layer descriptions of a single video sequence with different levels of quality while sharing the same frame size and rate. One application for this process was SNR scalable video coding. This paper presents two different adaptations of the stream morphing process to temporal scalability, one with bi-directional motion compensated prediction and one without. Both methods retain the attractive features of the original SNR scalable structure, notably the use of a single quantization cycle and single decoder motion-compensated prediction loop irrespective of the number of layers present and the ability to convert between single-layer and scalable forms without transcoding.

In this paper, we present a multisensor surveillance system that consists of an optical sensor and an infrared sensor. In this system, a background subtraction method based on zero-order statistics is utilized for the moving object segmentation. Additionally, we propose a generic approach to simultaneous object tracking and multisensor image registration. An efficient face detection system is shown as an application that will have enhanced performance from the registration and fusion of the information from the two sensors. Experimental results show the efficacy of the proposed system.

Stream morphing is a new approach to scalable video coding similar in scope to those already included in the MPEG standards. It works by exploiting redundancy between a hierarchy of single layer bitstreams of increasing quality. This paper provides an overview of the technique along with results for the constant quantizer SNR scalable case which demonstrate good performance, especially for the class of "low motion" video sequences. In addition to SNR scalability, stream morphing has also been applied to temporal and spatial scalability.

It is well known that tile boundary artefacts occur in wavelet-based lossy image coding. However, until now, their cause has not been well understood. We show that boundary artefacts are an intrinsic feature associated with the common method used to choose the tile size and the type of symmetric extension employed in wavelet-based image decomposition. A novel method of reducing tile boundary artefacts is presented. This method has recently been adopted as part of the JPEG 2000 verification model. In this technique, odd tile sizes of 2/sup N/+1 are chosen rather than the conventional even tile sizes of 2/sup N/. We show that, for the same bit-rate, an image compressed using odd tile length and low pass first convention (OTLPF) has significantly fewer boundary artefacts than an image compressed using even tile sizes.

In the real world, we commonly receive information simultaneously through two or more senses, with the brain fusing this data to produce a single coherent message. Lip-reading is one example of this phenomenon. Laboratory studies, on the other hand, often measure the response to a stimulus by a single sense and extrapolate these results to predict real-world behavior. In this paper, we show that semantics have a significant impact on viewers' sensitivity to the quality of a video sequence for spatially separated parts of the sequence and, more importantly, that this difference in sensitivity can be changed by the presence of an audio signal. This result is important for any testing of subjects' responses to visual material. One example is the subjective assessment of the quality of video in an audio-visual communications system (such as television or videoconferencing).

Signal-to-noise ratio (SNR) scalability has been incorporated into the MPEG-2 video-coding standard to allow for the delivery of two services with the same spatial and temporal resolution but different levels of quality. In this paper, we begin by reviewing the performance of a single-loop SNR scalable encoder that is compliant with the MPEG-2 standard and demonstrate that its performance is limited by drift in the base layer. We also look at an alternative single-loop drift-free noncompliant SNR scalable encoder, but discover that its coding efficiency is poor. We then review the performance of an MPEG-compliant two-loop SNR scalable encoder. Finally, we propose a new two-loop noncompliant encoder which achieves improved coding performance at the expense of some increase in encoder and decoder complexity.

It has been previously reported that variable bit rate video traffic displays signs of long-range dependence (Beran et al. 1995, and Garrett 1993), which is defined within the framework of weak stationarity. The concept of weak stationarity is of fundamental practical interest, because it implies the existence and independence with time of mean and autocovariance. Since mean and variance might be used as parameters for policing the traffic stream on a broadband communication network, it is important to verify the underlying assumption of weak stationarity. The results presented in this paper strongly call into question this assumption. As a possible explanation, we offer shifting level processes which are asymptotically weakly stationary for infinite sample sizes.

We describe a robust codec for the transmission of very low bit-rate video over channels with a variety of errors, including random and bursty bit errors and packet loss. The codec exploits adaptivity to give good performance with a low overhead. By only protecting macroblocks which would otherwise be poorly concealed by the decoder the codec allows adaptive selection of the parts of video to protect. For protection, it uses multiple description codes which indirectly provide frequency-based adaptivity by protecting the more significant DCT coefficients. Simulations show significant improvements in the performance of the codec when compared to codecs which use intra macroblock updating (raster scan and random) at the same overhead. The codec is efficient in its use of bits and has good error resilience properties both objectively and subjectively over a wide range of conditions. Further, transcoding of the received bit stream to the standard H.263 syntax is relatively easy.

Audio-visual and other multimedia services are seen as important sources of traffic for future telecommunication networks, including wireless networks. A major drawback with some wireless networks is that they introduce a significant number of transmission errors into the digital bitstream. For video, such errors can have the effect of degrading the quality of service to the point where it is unusable. We introduce a technique that allows for the concealment of the impact of these errors. Our work is based on MPEG-2 encoded video transmitted over a wireless network whose data structures are similar to those of asynchronous transfer mode (ATM) networks. Our simulations include the impact of the MPEG-2 systems layer and cover cell-loss rates up to 5%. This is substantially higher than those that have been discussed in the literature up to this time. We demonstrate that our new approach can significantly increase received video quality, but at the cost of a considerable computational overhead. We then extend our technique to allow for higher computational efficiency and demonstrate that a significant quality improvement is still possible.

In the real world, we commonly receive information simultaneously through two or more senses, with the brain fusing this data to produce a single-coherent message. Lip-reading is one example of this phenomenon. Laboratory studies, on the other hand often measure the response to a stimulus by a single sense and extrapolate these results to predict real-world behaviour. In this paper, we show that semantics have a significant impact on viewers' sensitivity to the quality of a video sequence for spatially separated parts of the sequence-and, more importantly, that this difference in sensitivity can be changed by the presence of an audio signal. This result is important for any testing of subjects' responses to visual material. One example is the subjective assessment of the quality of video-in an audio-visual communications system (such as television or video conferencing).

The protocols used to multiplex the various streams in an audio-visual service (such as MPEG 2 systems and ITU-T H.223) have an important impact on the quality of the service. The fact that wireless channels are often associated with high bit-error rates which are often bursty means that the multiplexing protocols should be designed to be robust against such errors. We suggest two departures from traditional practice which can improve significantly the performance of a packet-oriented multiplex for use with audio-visual services: (1) where the packet header is protected against errors using forward error correcting codes, the use of a synchronization codeword (or flag) is often unnecessary, and (2) for channels where errors tend to occur in bursts, the occurrence of errors in the header can be reduced by transmitting the header information twice (at both ends of the packet) with a low level of FEC protection instead of using the same number of bits to obtain an increased level of protection using a single code word. The value of these approaches is confirmed by simulations.

We describe a robust codec for the transmission of very low bit-rate video over channels with bursty errors. The codec uses a diversity-based method in the form of multiple description codes to reduce the effect of errors in the video bit-stream. To improve the efficiency for transmission of very low bit-rate video, the codec also exploits adaptivity by only protecting macroblocks (using multiple description codes) which would otherwise be poorly concealed by the decoder. Simulations show significant improvements in the performance of the codec when compared to codecs which use intra macroblock updating (raster scan and random) with the same overhead for reducing the effects of errors.

The transmission of audio-visual services on low-bit-rate, wireless telecommunications systems requires the use of coding techniques that are both efficient in their use of bits and robust against errors introduced in transmission. In this paper, we present efficient techniques for improving the error resilience of audio-visual services. These techniques are based on coding simultaneously for synchronization and error protection or detection. We apply the techniques to improve the performance of the multiplexing protocol (which combines the video and audio streams so that they can be transmitted on a single circuit), and also to improve the robustness of the coded video. We show through simulations that the techniques are efficient in their use of bits and effective against bursty errors common in wireless channels. For a simulation of the DECT channel at a bit-error rate of 10/sup -3/, the techniques give an order of magnitude improvement in the probability of lost packets in the multiplexer layer over more conventional techniques. In the video layer, the techniques give an improvement of between 1-2 dB over ITU-T Recommendation H.263. The techniques proposed for the video layer also have the advantage of permitting simple transcoding with bit streams complying with H.263.

Audio-visual and other multimedia services are seen as an important source of traffic for future telecommunications networks, including wireless networks. We examine the impact of the properties of a 50 Mb/s asynchronous transfer mode (ATM)-based wireless local-area network (WLAN) on Moving Picture Experts Group phase 2 (MPEG 2) compressed video traffic, with emphasis on the network's error characteristics. The paper includes a description of the WLAN system used and its loss characteristics, a brief discussion of relevant aspects of the MPEG 2 standards and the associated error resilience techniques for minimizing the effect of transmission errors, and a description of the method by which the video data is organized for transmission on the network. We show results on the effect of cell loss due to transmission errors on the quality of the decoded video at the receiver, and demonstrate how error resilience techniques in both the systems and video layers of MPEG 2 can be used to improve the quality of service. Situations where up to 1% of the data is lost due to network transmission errors are examined. Most important among the findings are that error resilience experiments that do not take into account the effect of the MPEG 2 systems layer will tend to significantly overestimate the quality of received video, and that the error resilience techniques provided within the MPEG 2 standard are not sufficient to provide acceptable quality with acceptable overheads, but that this quality can be significantly increased by the addition of a small number of simple techniques.

It is well known that layered video coding is useful both for service interworking and as an aid to error resilience. A major drawback of layered coding is that it invariably results in a reduction in overall coding efficiency for the high quality service. An attractive approach would be one in which the high quality service is coded, and then the resulting bitstream is decomposed in such a way that the lower resolution services can be reconstructed using only a subset of the total generated data. This would mean that there would be no impact on the coding efficiency of the highest quality service. In this paper we demonstrate that the quality of the lower layer when this approach is used is fundamentally limited by drift. It is shown that even a coarsely quantized, low rate correction signal provides a major benefit to the quality of this lower layer service while still not impacting significantly on the coding efficiency of the high quality service. Of course, some overhead is introduced in the coding of the lower quality service but the total overhead is still significantly better than simulcast.

This paper presents a new fast search algorithm for block matching motion estimation called the adaptive search length (ASL) algorithm. The ASL algorithm adaptively varies the number of positions searched for each block while still maintaining control of the average number of searches per block for each frame. Experimental results show that the peak signal-to-noise ratio (PSNR) of decoded sequences which were coded using the ASL algorithm is within 0.25 dB of the PSNR of decoded sequences which were coded using the full search block matching algorithm. It is also shown that the ASL algorithm requires only 10% of the computations required by the full search algorithm to achieve this level of decoded image quality.

The structure of hyperspectral images reveals spectral responses that would seem ideal candidates for compression by vector quantization. This paper outlines the results of an investigation of lossless vector quantization of 224-band Airborne/Visible Infrared imaging Spectrometer (AVIRIS) images. Various vector formation techniques are identified and suitable quantization parameters are investigated. A new technique, mean-normalized vector quantization (M-NVQ), is proposed which produces compression performances approaching the theoretical minimum compressed image entropy of 5 bits/pixel. Images are compressed from original image entropies of between 8.28 and 10.89 bits/pixel to between 4.83 and 5.90 bits/pixel.

A new fast search algorithm for block matching motion estimation is presented in this paper. The new algorithm, called the adaptive search length (ASL) algorithm, allows the maximum number of searches for each block of the image to vary according to the difficulty in finding the optimum motion vector. The PSNR of the decoded images produced by a video coder operating with the ASL algorithm were compared with those produced by a coder operating with the full search block matching algorithm. The results presented show that, for a PSNR of within .25 dB of the full search PSNR, the ASL algorithm requires only 10% of the searches required by the full search algorithm.

Many techniques have been proposed for modeling source rates generated by variable bit rate video coders. However, little work has been done to verify these models, especially in the area of measuring the quality of their predictions for network performance and the grade of service experienced by a customer. In this paper, we summarize the previous work in this area, highlight some of the difficulties in effectively verifying these models, and suggest methods by which source rate models for VBR video might be more effectively verified in the future. In addition to studying the problem of model verification, a new model is proposed. Using this model, a video sequence is characterized by four parameters. As in previous cases, the number of bits generated by the coder for a sequence of video frames is modelled. The new model differs from previous models in that this sequence is not Markovian.>

An examination of the distribution of signals within the prediction loop of an interframe video codec reveals that small random fluctuations caused by the quantizer build up over a period of time. This quantization noise circulates within the prediction loop causing a gradual decrease in the signal to noise ratio of the reconstructed sequence and an increase in the required data rate. In general, both of these effects can be nullified by the inclusion of a low pass loop filter.>

Reversible image compression rarely achieves compression ratios larger than about 3:1. An explanation of this limit is offered, which hinges upon the additive noise the sensor introduces into the image. Simple models of this noise allow lower bounds on the bit rate to be estimated from sensor noise parameters rather than from ensembles of typical images. The model predicts that an 8-b single-band image subject to noise with unit standard deviation can be compressed reversibly to no less than 2.0 b/pixel, equivalent to a maximum compression ratio of about 4:1. The model has been extended to multispectral imagery. The Airborne Visible and Infra Red Imaging Spectrometer (AVIRIS) is used as an example, as the noise in its 224 bands is well characterized. The model predicts a lower bound on the bit rate for the compressed data of about 5.5 b/pixel when a single codebook is used to encode all the bands. A separate codebook for each band (i.e., 224 codebooks) reduces this bound by 0.5 b/pixel to about 5.0 b/pixel, but 90% of this reduction is provided by only four codebooks. Empirical results corroborate these theoretical predictions.>

This paper describes a rate control algorithm for a variable bit-rate (VBR) video coder. The algorithm described varies the quantizer step size of the coder according to properties of an image sequence that affect the perception of errors. The algorithm also limits the output bit-rate of the coder without the use of buffers to more efficiently use network bandwidth. It is shown that a VBR encoder using this algorithm will provide decoded image sequences with a consistent perceived quality that is comparable with, or better than, the perceived quality of images coded with a CBR encoder.

An image-coding technique is described in which an adaptive quadtree scheme is used to encode motion-compensated difference images. A mechanism is determined within the encoding process that allows a predetermined level of image quality to be selected by the user at call setup time and then maintained by the encoder. The coder operates in a variable-bit-rate mode providing a video service of near-constant image quality. A multilayered version of the coding scheme is also described in which the degradation caused by cell loss during transmission can be limited in its effect and removed completely within a short period of time. The recovery from cell loss is achieved by sending blocks from the original image sequence whenever cells are known to have been deleted by the local-network access node.>

Two modifications to the constant area quantization (CAQ) video bandwidth compression scheme are described which together reduce the mean square error (MSE) in the reconstructed image by up to 70 percent. The first modification is merely a change in the CAQ algorithm requiring no additional hardware, whereas the second modification exploits line-to-line redundancies and necessitates the inclusion of one line of memory.