Binocular rivalry, a challenging visual phenomenon occurs frequently for dissimilar monocular stimuli in 3D video communication particularly in error concealment. The quality assessment of distorted 3D video is a crucial element in designing and arranging advanced immersive media distribution platforms. In this paper we propose a quality assessment (QA) criterion that can be measured by considering the correlation of binocular rivalry impact. At first, subjective assessment is done for different frame freezing stereoscopic error concealment techniques and is compared with 2D to approximate the binocular rivalry characteristics. Secondly, a QA approach is proposed based on the perceptual characteristics of the Human Visual System (HVS). Finally, by incorporating the approximation with the perceptual measure, an assessment metric is generated for the transmitted 3D videos. Experimental analysis with common video datasets and comparison with different algorithms shows the efficiency of the proposed algorithm for video that is affected by asymmetric distortion and so is subject to binocular rivalry for stereoscopic views.

Asynchronous design is predicted to have a significant place in the future due to benefits of speed, power consumption, and design. Null Convention Logic (NCL) is a subcategory of asynchronous design that results in the most reliable and low-power asynchronous hardware. However, test strategies are not adopted to match the characteristics of this important asynchronous method, and the existing test methods for NCL use synchronous test hardware. In this paper asynchronous test hardware is proposed for testing NCL-based circuits to avoid problems of synchronous design as well as the need for asynchronous-synchronous interfaces and clock trees. The proposed test hardware is implemented in HDL and synthesised to define design characteristic and improvements over previous work.

Broadcasting of high definition stereoscopic 3D videos is growing rapidly because of increased demand in the mass consumer market. In spite of increasing consumer interest, poor quality, crosstalk or side effects and lack of defined broadcast standards has hampered the advancement of 3D displays. Real time transmission of 3DTV sequences over packet based networks may result in visual quality degradation due to packet loss and delay. In conventional 2D videos, different extrapolation and directional interpolation strategies have been used for concealing the missing blocks but in 3D, this is still an emerging field of research. Also, subjective testing is the most direct way to evaluate 3D quality and human perception on the concealed videos. This paper reviews state-of-the-art error concealment strategies and proposes a low complexity frame loss concealment method for a video decoder. Subjective testing on common 3D video sequences and its statistical comparison with existing concealment methods shows that the proposed method is efficient for the concealment of error frames of stereoscopic videos in terms of visual comfort and 3D quality.

ABSTRACT For many applications, it is desirable for a node to be able to identify the neighbor nodes currently within range. However, the identification procedures for terrestrial communication networks (TCNs) are inefficient when implemented in long‐delay networks (LDNs) such as underwater acoustic networks or satellite networks, where the propagation time of a packet cannot be neglected. Here, we propose a time‐efficient and power‐efficient procedure, which is insensitive to propagation delay, for identifying neighbors in an LDN by optimizing the network offered load using either unslotted or slotted protocols. The procedure is adapted to the scenario when node cardinality and distribution are either known or unknown in advance. We achieve improvements as high as 80% in time and 45% in power consumption for our proposed approach compared with approaches developed previously for TCN. Furthermore, we analyze our proposed approach with the inclusion of the capture effect and packet receive time variations. We also provide a closed form formula for finding the optimum guard time and a procedure to estimate the packet receive time variations. Copyright © 2012 John Wiley & Sons, Ltd.

Immunology inspired computer security has attracted enormous attention as its potential impacts on the next generation service-oriented network operation system. In this paper, we propose a new agent-based threat awareness assessment strategy inspired by the human immune system to dynamically adapt against attacks. Specifically, this approach is based on the dynamic reconfiguration of the file access right for system calls or logs (e.g., file rewritability) with balanced adaptability and vulnerability. Based on an information-theoretic analysis on the coherently associations of adaptability, autonomy as well as vulnerability, a generic solution is suggested to break down their coherent links. The principle is to maximize context-situation awared systems' adaptability and reduce systems' vulnerability simultaneously. Experimental results show the efficiency of the proposed biological behaviour-inspired vulnerability awareness system.

In this paper, multiple detection engines with multi-layered intrusion detection mechanisms are proposed for enhancing computer security. The principle is to coordinate the results from each single-engine intrusion alert system, which seamlessly integrates with a multiple layered distributed service-oriented structure. An improved hidden Markov model (HMM) is created for the detection engine which is capable of the immunology-based self/nonself discrimination. The classifications of normal and abnormal behaviours of system calls are further examined by an advanced fuzzy-based inference process tuned by HPSOWM. Considering a real benchmark dataset from the public domain, our experimental results show that the proposed scheme can greatly shorten the training time of HMM and significantly reduce the false positive rate. The proposed HPSOWM works especially well for the efficient classification of unknown behaviors and malicious attacks.

An important recent application of image registration is the estimation of global motion parameters for object-based video coding. However, the main disadvantage of standard approaches to global motion estimation (GME) is the increased computational complexity with the higher degree of motion models when compared to block-based local motion estimation approaches. In this paper, we propose a new low complexity GME algorithm. In our proposed algorithm, full-precision images are replaced with 1 bit-per-pixel images which allows many of the arithmetic operations in the standard GME approach to be replaced with logic operations. Experimental results show that our proposed algorithm achieves the same registration accuracy as the standard GME approach but with significantly reduced computational complexity. Our results also demonstrate the superior performance of the proposed algorithm when compared with previously proposed low-complexity GME approaches.

In this paper, multiple detection engines with multi-layered intrusion detection mechanisms are proposed. The principle is to coordinate the results from each single-engine intrusion alert system, by seamlessly integrating with the multiple layered distributed service-oriented structure. An improved hidden Markov model (HMM) is created for the detection engine which is capable of the immunology-based self/nonself discrimination. The classifications of normal and abnormal behaviours of system calls are further examined by an advanced fuzzy-based inference process called HPSOWM. Considering a real benchmark dataset from the public domain, our experimental results show that the proposed scheme can greatly shorten the training time of HMM and reduce the false positive rate significantly. The proposed HPSOWM especially works for the efficient classification of unknown behaviors and malicious attacks.

The limited power consumption, as a major constraint, presents challenges in improving the network throughput for Wireless Sensor Networks (WSNs). Due to the limited computational power, the applications of WSNs in Terrestrial Networks require the capability to pre-process the observation data so as to remove irrelevant features or factors from multi-dimensional dataset. This paper proposes a intelligent distributed energy efficient routing algorithm inspired from natural learning and adaptation process with the aid of hybrid Neural Rough Sets theory, which is used to efficiently reduce the dimensionality of input dataset. The algorithmic implementation and experimental validation are described in this paper. Details of the algorithm and its testing procedures are presented in comparison with the other power-aware protocols, e.g., mini-hop. The validation of the proposed model is carried out via a wireless sensor network test-bed implemented in Castalia Simulator. The experimental results show the network performance measurements such as delay, throughput and packet loss that have been greatly improved as the outcome of applying this integration with Neural Rough Sets.

In this paper we propose a new approach to depth estimation which combines structured light and passive stereo disparity estimation techniques to generate accurate disparity maps in both textured and textureless areas of a scene. We project a structured light pattern with adaptive colors onto the scene and simultaneously capture stereo images with a pair of cameras. By matching points in the projected pattern and the left image we first acquire a sparse disparity map. This sparse disparity map is interpolated and used to initialize a passive stereo disparity algorithm to improve disparity accuracy in textured areas. Finally, this map is interpolated in the textureless areas. By comparing our final map with efficient be lief propagation and the initial interpolated disparity map, we show our approach performs better than using passive-only or active-only techniques.

In block-based video coding, the Discrete Cosine Transform (DCT) has been adopted for signal decorrelation in state-of-the-art standards. Although the Karhunen Loeve Transform (KLT) is known to achieve optimal energy compaction, it has been reported to offer only moderate compression as the KLT basis functions are source dependent and hence require the transform itself to be coded. This paper describes a technique for prediction-error block coding using the KLT. The proposed method does not require coding of the KLT bases. Instead the basis functions can be derived at the decoder in a manner similar to the encoder. The proposed method is incorporated into a standard H.264 video codec using an adaptive transform selection approach. Our experiments show that the Peak Signal-to-Noise Ratio (PSNR) improvement of up to 0.9 dB is achieved with the proposed technique when compared with the standard H.264 codec.

This paper introduces a new approach to motion compensated prediction for video compression. A two-way motion compensation approach is utilized in which the motion compensated prediction is generated either from a super resolution mosaic accumulated using a number of previously decoded frames or from a low-resolution reference frame. The super resolution mosaic is constructed using our improved global motion estimation technique. Experimental results show that the two-way motion compensation provides a better coding gain compared with a standard motion compensation approach and moreover, in the process of super resolution mosaic construction, our robust global motion estimation process outperforms other recent motion estimation techniques when utilizing the same compression framework.

One important recent application of image registration has been in the estimation of global motion parameters for object-based video coding. A limitation of current global motion estimation approaches is the additional complexity of the gradient-descent optimization that is typically required to calculate the optimal set of global motion parameters. In this paper we propose a new low-complexity algorithm for global motion estimation. The complexity of the proposed algorithm is reduced by performing the majority of the operations in the gradient-descent optimization using logic operations rather than full-precision arithmetic operations. This use of logic operations means that the algorithm can be implemented much more easily in hardware platforms such as field programmable gate arrays (FPGAs). Experimental results show that the execution time for software implementations of the new algorithm is reduced by a factor of almost four when compared to existing fast implementations without any significant loss in registration accuracy.

Motion-compensated prediction is the key to high-performance video coding. Previous works have explored alternatives to the classical translational motion model in video coding, but the cumulative rate-distortion performance has not been significant enough to see such approaches adopted in mainstream standards. In this paper, we propose a new extended prediction strategy that incorporates non-translational motion prediction. This method uses an elastic motion model with 2-D cosine basis functions to estimate non-translational motion between the blocks. To achieve superior performance, the proposed scheme takes advantage of larger blocks with multi-level partitioning. Experimental results show that this combined framework outperforms the existing techniques, including those available in the recent H.264 standard.

It is well known that video material with a static background allows easier segmentation than that with a moving background. One approach to segmentation of sequences with a moving background is to use preprocessing to create a static background, after which conventional background subtraction techniques can be used for segmenting foreground objects. It has been recently shown that global motion estimation and/or background sprite generation techniques are reliable. We propose a new background modeling technique for object segmentation using local background sprite generation. Experimental results show the excellent performance of this new method compared to recent algorithms proposed.

In the most common form of wavelet video coding, a three-dimensional discrete wavelet transform (DWT) is applied on a group of video frames (GOF). This approach suffers in compression performance because no motion compensation is used. Other approaches to motion compensated wavelet video coding fails to achieve good video quality in packet loss network due to the absence of a resilient transmission method for the motion data. This paper describes a resilient transmission method for the motion parameters. A multiple description (MD) codec is used to encode the video for lossy network. Motion coding and transmission are incorporated in the MD framework. The resulting codec achieves good compression performance in lossless network as well as good resilience in unreliable network. Experimental results also show encouraging performance of the proposed codec compared to state-of-the-art video codec.

Sprite-based video coding offers higher compression efficiency than conventional block-based hybrid video coders. In sprite coding a sequence is divided into a model of its background, i.e. a so-called background sprite image, and a foreground object sequence. These are then encoded and transmitted to the receiver. At the decoder the output video sequence is synthesized using content previously transmitted. The background information of the video sequence is, other than the foreground object sequence, encoded indirectly, meaning for a sequence with N frames one single image is encoded and not N background frames. The background for a single frame is reconstructed by coordinate transformation at the decoder. An important issue to address here is an appropriate rate-distortion technique for optimization of the sprite-based coding approach, since foreground and background are encoded independently. In this paper, the Lagrangian cost function is considered for rate-constrained encoder control. Experimental evaluation shows the correct use of the optimization method and the gain in terms of rate-distortion performance compared to H.264/AVC.

To progress the compression performance of standard video coding algorithms, emerging motion compensation techniques will need to be integrated with the current standard techniques such as those used in the H.264. Higher order motion models, geometry-adaptive partitioning and motion-assisted merging are such techniques that can be considered for the next generation of video coders. In this paper, we examine how geometry information can benefit the use of elastic motion models to accomplish better prediction. Relative complexity issues are also discussed which is important in the standardization process. Experimental results suggest that geometry-adaptive block partitioning can add to the performance of elastic motion models to a certain extent, although the increased complexity is of some concern for real-time coding applications.

Resilient delivery of the motion parameters is critical in video transmission over unreliable networks. Several previous approaches to motion compensated wavelet video coding have failed to achieve good video quality due to the absence of a resilient transmission method for the motion vectors (MV). A codec employing a wavelet transform with no motion compensation is also not attractive, since it suffers from poor compression performance. This paper therefore investigates packetization methods for the motion parameters. A multiple description (MD) codec is used to encode the video for lossy networks. The motion packetization techniques are incorporated in the MD framework. We compare the performance of the investigated techniques in both lossless and lossy networks and conclude by proposing the best packet structure for motion transmission. Our proposed approach provides better performance than a recently proposed MD variant of H.264.

State-of-the-art video compression standards such as H.264 employ tree-structured motion compensation by splitting macro-blocks into fixed square or rectangular sub-blocks. Although, this approach leads to improved compression performance, recent studies have shown that further gain can be achieved via slicing blocks with arbitrary line segments to better match the boundaries between moving objects. However, finding the best partition remains an extremely computationally-intensive task. In this paper, we propose a fast method to identify efficient partitions using a two-step search of the radius and angle of the line segment. Experimental results show that this scheme is able to identify efficient motion boundaries using less iterations than existing techniques while maintaining comparable performance.

We investigate the problem of robust video transmission over lossy packet networks. A resilient video coding framework is important to ensure quality video over these networks. We propose a combination of a rate-distortion optimized multiple description codec and an integrated packetization method to constitute the error resilient codec. Multiple description (MD) coding techniques utilize path-diversity (multiple paths between sender and receiver) in networks by sending the descriptions along different paths. Two different optimization controls for the MD codec are proposed that are suited to variable rates of packet loss for multipath transmission of the MD coded video. An optimized strategy for packetizing the descriptions is also proposed which guarantees that each packet is self-contained and efficient. Simulations done under various packet loss scenarios show the need for two different optimization strategies and also that the developed MD codec achieves significantly improved video quality when compared with similar techniques.

Many video compression algorithms require decisions to be made to select between different coding modes. In the case of H.264, this includes decisions about whether or not motion compensation is used, and the block size to be used for motion compensation. It has been proposed that constrained optimization techniques, such as the method of Lagrange multipliers, can be used to trade off between the quality of the compressed video and the bit rate generated. In this paper, we show that in many cases of practical interest, very similar results can be achieved with much simpler optimizations. Mode selection by simply minimizing the distortion with motion vectors and header information produces very similar performance to the full constrained optimization, while it reduces the mode selection and over all encoding time by 31% and 12%, respectively. The proposed approach can be applied together with fast motion search algorithms and the mode filtering algorithms for further speed up.

We investigate the problem of robust video transmission over packet loss networks. A resilient video coding framework is important to ensure quality video over these networks. We propose a combination of multiple description coding and an optimized packetization method to constitute the error resilient codec. Multiple description coding technique utilizes path-diversity (multiple paths between sender and receiver) in networks by sending the descriptions along different paths. An optimized strategy for packetizing the descriptions is also proposed which guarantees that each packet is self-contained and efficient. Experimental results show that the proposed method achieves improved video quality over lossy networks.

The problem of resilient video transmission over lossy packet networks is addressed in this paper. We propose a rate-distortion optimized multiple description (MD) codec. Two different optimization controls of the codec are described that are suited to rates of packet loss, including the case where packets can travel over multiple paths through the network, with each path-dependent packet-loss probabilities. A packetization method optimized to work seamlessly with the proposed MD codec is also proposed. Simulations performed under various packet loss scenarios show the importance of the two optimizations and also that the proposed framework achieves significantly improved video quality when compared with similar techniques.

Motion estimation and compensation contribute greatly to video compression efficiency. Previous work has explored the effects of higher-order motion models with standard block sizes on video coding, but the impact on compression efficiency has not been sufficient to see such approaches adopted in mainstream video-compression standards. In this paper, we put forward a new strategy to augment the performance of extended motion models in the perspective of classical block-matching algorithms. To accomplish superior compression, the proposed scheme takes advantage of larger blocks with multi-level partitioning and a higher order elastic motion model. Performance evaluations show that these two features, when combined under a common framework, can significantly outperform the existing techniques including those available in the recent H.264 standard.

In recent years, global motion estimation (GME) has become an important tool in the fields of video coding, video compression and computer vision. Estimating the correct global motion is often more difficult when the video scene contains large foreground objects. Previous approaches to addressing this problem have required the application of algorithm parameters that are sequence dependent or give inconsistent results for different video sequences. In this paper, we propose a fully automatic approach that can successfully estimate global motion in the presence of large foreground objects. The proposed algorithm determines an initial estimate of the foreground pixels and then reduces the effect of the remaining foreground by using a modified Lorentzian estimator. Experimental results show the proposed method produces superior and more consistent performance than some recent approaches for a wide range of sequences.

The efficiency of real-time digital image processing operations has an important impact on the cost and realizability of complex algorithms. Global motion estimation is an example of such a complex algorithm. Most digital image processing is carried out with a precision of 8 bits per pixel, however there has always been interest in low-complexity algorithms. One way of achieving low complexity is through low precision, such as might be achieved by quantization of each pixel to a single bit. Previous approaches to one-bit motion estimation have achieved quantization through a combination of spatial filtering/averaging and threshold setting. In this paper we present a generalized framework for precision reduction. Motivated by this framework, we show that bit-plane selection provides higher performance, with lower complexity, than conventional approaches to quantization.

We describe a multiple description (MD) video coding technique using 3D-set partitioning in hierarchical tree algorithm (3D-SPIHT). Multiple description coding (MDC) is a way of coding video information into multiple bitstreams and is useful in a number of scenarios including peer-to-peer video streaming and video transmission over error-prone networks. We propose a modified tree structure for 3D-SPIHT that is more efficient for MD coding. We also propose a branch-pruning technique to generate multiple descriptions. We then show how rate-distortion optimization can be incorporated into the MDC framework to develop an efficient MD video coding system. Experimental results show that the proposed MD system achieves significant improvement in performance compared to existing methods.

Shape masks are used in object-based video coding, such as MPEG-4 part 2, to specify the boundaries of a video object. The term ldquobinary shaperdquo is used to describe shape masks where each pixel is either completely inside the object or completely outside it, i.e., there is no blending of pixels at object boundaries. In this paper, we present a new lossless block-based coding algorithm for binary shapes that combines a quad-tree structure with context-based arithmetic coding. Experimental results demonstrate that this new approach provides a saving in bits generated up to approximately 66% compared to the MPEG-4 binary shape coding algorithm.

In this paper we propose a new texture segmentation technique that produces segmentation results which more closely match the manual segmentation that would be performed by a human operator. To perform this type of segmentation, we propose a new texture feature based on the double dyadic dual-tree complex wavelet transform (D3T-CWT) which provides the ability to analyse a signal at and between dyadic scales. This new texture feature is invariant to shift, rotation and scale and hence can group the texture features in a single object (which may have different sizes and orientations) into a single more meaningful segment. When compared with other texture segmentation approaches, the proposed approach provides segmentation results which more closely match the semantically meaningful objects in the scene.

Multiple description coding (MDC) of video is a useful technique for robust video transmission over unreliable networks. MDC not only provides at least acceptable quality of video in error-prone network but it also efficiently utilizes the multipath nature of Internet and wireless networks. In this paper, multiple description video coding with 3-D set partitioning in hierarchical tree (3D-SPIHT) codec has been proposed. In particular, we propose branch-pruning technique to generate multiple descriptions. We then propose a new tree structure for 3D-SPIHT which is particularly efficient for MDC. Experimental results prove the effectiveness of the proposed method.

We present an unsupervised image registration algorithm to estimate the background object motion in a real video sequence. The algorithm is based on a Gaussian minimisation technique. It has been shown earlier that initialization of such an approach is very important to achieve the motion parameters of the background object precisely, and that the use of a windowing technique can give better background object motion estimation results, even with large background occlusions. In some cases, however, the fixed window size initializes the gradient descent algorithm in a sub-optimal way. Here, another window size would bring the desired estimation direction. In this paper, we present a technique where variable window sizes are used to prevent these outliers. Experimental results show that the technique works very well with the considered test sequences.

In spite of recent progress in the development of hybrid block-based video codecs, it has been shown that for low-bitrate scenarios there is still coding gain applying object-based techniques. We present a sprite-based codec, based on latest H.264 features using an inbuilt segmentation approach for scenes recorded by a rotating camera. The segmentation itself is built up on reliable background estimation from the sprite and short-term image registration. Moreover, we generate multiple sprites based on physical camera parameter estimation that overcome three of the main drawbacks of sprite coding techniques. First, the coding cost for the sprite image is minimized. Second, multiple sprites allow temporal background refresh and finally, registration error accumulation is kept very small. Experimental results show that this coding approach significantly outperforms latest H.264 extensions applying hierarchical B pictures.

In object-based video coding, shape masks are used to specify the shape of a video object. The binary shape means each pixel is either background or object. In this paper, we present a new lossless block-based coding algorithm for binary shapes that combines a quad-tree structure with block-based arithmetic coding; an optimal pruning quadtree structure searching algorithm is proposed to further reduce the spatial redundancy of quad-tree blocks. Experimental results demonstrate that this new approach provides a saving in bits generated between approximately 40-75% compared to the MPEG-4 binary shape coding algorithm

We propose an enhanced window-based approach to local image registration for robust video mosaicing in scenes with arbitrarily moving foreground objects. Unlike other approaches, we estimate accurately the image transformation without any pre-segmentation even if large background regions are occluded. We apply a windowed hierarchical frame-to-frame registration based on image pyramid decomposition. In the lowest resolution level phase correlation for initial parameter estimation is used while in the next levels robust Newton-based energy minimization of the compensated image mean-squared error is conducted. To overcome the degradation error caused by spatial image interpolation due to the warping process, i.e. aliasing effects from under-sampling, final pixel values are assigned in an up-sampled image domain using a Daubechies bi-orthogonal synthesis filter. Experimental results show the excellent performance of the method compared to recently published methods. The image registration is sufficiently accurate to allow open-loop parameter accumulation for long-term motion estimation.

In this paper we propose a method for extending the standard motion estimation algorithms used in video compression algorithms, to incorporate motion parameters that describe non-translational motion. This method uses an elastic image registration algorithm with two-dimensional discrete cosine basis functions to estimate the motion between block partitions in two video frames. A rate and a distortion are then determined for the elastic motion compensation and these are included in the mode decision calculations of the encoder algorithm. This allows the encoder to trade off bits to describe a more complex motion with bits to describe a larger residual error from a simpler translational motion model. Experimental results show an improvement in compression efficiency of 10-20% can be obtained with this enhanced motion compensation algorithm.

We consider the warping problem which appears in well-known image registration algorithms that use higher-order motion models. An implementation inspired by recent work and a new image registration algorithm are used for the analysis. Both approaches rely on frame-to-frame estimation. The key technique is the well-established gradient descent approach for the estimation of higher-order motion parameters. We show that using up-sampled input images in the last step of the algorithms improves the accuracy of the estimated motion parameters. It can be seen in the experimental results that the performance of the image registration algorithms increases significantly only by applying the gradient descent on up-sampled images in comparison to recent algorithms developed

We investigate streaming video over Differentiated Services (Diffserv) networks that can provide a number of aggregated traffic classes with increasing quality guarantee. We propose a method to measure the loss impact of a video packet on the quality of the decoded video images. We show how the optimal Quality-of-Service (QoS) mapping from the video packets into a set of traffic classes depends on the loss rates of the classes and the pricing model, and we develop an algorithm to accurately find the optimal QoS mapping. The performance of our algorithm is evaluated through computer simulations and compares favorably to an existing algorithm.

In this paper, we propose a robust approach to super-resolution static sprite generation from multiple low-resolution images. Considering both short-term and long-term motion influences, a hybrid global motion estimation technique is first presented for sprite generation. An iterative super-resolution reconstruction algorithm is then proposed for the super-resolution sprite construction. This algorithm is robust to outliers and results in a sprite image with high visual quality. In addition, the generated super-resolution sprite can provide better reconstructed images than a conventional low-resolution sprite. Experimental results demonstrate the effectiveness of the proposed methods.

In this paper, we propose some extensions of an efficient gradient-based image registration method called the inverse compositional algorithm. Specifically, these extensions include cumulative multi-image registration and incorporations of illumination change and lens distortion correction. By combining these extensions, we propose efficient cumulative multi-image registration methods with illumination and lens distortion correction. It is shown that high efficiency can still be achieved for multiple images using the proposed methods. Some experimental results show the efficacy of the proposed methods.

In many scenarios where it is desirable to provide a number of different descriptions of a video sequence, the simulcasting of single-layer bitstreams is impractical due to the amount of data involved. The new technique "stream morphing" introduced in this paper exploits the redundancy that exists between different compressed descriptions of the same video sequence at different quality levels. This works to dramatically reduce the data rates required to achieve the same effect as simulcast. The results shown here indicate that an SNR scalable video system constructed using this principle outperforms the structures defined in MPEG-2 and the MPEG-4 Streaming Video Profile. Stream morphing has a number of additional advantages: no loss of subjective quality compared to single-layer bitstreams at the same peak SNR regardless of the number of layers involved (unlike the MPEG-2 approach), the ability to generate a scalable representation of a video sequence from a set of single-layer descriptions in a computationally inexpensive way, and compatibility with existing single-layer decoders via a simple preprocessor without the need for transcoding. Stream morphing is demonstrated here in the context of MPEG-4; however, it is compatible with other similar video coding standards.

It is well known that tile-boundary artifacts occur in wavelet-based lossy image coding. However, until now, their cause has not been understood well. In this paper, we show that boundary artifacts are an inescapable consequence of the usual methods used to choose tile size and the type of symmetric extension employed in a wavelet-based image decomposition system. This paper presents a novel method for reducing these tile-boundary artifacts. The method employs odd tile sizes (2N + 1 samples) rather than the conventional even tile sizes (2N samples). It is shown that, for the same bit rate, an image compressed using an odd tile length low-pass first (OTLPF) convention has significantly less boundary artifacts than an image compressed using even tile sizes. The OTLPF convention can also be incorporated into the JPEG 2000 image compression algorithm using extensions defined in Part 2 of this standard.

Differentiated services networks can only function effectively if an appropriate mix of traffic can be maintained in all service classes using a practical pricing model. Previous work has noted that the same video sequence being distributed to clients who are prepared to pay different amounts for the service results in traffic distributions that vary considerably between those clients. It is shown that this variation can be reduced by trading off coding distortion and the amount of protection given to packets. As a result, a simple pricing model provides stable traffic distributions over a wide range of network conditions and client budgets.

Image segmentation can be viewed as the process of classifying regions in a picture into groups with common properties (e.g., texture). A difficulty arising is that a common texture can be classified differently when viewed at different scales and rotated viewpoints. The paper presents a feature vector based on the DT-CWT (dual-tree complex wavelet transform) (Kingsbury, N., Applied and Computational Harmonic Anal., vol.10, p.234-53, 2001) that is invariant to scale and rotation. The promising image segmentation results (without cleaning misclassified regions) demonstrate the suitability of this feature vector in representing texture.

Stream morphing is a technique that allows for two or more video bitstreams coded from the same original material with different quality settings to be simulcast in an efficient way. This paper describes our second generation stream morphing system that is based on the H.264 standard rather than MPEG-4 as used in our original implementation. While the basic operation of stream morphing remains largely unchanged for H.264, a number of enhancements have been made. Firstly, enhancement layers are coded using the same low-complexity, adaptive arithmetic coder utilized by H.264 itself. Furthermore, a scalable motion model has been added which allows for the use of very low bit rates in the base layer compared to the previous approach that used the same motion vectors and macroblock partitioning modes in all layers.

The reliance of many video processing techniques on motion estimation requires a good motion confidence measure, in order to ensure that unreliable estimations are not accepted into a system. This paper proposes a new measure of motion confidence based on phase information from Fourier transform of a video frame. Using the measure, blocks associated with unreliable motion estimation are detected efficiently and accurately. The merit of the technique is a much improved sensitivity over circumstances where other measures show ambiguity. This improvement is demonstrated through our experimental and analytical results.

We investigate streaming packet video over relative Differentiated Services (DiffServ) networks which can provide a number of aggregated traffic classes, ordered in a way such that class q+l is better or at least no worse than class q in terms of packet loss. We propose an algorithm for optimal quality of service (QoS) mapping from the video packets to a set of available DiffServ classes. The performance of our algorithm is evaluated through experimental tests and compares favorably to previous works.

Stream morphing has been previously introduced as an efficient way of simulcasting a hierarchy of single-layer descriptions of a single video sequence with different levels of quality while sharing the same frame size and rate. One application for this process was SNR scalable video coding. This paper presents two different adaptations of the stream morphing process to temporal scalability, one with bi-directional motion compensated prediction and one without. Both methods retain the attractive features of the original SNR scalable structure, notably the use of a single quantization cycle and single decoder motion-compensated prediction loop irrespective of the number of layers present and the ability to convert between single-layer and scalable forms without transcoding.

In this paper, we present a multisensor surveillance system that consists of an optical sensor and an infrared sensor. In this system, a background subtraction method based on zero-order statistics is utilized for the moving object segmentation. Additionally, we propose a generic approach to simultaneous object tracking and multisensor image registration. An efficient face detection system is shown as an application that will have enhanced performance from the registration and fusion of the information from the two sensors. Experimental results show the efficacy of the proposed system.

Stream morphing is a new approach to scalable video coding similar in scope to those already included in the MPEG standards. It works by exploiting redundancy between a hierarchy of single layer bitstreams of increasing quality. This paper provides an overview of the technique along with results for the constant quantizer SNR scalable case which demonstrate good performance, especially for the class of "low motion" video sequences. In addition to SNR scalability, stream morphing has also been applied to temporal and spatial scalability.

It is well known that tile boundary artefacts occur in wavelet-based lossy image coding. However, until now, their cause has not been well understood. We show that boundary artefacts are an intrinsic feature associated with the common method used to choose the tile size and the type of symmetric extension employed in wavelet-based image decomposition. A novel method of reducing tile boundary artefacts is presented. This method has recently been adopted as part of the JPEG 2000 verification model. In this technique, odd tile sizes of 2/sup N/+1 are chosen rather than the conventional even tile sizes of 2/sup N/. We show that, for the same bit-rate, an image compressed using odd tile length and low pass first convention (OTLPF) has significantly fewer boundary artefacts than an image compressed using even tile sizes.

In the real world, we commonly receive information simultaneously through two or more senses, with the brain fusing this data to produce a single coherent message. Lip-reading is one example of this phenomenon. Laboratory studies, on the other hand, often measure the response to a stimulus by a single sense and extrapolate these results to predict real-world behavior. In this paper, we show that semantics have a significant impact on viewers' sensitivity to the quality of a video sequence for spatially separated parts of the sequence and, more importantly, that this difference in sensitivity can be changed by the presence of an audio signal. This result is important for any testing of subjects' responses to visual material. One example is the subjective assessment of the quality of video in an audio-visual communications system (such as television or videoconferencing).

Signal-to-noise ratio (SNR) scalability has been incorporated into the MPEG-2 video-coding standard to allow for the delivery of two services with the same spatial and temporal resolution but different levels of quality. In this paper, we begin by reviewing the performance of a single-loop SNR scalable encoder that is compliant with the MPEG-2 standard and demonstrate that its performance is limited by drift in the base layer. We also look at an alternative single-loop drift-free noncompliant SNR scalable encoder, but discover that its coding efficiency is poor. We then review the performance of an MPEG-compliant two-loop SNR scalable encoder. Finally, we propose a new two-loop noncompliant encoder which achieves improved coding performance at the expense of some increase in encoder and decoder complexity.

It has been previously reported that variable bit rate video traffic displays signs of long-range dependence (Beran et al. 1995, and Garrett 1993), which is defined within the framework of weak stationarity. The concept of weak stationarity is of fundamental practical interest, because it implies the existence and independence with time of mean and autocovariance. Since mean and variance might be used as parameters for policing the traffic stream on a broadband communication network, it is important to verify the underlying assumption of weak stationarity. The results presented in this paper strongly call into question this assumption. As a possible explanation, we offer shifting level processes which are asymptotically weakly stationary for infinite sample sizes.

We describe a robust codec for the transmission of very low bit-rate video over channels with a variety of errors, including random and bursty bit errors and packet loss. The codec exploits adaptivity to give good performance with a low overhead. By only protecting macroblocks which would otherwise be poorly concealed by the decoder the codec allows adaptive selection of the parts of video to protect. For protection, it uses multiple description codes which indirectly provide frequency-based adaptivity by protecting the more significant DCT coefficients. Simulations show significant improvements in the performance of the codec when compared to codecs which use intra macroblock updating (raster scan and random) at the same overhead. The codec is efficient in its use of bits and has good error resilience properties both objectively and subjectively over a wide range of conditions. Further, transcoding of the received bit stream to the standard H.263 syntax is relatively easy.

Audio-visual and other multimedia services are seen as important sources of traffic for future telecommunication networks, including wireless networks. A major drawback with some wireless networks is that they introduce a significant number of transmission errors into the digital bitstream. For video, such errors can have the effect of degrading the quality of service to the point where it is unusable. We introduce a technique that allows for the concealment of the impact of these errors. Our work is based on MPEG-2 encoded video transmitted over a wireless network whose data structures are similar to those of asynchronous transfer mode (ATM) networks. Our simulations include the impact of the MPEG-2 systems layer and cover cell-loss rates up to 5%. This is substantially higher than those that have been discussed in the literature up to this time. We demonstrate that our new approach can significantly increase received video quality, but at the cost of a considerable computational overhead. We then extend our technique to allow for higher computational efficiency and demonstrate that a significant quality improvement is still possible.

In the real world, we commonly receive information simultaneously through two or more senses, with the brain fusing this data to produce a single-coherent message. Lip-reading is one example of this phenomenon. Laboratory studies, on the other hand often measure the response to a stimulus by a single sense and extrapolate these results to predict real-world behaviour. In this paper, we show that semantics have a significant impact on viewers' sensitivity to the quality of a video sequence for spatially separated parts of the sequence-and, more importantly, that this difference in sensitivity can be changed by the presence of an audio signal. This result is important for any testing of subjects' responses to visual material. One example is the subjective assessment of the quality of video-in an audio-visual communications system (such as television or video conferencing).

The protocols used to multiplex the various streams in an audio-visual service (such as MPEG 2 systems and ITU-T H.223) have an important impact on the quality of the service. The fact that wireless channels are often associated with high bit-error rates which are often bursty means that the multiplexing protocols should be designed to be robust against such errors. We suggest two departures from traditional practice which can improve significantly the performance of a packet-oriented multiplex for use with audio-visual services: (1) where the packet header is protected against errors using forward error correcting codes, the use of a synchronization codeword (or flag) is often unnecessary, and (2) for channels where errors tend to occur in bursts, the occurrence of errors in the header can be reduced by transmitting the header information twice (at both ends of the packet) with a low level of FEC protection instead of using the same number of bits to obtain an increased level of protection using a single code word. The value of these approaches is confirmed by simulations.

We describe a robust codec for the transmission of very low bit-rate video over channels with bursty errors. The codec uses a diversity-based method in the form of multiple description codes to reduce the effect of errors in the video bit-stream. To improve the efficiency for transmission of very low bit-rate video, the codec also exploits adaptivity by only protecting macroblocks (using multiple description codes) which would otherwise be poorly concealed by the decoder. Simulations show significant improvements in the performance of the codec when compared to codecs which use intra macroblock updating (raster scan and random) with the same overhead for reducing the effects of errors.

The transmission of audio-visual services on low-bit-rate, wireless telecommunications systems requires the use of coding techniques that are both efficient in their use of bits and robust against errors introduced in transmission. In this paper, we present efficient techniques for improving the error resilience of audio-visual services. These techniques are based on coding simultaneously for synchronization and error protection or detection. We apply the techniques to improve the performance of the multiplexing protocol (which combines the video and audio streams so that they can be transmitted on a single circuit), and also to improve the robustness of the coded video. We show through simulations that the techniques are efficient in their use of bits and effective against bursty errors common in wireless channels. For a simulation of the DECT channel at a bit-error rate of 10/sup -3/, the techniques give an order of magnitude improvement in the probability of lost packets in the multiplexer layer over more conventional techniques. In the video layer, the techniques give an improvement of between 1-2 dB over ITU-T Recommendation H.263. The techniques proposed for the video layer also have the advantage of permitting simple transcoding with bit streams complying with H.263.

Audio-visual and other multimedia services are seen as an important source of traffic for future telecommunications networks, including wireless networks. We examine the impact of the properties of a 50 Mb/s asynchronous transfer mode (ATM)-based wireless local-area network (WLAN) on Moving Picture Experts Group phase 2 (MPEG 2) compressed video traffic, with emphasis on the network's error characteristics. The paper includes a description of the WLAN system used and its loss characteristics, a brief discussion of relevant aspects of the MPEG 2 standards and the associated error resilience techniques for minimizing the effect of transmission errors, and a description of the method by which the video data is organized for transmission on the network. We show results on the effect of cell loss due to transmission errors on the quality of the decoded video at the receiver, and demonstrate how error resilience techniques in both the systems and video layers of MPEG 2 can be used to improve the quality of service. Situations where up to 1% of the data is lost due to network transmission errors are examined. Most important among the findings are that error resilience experiments that do not take into account the effect of the MPEG 2 systems layer will tend to significantly overestimate the quality of received video, and that the error resilience techniques provided within the MPEG 2 standard are not sufficient to provide acceptable quality with acceptable overheads, but that this quality can be significantly increased by the addition of a small number of simple techniques.

This paper presents a new fast search algorithm for block matching motion estimation called the adaptive search length (ASL) algorithm. The ASL algorithm adaptively varies the number of positions searched for each block while still maintaining control of the average number of searches per block for each frame. Experimental results show that the peak signal-to-noise ratio (PSNR) of decoded sequences which were coded using the ASL algorithm is within 0.25 dB of the PSNR of decoded sequences which were coded using the full search block matching algorithm. It is also shown that the ASL algorithm requires only 10% of the computations required by the full search algorithm to achieve this level of decoded image quality.

A new fast search algorithm for block matching motion estimation is presented in this paper. The new algorithm, called the adaptive search length (ASL) algorithm, allows the maximum number of searches for each block of the image to vary according to the difficulty in finding the optimum motion vector. The PSNR of the decoded images produced by a video coder operating with the ASL algorithm were compared with those produced by a coder operating with the full search block matching algorithm. The results presented show that, for a PSNR of within .25 dB of the full search PSNR, the ASL algorithm requires only 10% of the searches required by the full search algorithm.

We examine the implications of our recent analysis of the local minima escape behavior of the Godard (or CMA) blind equalizer. This analysis suggests asymptotic estimates for the expected value of the escape time when leaving the region of attraction of local equilibria. For a particular source, channel, and equalizer local minimum parameterization, the log of the inverse of the mean escape time is estimated to be proportional to a weighted sum of the stepsize and the channel noise variance. We examine the validity of the functional form of this relationship for two examples. This experimental evidence corroborates our earlier analysis and suggests that the mean time to escape can be very large for the Godard (1980) algorithm (also known as the constant modulus algorithm).>

Many techniques have been proposed for modeling source rates generated by variable bit rate video coders. However, little work has been done to verify these models, especially in the area of measuring the quality of their predictions for network performance and the grade of service experienced by a customer. In this paper, we summarize the previous work in this area, highlight some of the difficulties in effectively verifying these models, and suggest methods by which source rate models for VBR video might be more effectively verified in the future. In addition to studying the problem of model verification, a new model is proposed. Using this model, a video sequence is characterized by four parameters. As in previous cases, the number of bits generated by the coder for a sequence of video frames is modelled. The new model differs from previous models in that this sequence is not Markovian.>

The estimation of the statistics of buffer overflows in networks of queues by means of simulation is inherently costly, simply because of the rarity of these events. An alternative analytic approach is presented, with very low computational cost, for calculating the recurrence time of buffer overflows for Jackson networks in which the recurrence times of buffer overflows in a network are expressed in terms of the recurrence times for overflows of individual buffers, isolated from the network. This result is applied to the buffer allocation problem for queueing networks, providing extensions and further justification for a previously derived heuristic approach to this problem.>