In this paper, we explore tying together the ideas from Scattering Transforms and Convolutional Neural Networks (CNN) for Image Analysis by proposing a learnable ScatterNet. Previous attempts at tying them together in hybrid networks have tended to keep the two parts separate, with the ScatterNet forming a fixed front end and a CNN forming a learned backend. We instead look at adding learning between scattering orders, as well as adding learned layers before the ScatterNet. We do this by breaking down the scattering orders into single convolutional-like layers we call `locally invariant' layers, and adding a learned mixing term to this layer. Our experiments show that these locally invariant layers can improve accuracy when added to either a CNN or a ScatterNet. We also discover some surprising results in that the ScatterNet may be best positioned after one or more layers of learning rather than at the front of a neural network.

This paper proposes a generative ScatterNet hybrid deep learning (G-SHDL) network for semantic image segmentation. The proposed generative architecture is able to train rapidly from relatively small labeled datasets using the introduced structural priors. In addition, the number of filters in each layer of the architecture is optimized resulting in a computationally efficient architecture. The G-SHDL network produces state-of-the-art classification performance against unsupervised and semi-supervised learning on two image datasets. Advantages of the G-SHDL network over supervised methods are demonstrated with experiments performed on training datasets of reduced size.

We introduce a ScatterNet that uses a parametric log transformation with Dual-Tree complex wavelets to extract translation invariant representations from a multi-resolution image. The parametric transformation aids the OLS pruning algorithm by converting the skewed distributions into relatively mean-symmetric distributions while the Dual-Tree wavelets improve the computational efficiency of the network. The proposed network is shown to outperform Mallat's ScatterNet [1] on two image datasets, both for classification accuracy and computational efficiency. The advantages of the proposed network over other supervised and some unsupervised methods are also presented using experiments performed on different training dataset sizes.

It is well known that many more than three or four spectral measurements are required for accurate measurement of color. Previous work has shown seven to ten measurements can yield accurate results on average, but with significant numbers of errors above the threshold of obvious visual detection. Furthermore, the filters used for these measurements are very difficult to fabricate. We show that such filters are not needed and, in fact, have much poorer performance, in perceptual quality measured in ΔEab, than simple narrow-band filters. This is especially true in the presence of Poisson noise at a level common in current digital cameras. In realistic Poisson noise, our filter sets of up to 12 filters allow average ΔEab values around 0.5, with maximum errors below 3.

In this paper, we propose a new algorithm to solve linear inverse problems using an approximate l0penalty on overlapped groups of wavelet coefficients, and apply this to the deconvolution problem specifically. Prior work has shown the improvements gained from using group-sparse penalties over coefficient-sparse penalties for deconvolution and compressed sensing. Instead of minimizing an l1-norm, as done in prior work, we instead design an overlapping group prior which utilizes the Gaussian scale mixture model, and use this to promote l0sparsity. Using a Bayesian argument, we derive a novel convex penalty function, which is a reweighted l2approximation to the l0-norm that can be efficiently minimized. We show that the new group-sparse algorithm produces superior deconvolution results compared to the same algorithm utilizing an unstructured coefficient-sparse penalty.

In this paper, we propose to incorporate wavelet tree structures into a recently developed wavelet modeling method, called VBMM. We show that, using overlapped groups, tree-structured modeling can be integrated into the high-performance non-convex sparsity-inducing VBMM method, and can achieve significant performance gains over the coefficient-sparse version of the algorithm.

The importance of properly exploiting a classifier's inherent geometric characteristics when developing a classification methodology is emphasized as a prerequisite to achieving near optimal performance when carrying out thematic mapping. When used properly, it is argued that the long-standing maximum likelihood approach and the more recent support vector machine can perform comparably. Both contain the flexibility to segment the spectral domain in such a manner as to match inherent class separations in the data, as do most reasonable classifiers. The choice of which classifier to use in practice is determined largely by preference and related considerations, such as ease of training, multiclass capabilities, and classification cost.

In this paper we propose a new algorithm for reconstructing phase-encoded velocity images of catalytic reactors from undersampled NMR acquisitions. Previous work on this application has employed total variation and nonlinear conjugate gradients which, although promising, yields unsatisfactory, unphysical visual results. Our approach leverages prior knowledge about the piecewise-smoothness of the phase map and physical constraints imposed by the system under study. We show how iteratively regularizing the real and imaginary parts of the acquired complex image separately in a shift-invariant wavelet domain works to produce a piecewise-smooth velocity map, in general. Using appropriately defined metrics we demonstrate higher fidelity to the ground truth and physical system constraints than previous methods for this specific application.

Restoring a scene distorted by atmospheric turbulence is a challenging problem in video surveillance. The effect, caused by random, spatially varying, perturbations, makes a model-based solution difficult and in most cases, impractical. In this paper, we propose a novel method for mitigating the effects of atmospheric distortion on observed images, particularly airborne turbulence which can severely degrade a region of interest (ROI). In order to extract accurate detail about objects behind the distorting layer, a simple and efficient frame selection method is proposed to select informative ROIs only from good-quality frames. The ROIs in each frame are then registered to further reduce offsets and distortions. We solve the space-varying distortion problem using region-level fusion based on the dual tree complex wavelet transform. Finally, contrast enhancement is applied. We further propose a learning-based metric specifically for image quality assessment in the presence of atmospheric distortion. This is capable of estimating quality in both full- and no-reference scenarios. The proposed method is shown to significantly outperform existing methods, providing enhanced situational awareness in a range of surveillance scenarios.

This paper presents new methods for computing the step sizes of the subband-adaptive iterative shrinkage-thresholding algorithms proposed by Bayram & Selesnick and Vonesch & Unser. The method yields tighter wavelet-domain bounds of the system matrix, thus leading to improved convergence speeds. It is directly applicable to non-redundant wavelet bases, and we also adapt it for cases of redundant frames. It turns out that the simplest and most intuitive setting for the step sizes that ignores subband aliasing is often satisfactory in practice. We show that our methods can be used to advantage with reweighted least squares penalty functions as well as L1 penalties. We emphasize that the algorithms presented here are suitable for performing inverse filtering on very large datasets, including 3D data, since inversions are applied only to diagonal matrices and fast transforms are used to achieve all matrix-vector products.

This paper describes a new method for mitigating the effects of atmospheric distortion on observed images, particularly airborne turbulence which degrades a region of interest (ROI). In order to provide accurate detail from objects behind the distorting layer, a simple and efficient frame selection method is proposed to pick informative ROIs from only good-quality frames. We solve the space-variant distortion problem using region-based fusion based on the Dual Tree Complex Wavelet Transform (DT-CWT). We also propose an object alignment method for pre-processing the ROI since this can exhibit significant offsets and distortions between frames. Simple haze removal is used as the final step. The proposed method performs very well with atmospherically distorted videos and outperforms other existing methods.

An anomaly detection approach is considered for the mine hunting in sonar imagery problem. The authors exploit previous work that used dual-tree wavelets and fractal dimension to adaptively suppress sand ripples and a matched filter as an initial detector. Here, lacunarity inspired features are extracted from the remaining false positives, again using dual-tree wavelets. A one-class support vector machine is then used to learn a decision boundary, based only on these false positives. The approach exploits the large quantities of ‘normal’ natural background data available but avoids the difficult requirement of collecting examples of targets in order to train a classifier.

We present a novel method to perform an accurate registration of 3-D nonrigid bodies by using phase-shift properties of the dual-tree complex wavelet transform [Formula: see text]. Since the phases of [Formula: see text] coefficients change approximately linearly with the amount of feature displacement in the spatial domain, motion can be estimated using the phase information from these coefficients. The motion estimation is performed iteratively: first by using coarser level complex coefficients to determine large motion components and then by employing finer level coefficients to refine the motion field. We use a parametric affine model to describe the motion, where the affine parameters are found locally by substituting into an optical flow model and by solving the resulting overdetermined set of equations. From the estimated affine parameters, the motion field between the sensed and the reference data sets can be generated, and the sensed data set then can be shifted and interpolated spatially to align with the reference data set.

We present a matching framework to find robust correspondences between image features by considering the spatial information between them. To achieve this, we define spatial constraints on the relative orientation and change in scale between pairs of features. A pairwise similarity score which measures the similarity of features based on these spatial constraints is considered. The pairwise similarity scores for all pairs of candidate correspondences are then accumulated in a 2D similarity space. Robust correspondences can be found by searching for clusters in the similarity space, since actual correspondences are expected to form clusters that satisfy similar spatial constraints in this space. As it is difficult to achieve reliable and consistent estimates of scale and orientation, an additional contribution is that these parameters do not need to be determined at the interest point detection stage, which differs from conventional methods. Polar matching of dual-tree complex wavelet transform features is used, since it fits naturally into the framework with the defined spatial constraints. Our tests show that the proposed framework is capable of producing robust correspondences with higher correspondence ratios and reasonable computational efficiency, compared to other well-known algorithms.

In this paper, a novel cortex-inspired feed-forward hierarchical object recognition system based on complex wavelets is proposed and tested. Complex wavelets contain three key properties for object representation: shift invariance, which enables the extraction of stable local features; good directional selectivity, which simplifies the determination of image orientations; and limited redundancy, which allows for efficient signal analysis using the multi-resolution decomposition offered by complex wavelets. In this paper, we propose a complete cortex-inspired object recognition system based on complex wavelets. We find that the implementation of the HMAX model for object recognition in [1, 2] is rather over-complete and includes too much redundant information and processing. We have optimized the structure of the model to make it more efficient. Specifically, we have used the Caltech 5 standard dataset to compare with Serre's model in [2] (which employs Gabor filter bands). Results demonstrate that the complex wavelet model achieves a speed improvement of about 4 times over the Serre model and gives comparable recognition performance.

Statistical dependencies among wavelet coefficients are commonly represented by graphical models such as hidden Markov trees (HMTs). However, in linear inverse problems such as deconvolution, tomography, and compressed sensing, the presence of a sensing or observation matrix produces a linear mixing of the simple Markovian dependency structure. This leads to reconstruction problems that are non-convex optimizations. Past work has dealt with this issue by resorting to greedy or suboptimal iterative reconstruction methods. In this paper, we propose new modeling approaches based on group-sparsity penalties that leads to convex optimizations that can be solved exactly and efficiently. We show that the methods we develop perform significantly better in de-convolution and compressed sensing applications, while being as computationally efficient as standard coefficient-wise approaches such as lasso.

Polar matching is a recently developed shift and rotation invariant object detection method that is based upon dual-tree complex wavelet transforms or equivalent multiscale directional filterbanks. It can be used to facilitate both keypoint matching, neighborhood search detection, or detection and tracking with particle filters. The theory is extended here to incorporate an allowance for local spatial and dilation perturbations. With experiments, we demonstrate that the robustness of the polar matching method is strengthened at modest computational cost.

International audience

The dual-tree wavelet transform is here applied to the problem of fractal dimension estimation. The Hurst parameter of fractional Brownian surfaces is estimated using various wavelet bases. Results are given for global, local, anisotropic, and both local and anisotropic Hurst parameters. It is shown that the directional selectivity of the dual-tree wavelets can be exploited effectively to compute and distinguish Hurst parameters that vary non-trivially with direction and space.

We present an algorithm for finding robust matches between images by considering the spatial constraints between pairs of interest points. By considering these constraints, we account for the layout and structure of features during matching, which produces more robust matches compared to the common approach of using local feature appearance for matching alone. We calculate the similarity between interest point pairs based on a set of spatial constraints. Matches are then found by searching for pairs which satisfy these constraints in a similarity space. Our results show that the algorithm produces more robust matches compared to baseline SIFT matching and spectral graph matching, with correspondence ratios up to 33% and 28% higher (respectively) across various viewpoints of the test objects while the computational load is only increased by about 25% over baseline SIFT. The algorithm may also be used with other feature descriptors apart from SIFT.

Image convolution is conventionally approximated by the LTI discrete model. It is well recognized that the higher the sampling rate, the better is the approximation. However sometimes images or 3D data are only available at a lower sampling rate due to physical constraints of the imaging system. In this paper, we model the under-sampled observation as the result of combining convolution and subsampling. Because the wavelet coefficients of piecewise smooth images tend to be sparse and well modelled by tree-like structures, we propose the L0 reweighted-L2 minimization (L0RL2) algorithm to solve this problem. This promotes model-based sparsity by minimizing the reweighted L2 norm, which approximates the L0 norm, and by enforcing a tree model over the weights. We test the algorithm on 3 examples: a simple ring, the cameraman image and a 3D microscope dataset; and show that good results can be obtained.

This paper proposes to use an extended Gaussian Scale Mixtures (GSM) model instead of the conventional l1norm to approximate the sparseness constraint in the wavelet domain. We combine this new constraint with subband-dependent minimization to formulate an iterative algorithm on two shift-invariant wavelet transforms, the Shannon wavelet transform and dual-tree complex wavelet transform (DTCWT). This extented GSM model introduces spatially varying information into the deconvolution process and thus enables the algorithm to achieve better results with fewer iterations in our experiments.

This paper describes an approach to image modelling using interscale phase relationships of wavelet coefficients for use in image estimation applications. The method is based on the dual tree complex wavelet transform, but a phase rotation is applied to the coefficients to create complex "derotated" coefficients. These derotated coefficients are shown to have increased correlation compared to standard wavelet coefficients near edge and ridge features allowing improved signal estimation in these areas. The nature of the benefits brought by the derotated coefficients are analyzed and the implications for image estimation algorithm design noted. The observations and conclusions provide a basis for design of the denoising algorithm in [1].

A method for removing additive Gaussian noise from digital images is described. It is based on statistical modeling of the coefficients of a redundant, oriented, complex multiscale transform. Two types of modeling are used to model the wavelet coefficients. Both are based on Gaussian scale mixture (GSM) modeling of neighborhoods of coefficients at adjacent locations and scales. Modeling of edge and ridge discontinuities is performed using wavelet coefficients derotated by twice the phase of the coefficient at the same location and the next coarser scale. Other areas are modeled using standard wavelet coefficients. An adaptive Bayesian model selection framework is used to determine the modeling applied to each neighborhood. The proposed algorithm succeeds in providing improved denoising performance at structural image features, reducing ringing artifacts and enhancing sharpness, while avoiding degradation in other areas. The method outperforms previously published methods visually and in standard tests.

The purpose of the Applied Multi-dimensional Fusion Project is to investigate the benefits that data fusion and related techniques may bring to future military Intelligence Surveillance Target Acquisition and Reconnaissance systems. In the course of this work, it is intended to show the practical application of some of the best multi-dimensional fusion research in the UK. This paper highlights the work done in the area of multi-spectral synthetic data generation, super-resolution, joint fusion and blind image restoration, multi-resolution target detection and identification and assessment measures for fusion. The paper also delves into the future aspirations of the work to look further at the use of hyper-spectral data and hyper-spectral fusion. The paper presents a wide work base in multi-dimensional fusion that is brought together through the use of common synthetic data, posing real-life problems faced in the theatre of war. Work done to date has produced practical pertinent research products with direct applicability to the problems posed.

A novel method for modelling the statistics of 2D photographic images useful in image restoration is defined. The new method is based on the dual tree complex wavelet transform (DT-CWT) but a phase rotation is applied to the coefficients to create complex coefficients whose phase is shift-invariant at multiscale edge and ridge features. This is in addition to the magnitude shift invariance achieved by the DT-CWT. The increased correlation between coefficients adjacent in space and scale provides an improved, mechanism for signal estimation

We present a novel approach to detecting multiscale keypoints using the dual tree complex wavelet transform (DTCWT). We show that it is a well-suited basis for this problem as it is directionally selective, smoothly shift invariant, optimally decimated at coarse scales and invertible (no loss of information). Our detection scheme is fast because of the decimated nature of the DTCWT and yet provides accurate and robust keypoint localisation, thanks to the use of the "accumulated energy map". The regularity of this map is used to introduce a new mechanism for robust keypoint scale selection. Key-points of different nature and size can be detected with limited redundancy, in a way which is consistent with our visual perception. Furthermore results show better robustness against rotation compared to the SIFT detector.

An orthonormal Hilbert-pair consists of a pair of conjugate-quadrature-filter (CQF) banks such that the equivalent wavelet function of both banks are approximate Hilbert transforms of each other. We found that the celebrated orthonormal wavelets of Daubechies, which have maximum vanishing-moment (VM), cannot be used to construct good Hilbert-pairs. In this letter, we reduce the number of VM by one and construct a Hilbert-pair with almost maximum VM. Each pair of wavelets are time-reverse versions of each other, and the individual wavelets are of the least asymmetric type (i.e., approximate linear phase CQF)

The aim of seismic imaging is to reconstruct subsurface reflectivity from scattered acoustic data. In standard reconstruction techniques, the reflectivity model parameters are usually defined as a grid of point scatterers over the area or volume of the subsurface to be imaged. We propose an approach to subsurface imaging using the dual tree complex wavelet transform (DT-CWT) as a basis for the reflectivity. This basis is used in conjunction with an iterative optimization which frames the problem as a linearized inverse scattering problem. We demonstrate the method on a marine seismic data set acquired over the Gippsland Basin near Australia. The technique is shown to reduce noise and processing artifacts while preserving discontinuities. It is likely to be particularly useful in cases where the acquired data are incomplete.

This paper introduces the interlevel product (ILP) which is a transform based upon the dual-tree complex wavelet. Coefficients of the ILP have complex values whose magnitudes indicate the amplitude of multilevel features, and whose phases indicate the nature of these features (e.g. ridges vs. edges). In particular, the phases of ILP coefficients are approximately invariant to small shifts in the original images. We accordingly introduce this transform as a solution to coarse scale template matching, where alignment concerns between decimation of a target and decimation of a larger search image can be mitigated, and computational efficiency can be maintained. Furthermore, template matching with ILP coefficients can provide several intuitive "near-matches" that may be of interest in image retrieval applications.

We present a new approach based on discriminant analysis to map a high dimensional image feature space onto a subspace which has the following advantages: 1) each dimension corresponds to a semantic likelihood, 2) an efficient and simple multiclass classifier is proposed and 3) it is low dimensional. This mapping is learnt from a given set of labeled images with a class groundtruth. In the new space a classifier is naturally derived which performs as well as a linear SVM. We show that projecting images in this new space provides a database browsing tool which is meaningful to the user. Results are presented on a remote sensing database with eight classes, made available online. The output semantic space is a low dimensional feature space which opens perspectives for other recognition tasks.

This paper presents a robust method to estimate the unknown standard deviation of a centred normal distribution from a mixture density. This method is applied to different signal processing problems. The first one concerns silence segmentation from audio data. The second application deals with colour class parameter extraction. In this later case, the mean is also estimated from the observations.

Spread transform (ST) is a quantization watermarking algorithm in which vectors of the wavelet coefficients of a host work are quantized, using one of two dithered quantizers, to embed hidden information bits; Loo (2002), had some success in applying such a scheme to still images. We extend ST to the video watermarking problem. Visibility considerations require that each spreading vector refer to corresponding pixels in each of several frames, that is, a multiframe embedding approach. Use of the hierarchical complex wavelet transform (CWT) for a visual mask reduces computation and improves robustness to jitter and valumetric scaling. We present a method of recovering temporal synchronization at the detector, and give initial results demonstrating the robustness and capacity of the scheme.

This paper proposes a new method of designing finite-support wavelet filters, based on minimization of energy in key parts of the frequency domain. In particular this technique is shown to be very effective for designing families of filters that are suitable for use in the shift-invariant dual-tree complex wavelet structure that has been developed by the author recently, and has been shown to be important for a range of image processing applications. The dual-tree structure requires most of the wavelet filters to have a well-controlled group delay, equivalent to one quarter of a sample period, in order to achieve optimal shift invariance. The proposed new design technique allows this requirement to be included along with the usual smoothness and perfect reconstruction properties to yield wavelet filters with a unique combination of features: linear phase, tight frame, compact spatial support, good frequency domain selectivity with low sidelobe levels, approximate shift invariance, and good directional selectivity in two or more dimensions.

Overcomplete transforms, like the dual-tree complex wavelet transform, offer more flexible signal representations than critically-sampled transforms, due to their properties of shift invariance and directional selectivity. We show that many transform coefficients can be discarded without much reconstruction quality loss by forcing compensatory changes in the remaining coefficients. We consider the convergence properties of an iterative projection system for achieving the usual coding aims of good sparsity with low reconstruction error. Results show how these measures translate to useful image compression performance.

Overcomplete transforms, like the dual-tree complex wavelet transform, offer more flexible signal representations than critically-sampled transforms. Large numbers of transform coefficients can be discarded without much reconstruction quality loss by forcing compensatory changes in the remaining coefficients. We develop and demonstrate a new and unique system for achieving these transform coding aims of coefficient elimination and compensation. The system is based on iterative projection of signals between the image domain and transform domain.

The goal in image segmentation is to label pixels in an image based on the properties of each pixel and its surrounding region. Recently content-based image retrieval (CBIR) has emerged as an application area in which retrieval is attempted by trying to gain unsupervised access to the image semantics directly rather than via manual annotation. To this end, we present an unsupervised segmentation technique in which colour and texture models are learned from the image prior to segmentation, and whose output (including the models) may subsequently be used as a content descriptor in a CBIR system. These models are obtained in a multiresolution setting in which hidden Markov trees (HMT) are used to model the key statistical properties exhibited by complex wavelet and scaling function coefficients. The unsupervised mean shift iteration (MSI) procedure is used to determine a number of image regions which are then used to train the models for each segmentation class.

In this paper, we propose to use a hidden Markov tree modeling of the complex wavelet packet transform, to capture the inter-scale dependencies of natural images. First, the observed image, blurred and noisy, is deconvolved without regularization. Then its transform is denoised within a Bayesian framework using the proposed model, whose parameters are estimated by an EM technique. The total complexity of this new deblurring algorithm remains O(N).

This paper proposes a new algorithm for image restoration (deconvolution and denoising) which employs the recently developed dual-tree complex wavelet transform in an iterative Bayesian framework. Complex wavelets are selected for their key features: shift invariance, directional selectivity and efficiency. The aim is to find an optimal description of the restored image in the complex wavelet domain, which minimises a quadratic energy function of the wavelet coefficients. The algorithm searches for this minimum using an efficient conjugate gradient method. We show that this can improve the SNR performance of a good minimax deconvolution method, WaRD, which is used to initialise the iterations, by typically 1.2 dB. Convergence is quite rapid, achieving 80% of the ultimate performance gain in about 20 iterations. Each iteration takes around 5 seconds using MatLab on a 400 MHz Pentium computer with 256/spl times/256 pixel images.

Multiresolution signal and image models such as the hidden Markov tree aim to capture the statistical structure of smooth and singular (edgy) regions. Unfortunately, models based on the orthogonal wavelet transform suffer from shift-variance, making them less accurate and realistic. We extend the HMT modeling framework to the complex wavelet transform, which features near shift-invariance and improved angular resolution compared to the standard wavelet transform. The model is computationally efficient (with linear-time computation and processing algorithms) and applicable to general Bayesian inference problems as a prior density for the data. In a simple estimation experiment, the complex wavelet HMT model outperforms a number of high-performance denoising algorithms, including redundant wavelet thresholding (cycle spinning) and the redundant HMT.

Prediction of fine scale DWT coefficients from coarser scales is not generally possible. On the other hand, the redundancy of the shift-invariant complex wavelet transform (CWT) should allow good coarse-to-fine scale prediction. The CWT's magnitude and phase step responses are well-behaved functions of a coefficient's normal distance to an edge. We present a coefficient prediction algorithm which assumes an edge-based image model. The algorithm infers information about local edges from coarse scale coefficients, and predicts the effect of these edges on the magnitudes and phases of finer level coefficients. Accurate coarse-to-fine prediction of coefficients has important potential benefits for image coding and resolution enhancement systems.

Texture description is important for image retrieval from various large image databases. A method for representing texture information in images utilizing dual tree complex wavelet transform (DT-CWT) is introduced. Approximate shift invariance, good directional selectivity and computational efficiency properties of DT-CWT are effective in describing texture patterns. The perfect reconstruction property of DT-CWT enables us to store the image data using DT-CWT coefficients. Experiments are performed with Brodatz (1966) texture database and nature images that have significant amount of texture information to investigate the efficiency of our texture description algorithm.

We present a new form of the dual-tree complex wavelet transform (DT CWT) with improved orthogonality and symmetry properties. Beyond level 1, the previous form used alternate odd-length and even-length bi-orthogonal filter pairs in the two halves of the dual-tree, whereas the new form employs a single design of even-length filter with asymmetric coefficients. These are similar to the Daubechies orthonormal filters, but designed with the additional constraint that the filter group delay should be approximately one quarter of the sample period. The filters in the two trees are just the time-reverse of each other, as are the analysis and reconstruction filters. This leads to a transform, which can use shorter filters, which is orthonormal beyond level 1, and in which the two trees are very closely matched and have a more symmetric sub-sampling structure, but which preserves the key DT CWT advantages of approximate shift-invariance and good directional selectivity in multiple dimensions.

We propose a blind watermarking algorithm in the complex wavelet domain. We discuss why complex wavelets are preferable to real ones in terms of image adaptability and comment on the precautions required in embedding watermarks in an oversampled domain like the complex wavelet transform. A simple but effective human visual model in the complex wavelet domain is described. By modelling watermarking as a communication process, our results also show that this domain has higher capacity than the real wavelet and spatial domains, hence that complex wavelets provide a potentially good domain for watermarking.

Active contour methods are a powerful approach to image segmentation. The first approaches were based on the direct evolution of a contour but recently level set methods have been found to give more robust solutions. These methods are based on the iterative deformation of a surface and their main drawback is the large number of iterations required. We propose a new energy formulation for which it is possible to obtain a good estimate for the optimum step size in a gradient descent algorithm and which therefore produces much faster convergence. This is made possible by representing the surface with the coefficients of a complex wavelet transform. The energy function has one term that is minimised for smooth contours, and one term that is minimised for contours close to edges in an image. We explain how the complex wavelet transform can efficiently represent both these terms and show experimental results on real and synthetic images confirming that the method gives good results within a few iterations.

Multiresolution signal and image models such as the hidden Markov tree (HMT) aim to capture the statistical structures of smooth and singular (textured and edgy) regions. Unfortunately, models based on the orthogonal wavelet transform suffer from shift-variance, making them less accurate and realistic. We extend the HMT modeling framework to the complex wavelet transform, which features near shift-invariance and improved angular resolution compared to the standard wavelet transform. The model is computationally efficient (featuring linear-time computation and processing algorithms) and applicable to general Bayesian inference problems as a prior density for the data. We develop a simple multiscale maximum likelihood classification scheme based on the complex wavelet HMT that outperforms methods based on real-valued wavelet HMTs. The resulting classifier can be used as a front end in a more sophisticated multiscale segmentation algorithm.

We discuss the shift invariant properties of a new implementation of the discrete wavelet transform, which employs a dual-tree of wavelet filters to obtain the real and imaginary parts of complex wavelet coefficients. This introduces limited redundancy (2/sup m/:1 for m-dimensional signals) and allows the transform to provide approximate shift invariance and directionally selective filters (properties lacking in the traditional wavelet transform) while preserving the usual properties of perfect reconstruction and computational efficiency with good well-balanced frequency responses.

Digital libraries and multimedia databases are being rapidly developed and efficient search algorithms must now be developed. Gabor features have been experimentally shown to be the most accurate but have the disadvantage of slow computation. This paper shows how a new complex wavelet transform can be used to approximate the Gabor features and derives a distance metric based on statistical hypothesis testing that gives a better performance than the usual metric. The new features are experimentally compared with both Gabor and standard wavelet techniques.

Zerotree based methods provide some of the best image compression currently available in terms of rate-distortion. They are however susceptible to any errors which may be introduced by wireless transmission. We consider three methods of enhancing the resilience of the popular Said and Pearlman (see IEEE Trans. Image Processing, vol.6, no.10, p.1436-41, 1997) zerotree image compression algorithm. One method has been previously proposed by Creusere (see IEEE Trans. on Circuits and Systems for Video Technology, vol.6, no.3, p.243-50, 1996), which we include for reference. We then include two solutions of our own based upon the error resilient entropy code (EREC).

The error resilient entropy code (EREC) has been well known for some time. It is a method of encoding the overall prefix codes which result from many forms of image compression algorithm such that they are largely independent of each other. Any error introduced into the data, for example during transmission over a radio link, will thus only affect a small amount of data. This paper discusses methods of enhancing the resilience of the EREC to errors and faster processing of EREC encoded image data.

The technique outlined in this paper extends the ability of current warping motion estimation schemes to allow occlusion and uncovering effects to be modelled. Current methods use a continuous rubber sheet approach, consisting of non-overlapping polygons. The new technique estimates and compensates affine and then translational motion within the scene. The latter is achieved by allowing polygons to overlap through the introduction of rips into the sheet which are located in areas where occlusion or uncovering is thought to occur. Results show a reduction in the prediction error when compared to both traditional block-based methods and recently developed warping schemes (without rips).

This paper describes a fast search algorithm for warping motion compensation schemes which is based on a first order approximation of a Taylor series. In comparison to a full search algorithm the technique significantly reduces the required computational load (by a factor of approximately fifty) whilst maintaining the performance in terms of prediction error. Further gains in prediction error performance are expected as the new algorithm is investigated further.

Wang and Adelson see (IEEE Trans. Image Proc., vol.3, no.5, p.625-38, 1994) proposed the layer representation as a convenient representation for video coding, format conversion and special effects. We present a technique to extract the near optimal layer representation of an image sequence using the EM algorithm. We incorporate forward/backward motion estimation to obtain a sharp segmentation in the presence of occlusion and uncovering. We demonstrate that the algorithm is capable of tracking layers within a sequence, and detecting when layers enter or leave the sequence.

This paper describes a method for incorporating the chrominance information when estimating the motion in a colour image sequence. It is based on a maximum likelihood formulation of the motion estimation problem which assumes homogeneous additive Gaussian noise in each colour component, with known inter-field correlation statistics. The formulation is applied to the complex-wavelet-domain matching algorithm of Magarey and Kingsbury (see Proc. IEEE Int. Conf. on Image Processing, p.969-72, 1996). We also define a noise-decorrelating colour space transform which provides a simple implementation of the ML formulation in the wavelet domain. Results for noisy synthesised colour sequences with known motion and noise statistics demonstrate the superiority of the exact ML formulation over straightforward, unweighted three-component estimation, most noticeably in high noise conditions.

We consider the transmission of MPEG-II compressed video over noisy wireless ATM channels. This has applications in mobile digital video, where high cell loss ratios are observed. We propose techniques for substantially improving the resilience of MPEG-II to cell losses without the addition of any redundancy. Using the techniques outlined in this paper, graceful degradation performance appears possible to cell loss ratios exceeding 1 in 10, long after unprotected MPEG-II pictures are lost.

This paper describes a new wavelet-based approach to the motion estimation problem for digital video. A complex-valued discrete wavelet transform is used to decompose each frame into a subsampled directionally bandpass filtered hierarchy. The transform is defined so that at each level there is an approximate correspondence between local translation and coefficient phase shift. This relationship is used to estimate motion within each orientation subband. The estimates are combined over all orientations and scales using a coarse-to-fine refinement strategy to produce a fractional-pel accurate motion field with a directional confidence measure. The technique is suitable for video compression schemes and can also be used for stereo vision and image registration.

This paper describes some improvements to our hierarchical wavelet-based algorithm for estimating motion in digital video (Magarey and Kingsbury, 1996). The algorithm uses a separable discrete wavelet transform (DWT) to analyse each frame into a multiresolution pyramid of subimages. Our analysis filters are Gabor-like (complex valued), so each subband at a given level of the pyramid contains information from a distinct spatial orientation. Starting at the coarsest (most sparsely sampled) level, our algorithm produces a single field of motion estimates from the oriented subimages at that level. Accompanying each motion estimate is a directionally dependent confidence measure. The estimate field is then progressively refined by incorporating the finer level subimages, using a standard coarse-to-fine control strategy. This paper describes an iterative strategy including a new interpolation scheme which increases accuracy at a given level, and a curvature correction method which improves the propagation of estimates from coarse to fine.

This paper is concerned with the performance of MPEG-II compressed video when transmitted over noisy channels, such as for digital terrestrial television. We present the results of bit sensitivity and resynchronisation sensitivity measurements, and propose techniques for substantially improving the resilience of MPEG-II to transmission errors without increasing the bit rate.

Many source and data compression schemes work by splitting the input signal into blocks and producing variable-length coded data for each block. If these variable-length blocks are transmitted consecutively, then the resulting coder is highly sensitive to channel errors. Synchronization code words are often used to provide occasional resynchronization at the expense of some added redundant information. This paper introduces the error-resilient entropy code (EREC) as a method for adapting existing schemes to give increased resilience to random and burst errors while maintaining high compression. The EREC has been designed to exhibit graceful degradation with worsening channel conditions. The EREC is applicable to many problems and is particularly effective when the more important information is transmitted near the start of each variable-length block and is not dependent on following data. The EREC has been applied to both still image and video compression schemes, using the discrete cosine transform (DCT) and variable-length coding. The results have been compared to schemes using synchronization code words, and a large improvement in performance for noisy channels has been observed.

A visual model that gives a distortion measure for blocking artifacts in images is presented. Given the original and reproduced image as inputs, the model output is a numerical value that quantifies the visibility of blocking error in the reproduced image. The model is derived based on the human visual sensitivity to horizontal and vertical edge artifacts that result from blocking. Psychovisual experiments have been carried out to measure the visual sensitivity to these artifacts. In the experiments, typical edge artifacts are shown to subjects and the sensitivity to them is measured with the variation of background luminance, background activity, edge length, and edge amplitude. Synthetic test patterns are used as background images in the experiments. The sensitivity measures thus obtained are used to estimate the model parameters. The final model is tested on real images, and the results show that the error visibility predicted by the model correlates well with the subjective ranking.

A visual model which gives a distortion measure for coding artifacts in images is presented. Given the original and the reproduced image as inputs, the model output is a numerical value, which quantifies the visibility of error in a region of the reproduced image. The model aims to predict accurately the visibility of the common types of coding artifacts blocking, ringing and blurring. Visual sensitivity to these artifacts is measured using psyche-visual experiments. The sensitivity measures thus obtained are used to derive the visual model which predicts the visibility of error. The final model is tested on real images, and the results show that the error visibility predicted by the model correlates well with the subjective ranking.>

We examine how image coding schemes can be designed for noisy channels such as radio links, cordless telephones or mobile communications. We start with a JPEG based scheme and discuss its performance over channels with random and burst errors. We propose a number of modifications to the JPEG scheme including: avoiding DPCM coding of the DC coefficients; using a highly protected short header, the error resilient entropy code (EREC), error concealment and a wavelet transform. We show that these methods can improve the error resilience, without any significant increase of bit rate or loss of error free picture quality.>

A visual model which gives a distortion measure for Blocking Artifacts in images is presented. Given the original and the reproduced image as inputs, the model output is a single numerical value which quantifies the visibility of blocking error in the reproduced image. The model is derived based on the human visual sensitivity to horizontal and vertical edge artifacts which result from blocking. Psychovisual experiments have been carried out using a novel experimental technique to measure the sensitivity to edge artifacts with the variation of edge length, edge amplitude, background luminance and background activity. The model parameters are estimated based on these sensitivity measures. The final model has been tested on real images, and the results show that the error predicted by the model correlate well with the subjective ranking.

An approach to designing multidimensional linear-phase FIR diamond subband filters having the perfect reconstruction property is presented. It is based on a transformation of variables technique and is equivalent to the generalized McClellan transformation. Methods for designing a whole class of transformation are given. The approach consists of two parts; design of the transformation and design of the 1-D filters. The use of Lagrange halfband filters to design the 1-D filters is discussed. The modification of a particular Lagrange halfband filter which gives a pair of simple 1-D filters that are almost similar to each other in their frequency characteristics but still form a perfect reconstruction pair is presented. The design technique is extended to other types of two-channel sampling lattice and subband shapes, in particular, the parallelogram and the diagonally quadrant subband cases. Several numerical design examples are presented to illustrate the flexibility of the design method.

A frequency-domain algorithm for motion estimation based on overlapped transforms of the image data is developed as an alternative to block matching methods. The complex lapped transform (CLT) is first defined by extending the lapped orthogonal transform (LOT) to have complex basis functions. The CLT basis functions decay smoothly to zero at their end points, and overlap by 2:1 when a data sequence is transformed. A method for estimating cross-correlation functions in the CLT domain is developed. This forms the basis of a motion estimation algorithm that calculates vectors for overlapping, windowed regions of data. The overlapping data window used has no block edge discontinuities and results in smoother motion fields. Furthermore, when motion compensation is performed using similar overlapping regions, the algorithm gives comparable or smaller prediction errors than standard models using exhaustive search block matching, and computational load is lower for larger displacement ranges and block sizes.

Structures for maximally decimated multidimensional filter banks which achieve perfect reconstruction are presented. There are no restrictions on the sampling lattice or dimensions. These are polyphase structures that can be cascaded to give finite impulse response (FIR) analysis and synthesis filters. The authors briefly review some important concepts and results of multidimensional filter banks and establish some notation. Then they present structures based on matrices whose elements are shifts. There are certain constraints that are imposed on these matrices. Next, structures based on elementary matrices that are used to effect elementary row or column operations are presented. Some useful properties of these structures are discussed.>

A frequency-domain algorithm for motion estimation based on overlapped transforms of the image data is developed. The transform used is referred to as the complex lapped transform (CLT) and is formed by extending the lapped orthogonal transform to have complex basis functions. It may also be viewed as a modified short time Fourier transform. A method for estimating cross-correlation functions in the CLT domain is derived. This forms the basis of a motion estimation algorithm that calculates vectors for overlapping, windowed regions of data, with no block edge discontinuities. When combined with motion compensation also using overlapping, windowed regions, the algorithm gives prediction error energies similar to or lower than those given by conventional block matching methods.>

In many source and data compression schemes, information relating to positions of high-energy samples or areas of importance often needs to be relayed to the decoder. The error resilient positional code (ERPC) is an efficient fixed-rate coding scheme for encoding such positional information, or, equivalently, sparse binary data patterns. It has also been designed with good channel error robustness properties, such that the decoded data quality degrades gracefully with worsening channel conditions, without the possibility of catastrophic breakdown or loss of sync. In this work, the coding efficiency of the ERPC is compared to a few other standard schemes. It is found to be efficient and its error extension in terms of the expected number of samples corrupted per bit error is reported and shown to be low and noncatastrophic. The ERPC is applied to an image coding example based on subband coding and vector quantization. It results in an efficient adaptive codec capable of operating in harsh channel conditions, without the aid of error correction or detection techniques.>

In many source and data compression schemes, information relating to positions of high energy samples or areas of importance often needs to be relayed to the decoder. The error resilient positional code (ERPC) is an efficient fixed rate coding scheme for encoding such positional information, or equivalently, sparse binary data patterns. It has also been designed with good channel error robustness properties, whose performance degrades gracefully with worsening channel conditions, without the possibility of breakdown or loss of sync. In this paper, the coding efficiency of the ERPC is compared to a few other standard schemes, and as well as being efficient, its error extension is shown to be low and non-catastrophic. The ERPC is then applied to an efficient error robust adaptive image coding example based on a SBC/VQ codec capable of operating in harsh channel conditions without the aid of channel coding.

A novel partially coherent detector for joint data detection and carrier phase estimation of continuous-phase-modulated (CPM) signal is presented. The algorithm differs from coherent Viterbi algorithm only in the metric which it maximizes over the possible transmitted data sequences. This metric is influenced both by the correlation of the hypothesized signal with the received one and the current carrier phase estimate. The system has been simulated and comparisons made with existing detection techniques on channels with additive white Gaussian noise and carrier phase jitter. Simulations of the detector are used to assess the relative susceptibility of different CPM (continuous phase modulation) schemes to phase jitter. The trend is that narrower band schemes are more sensitive to carrier phase inaccuracy than the less spectrally efficient ones.>

The use of the Viterbi algorithm to evaluate the minimum-distance parameters of trellis phase codes is extended to evaluate distances and paths of error events other than the free-distance event. Distances and paths are found for error events sequentially in order of increasing distance, starting with the free-distance event. Memory requirements increase slowly with the number of error events required. The time taken to find the next error event increases slowly with the number of error events already found. Results of sample calculations and computation times are included.>

A channel vocoder has been developed to operate in environments of high acoustic noise and high bit error rates. The 2400 b/s vocoder incorporates a robust pitch extractor and regenerator, automatic speech level control and forward error correction. It is implemented almost entirely digitally by using an lntel 8085 microprocessor system and a special purpose digital filter bank. The vocoder is currently packaged in a 1/4 ATR case (short) which occupies a volume of approximately 250 cubic ins, and it will meet full military environmental conditions. The first section of this paper deals with basic channel vocoder operation. The next section covers the implementation in the digital filter bank and microprocessor. The paper concludes with a brief description of the equipment and its performance in adverse conditions.