We present an application that demonstrates a proof of concept system for automated in-the-field monitoring of disease in wheat crops. Such in-situ applications are required to be robust in the presence of clutter, provide rapid and accurate analysis and are able to operate at scale. We propose a processing pipeline that detects key wheat diseases in cluttered field imagery. First, we describe and evaluate a high dimensional texture descriptor combined with a randomised forest approach for automated primary leaf recognition. Second, we show that a combined nearest neighbour classifier and voting system applied to segmented leaf regions can robustly determine the presence and type of disease. The system has been tested on a real-world database of images of wheat leaves captured in-the-field using a standard smart phone.

In this paper discrete wavelet transform (DWT) and empirical mode decomposition (EMD) are employed as a preprocessing stage in a multiclassifier and decision fusion system. The proposed method consists of three steps. In the first step, 2D-EMD is performed on each hyperspectral image band in order to obtain useful spatial information. Then, useful spectral information is obtained by applying the 1D-DWT to each signature of 2D-EMD performed bands. A novel feature set is generated using both spectral and spatial information. In the second step, each feature is independently classified by support vector machines (SVM), creating a multiclassifier system. In the last step, classification results are fused using a decision fusion criterion to produce one final classification. The proposed method improves overall classification accuracy over independent classifiers when reduced number of features are employed.

This paper presents a local disparity calculation algorithm on calibrated stereo images based on cost aggregation. Unlike most of the existing cost aggregation methods which are mainly based on the grouping of colour similarities, the proposed algorithm is grouped by local cost similarities. The proposed algorithm also applies a bilateral filter to enhance the normalised cost volume and, then, uses the winner-take-all technique to select the correspondence candidates. Finally, a quadratic polynomial interpolation is performed using the candidates and their neighbourhood values to achieve sub-pixel disparity resolution. The experimental results indicate that the proposed algorithm is able to provide dense disparity maps with sub-pixel resolution and achieves better accuracy compared to two similar stereo matching algorithms.

A channel adaptive multiple description video codec is presented with flexible redundancy allocation based on modeling and minimization of the end-to-end distortion. We employ a three-loop multiple description coding scheme for which we develop models that estimate the rate-distortion performance of the side encoders as well as the overall end-to-end distortion given channel statistics. A simple yet effective algorithm is formulated for determining appropriate levels of redundancy given a total bit rate and channel estimates in the form of packet error rates. The experimental results presented validate the proposed models over various channels conditions. The performance and adaptivity of the codec is evaluated through extensive simulations with a 22 wireless multiple input multiple output system. A gain of more than 10 dB can be achieved compared to a non-adaptive system and even larger gains can be had relative to typical single description transmissions.

The performance of an OFDM system can be improved by performing per-subcarrier antenna selection, which facilitates the exploitation of frequency and spatial diversity in the wireless channel. In this paper, we extend the concept of per-subcarrier antenna selection to a multiuser cognitive radio environment and present a practical subcarrier and antenna selection algorithm that exploits the multiuser, frequency and spatial diversities inherent in such systems while requiring only limited channel knowledge. The problem is formulated as an integer programming (IP) problem. We demonstrate that a linear relaxation of the problem still leads to an optimal solution, thus reducing the computational complexity relative to other approaches found in the literature. Simulation results demonstrate that the proposed resource allocation problem leads to an improvement in secondary users' link qualities, compared to a single-input single-output system and, at the same time, limits the interference to the primary user.

This paper presents an application of the Empirical Mode Decomposition (EMD) method to wavelet based dimensionality reduction, with an aim to generate the smallest set of features that leads to the best classification accuracy. Useful spectral information for hyper-spectral image (HSI) classification can be obtained by applying the Wavelet Transform (WT) to each hyperspectral signature. As EMD has the ability to describe short term spatial changes in frequencies, it helps to get a better understanding of the spatial information of the signal. In order to take advantage of both spectral and spatial information, a novel dimensionality reduction method is introduced, which relies on using the wavelet transform of EMD features. This leads to better class separability and hence to better classification. Specifically, the 2D-EMD is applied to each hyperspectral band and the 1D-DWT is applied to each EMD feature of all bands in order to get reduced Wavelet-based Intrinsic Mode Function Features (WIMF). Then, new features are generated by summing up the lower order WIMF features. The superiority of the proposed method compared to direct wavelet-based dimensionality reduction methods is proven by using the AVIRIS Indian Pine hyperspectral data. Compared to conventional direct wavelet-based dimensionality reduction methods, our proposed method offers up to 65% dimensionality reduction for the same classification performance.

Wireless sensor networks are an inherent part of decision making, object tracking, and location awareness systems. This work is focused on simultaneous localization of mobile nodes based on received signal strength indicators (RSSIs) with correlated in time measurement noises. Two approaches to deal with the correlated measurement noises are proposed in the framework of auxiliary particle filtering: with a noise augmented state vector and the second approach implements noise decorrelation. The performance of the two proposed multimodel auxiliary particle filters (MM AUX-PFs) is validated over simulated and real RSSIs and high localization accuracy is demonstrated.

Super-resolution and frame interpolation enhance low resolution low-framerate videos. Such techniques are especially important for limited bandwidth communications. This paper proposes a novel technique to up-scale videos compressed with H.264 at low bit-rate both in spatial and temporal dimensions. A quantisation noise model is used in the super-resolution estimator, designed for low bitrate video, and a weighting map for decreasing inaccuracy of motion estimation are proposed. Results show improvement both in rate-distortion and perceived image quality.

An automatic method for rapid and optimised surveillance camera deployment is proposed for outdoor settings. Given a target scene. potential camera locations are generated taking account of real-world camera parameters and environmental constraints. For example, each point on a target must be viewed by at least two cameras. We employ ray tracing to determine the visibility of any point from a given camera and camera-to-point viewing angle is used to measure visibility quality. Gradient ascent is used to optimise camera orientation for maximum quality coverage. Camera network optimisation was also addressed, and a new combinatorial algorithm is proposed. A comparative study showed that our approach significantly outperforms competing techniques providing solutions which better cover the search space, while offering multiple alternative solutions.

The aim of this paper is to introduce a novel statistical model-based image fusion method for Synthetic Aperture Radar (SAR) and optical images. The current fusion algorithms are effective only in specific areas of the scene. Hence, the fused image may not contain enough information for subsequent processing like classification and feature extraction. Our proposed method aims to keep the maximum contextual and spatial information from the source data by exploiting the relationship between spatial domain cumulants and wavelet domain cumulants. Our contributions are in integrating the relationship between spatial and wavelet domain cumulants of source images into an image fusion process as well as in employing these wavelet cumulants for optimization of weights in a Cauchy convolution based image fusion scheme. The superior performance of the proposed algorithm is demonstrated in comparison to existing fusion algorithms using real SAR and optical images.

This paper proposes a rate controlled redundancy-adaptive multiple description video coding method. The method adjusts the level of redundancy allocated to each macroblock based on an end to end distortion model that takes into account the effects of error propagation and concealment. Rate control at the macroblock level, ensures that the total bit-rate of all descriptions does not exceed the target rate. Results show that the proposed MDC codec can adapt very well to changing transmission conditions. When coupled with a MIMO architecture, performance gains of up to 14 dB were recorded over single description transmission at high packet error rates and no perceptible performance deficit at low packet error rates.

This paper describes a texture model application for future video compression algorithms. This employs region-based texture analysis techniques and advanced motion models to synthesise and warp video frames rather than encode the whole image or the prediction residual after traditional motion compensation. The proposed texture warping method and the dynamic texture model are integrated into an H.264 framework together with video quality assessment modules to prevent video artefacts. The results show impressive bitrate savings, up to 47%, over H.264 with similar visual quality.

This paper aims to combine adaptive subcarrier allocation and bit loading with the transmission of the H.264 SVC (Scalable Video Coding) encoded video sequences in order to increase the number of supported users in the system and provide the best quality of service (QoS) to the subscribers. We initially assume that the number of calls at the base station can be supported, and present an integer program (IP) formulation of the problem that considers the frequency selective nature of the channel, bit error rate requirement and the discrete rate requirements of the different layers of the medium grain scalable (MGS) video. It is shown how the IP can be extended to perform call admission control (CAC). Due to the complexity involved with IP, a sub-optimal scheme is then presented. Results demonstrate that our proposed scheme performs better than systems with a fixed resource allocation strategy by supporting more users and by always achieving acceptable QoS. Furthermore, the low complexity of the proposed CAC schemes makes it suitable for practical application.

This paper presents a novel framework to achieve scalable multiview image compression and view synthesis. The open-loop wavelet-lifting scheme for geometric filtering has been exploited to achieve signal-to-noise ratio scalability and view-type scalability (mono, stereo, or multiview). Spatial scalability is achieved by employing in-band prediction which removes correlations among subbands (level-by-level) via shift-invariant references obtained by overcomplete discrete wavelet transforms. We propose a novel in-band disparity compensated view filtering approach, akin to motion compensated temporal filtering, for achieving a scalable multiview codec. In our codec, hybrid prediction is proposed to deal with occlusions, and a novel cost function in dynamic programming (DP) for disparity estimation is introduced to improve view synthesis quality. Experiments show comparable results at full resolution and significant improvements at coarser resolutions, compared to a conventional spatial prediction scheme. View synthesis efficiency is extensively improved by utilizing disparity estimation from the proposed DP approach.

This letter presents a novel backward adaptive pixel-based fast predictive motion estimation (BAPME) scheme for lossless video compression. Unlike the widely used block-matching motion estimation techniques, this method predicts the motion on a pixel-by-pixel basis by comparing a group of past observed pixels in two adjacent frames, eliminating the need of transmitting side information. Combined with prediction and a fast search technique, the proposed algorithm achieves better entropy results and significant reduction in computation than pixel-based full search for a set of standard test sequences. Experimental results also suggest that BAPME is superior to block-based full search in terms of speed and zero-order entropy.

This paper describes a unified framework for object mining system for videos, which combines shot segmentation, clustering, retrieval and object mining using a single set of detected local invariant regions. The local invariant regions are tracked throughout a shot and stable tracks are extracted. The conventional key frame method is replaced with these stable tracks of local regions to characterize different shots. A grouping technique is introduced to combine the stable tracks into meaningful object clusters. These clusters are used to mine similar objects. Compared to other object mining systems, our approach mines more instances of similar objects in different shots. The proposed framework is applied to full length feature films and the results are compared with state of the art methods.

We propose a multicue gaze prediction framework for open signed video content, the benefits of which include coding gains without loss of perceived quality. We investigate which cues are relevant for gaze prediction and find that shot changes, facial orientation of the signer and face locations are the most useful. We then design a face orientation tracker based upon grid-based likelihood ratio trackers, using profile and frontal face detections. These cues are combined using a grid-based Bayesian state estimation algorithm to form a probability surface for each frame. We find that this gaze predictor outperforms a static gaze prediction and one based on face locations within the frame.

A novel region-based image fusion framework based on multiscale image segmentation and statistical feature extraction is proposed. A dual-tree complex wavelet transform (DT-CWT) and a statistical region merging algorithm are used to produce a region map of the source images. The input images are partitioned into meaningful regions containing salient information via symmetric alpha-stable (S alphaS) distributions. The region features are then modeled using bivariate alpha-stable (B alphaS) distributions, and the statistical measure of similarity between corresponding regions of the source images is calculated as the Kullback-Leibler distance (KLD) between the estimated B alphaS models. Finally, a segmentation-driven approach is used to fuse the images, region by region, in the complex wavelet domain. A novel decision method is introduced by considering the local statistical properties within the regions, which significantly improves the reliability of the feature selection and fusion processes. Simulation results demonstrate that the bivariate alpha-stable model outperforms the univariate alpha-stable and generalized Gaussian densities by not only capturing the heavy-tailed behavior of the subband marginal distribution, but also the strong statistical dependencies between wavelet coefficients at different scales. The experiments show that our algorithm achieves better performance in comparison with previously proposed pixel and region-level fusion approaches in both subjective and objective evaluation tests.

This paper explores the utilization of run-time partial dynamic reconfiguration in the LEON3 open-source soft core processor, which is a highly configurable SPARC (scalable processor architecture) V8 instruction set processor. The work explores the possibilities of sharing different arithmetic functions tightly coupled to the integer pipeline and mapped to the same silicon area, saving power consumption and area utilisation. The same strategy can be used to extend the instruction set architecture of the processor with new instructions that are optimized for DSP applications. The logic necessary to support these instructions could then be swapped as demanded by the application.

This work explores the potential of sharing different arithmetic hardware operators tightly coupled to the integer pipeline of the open-source LEON3 processor. The idea is to map these modules to the same silicon area saving power consumption and area utilisation. The same strategy can be used to extend the architecture of processors optimized for applications with specific energy constraints. The proposed platform serves as a guideline to illustrate gains obtained through partial reconfiguration that need to adapt to changing standards and protocols with a limited number of resources.

Contour tracking for a single source emission is addressed in this paper. This problem is solved by estimating the contour boundary positions using a set of particle filters. The use of Sequential Monte Carlo techniques enables the tracking to performed when the measurements are noisy and the tracking results also includes the estimation uncertainty. The proposed technique is illustrated for a SCIPUFF generated single emission scenario and simulation experiments showed the successful tracking throughout the tracking period.

A novel context enhancement technique is presented to automatically combine images of the same scene captured at different times or seasons. A unique characteristic of the algorithm is its ability to extract and maintain the meaningful information in the enhanced image while recovering the surrounding scene information by fusing the background image. The input images are first decomposed into multiresolution representations using the Dual-Tree Complex Wavelet Transform (DT-CWT) with the subband coefficients modelled as Cauchy random variables. Then, the convolution of Cauchy distributions is applied as a probabilistic prior to model the fused coefficients, and the weights used to combine the source images are optimised via Maximum Likelihood (ML) estimation. Finally, the importance map is produced to construct the composite approximation image. Experiments show that this new model significantly improves the reliability of the feature selection and enhances fusion process.

This paper proposes a gaze prediction model for open signed video content. A face detection algorithm is used to locate faces across each frame in both profile and frontal orientations. A grid-based likelihood ratio track before detect routine is used to predict the orientation of the signer's head, which allows the gaze location to be localised to either the signer or the inset. The face detections are then used to narrow down the gaze prediction further. The gaze predictor is able to predict the results of an eye tracking study with up to 95% accuracy, and an average accuracy of over 80%.

A novel technique for exploring the use of indexing metadata in improving coding efficiency is proposed in this paper. The technique uses an MPEG-7 descriptor as the basis for a fast mode decision algorithm for H.264/AVC encoders. The descriptor is used to form homogenous clusters for each frame, within which limited available coding modes for each macroblock are defined. The coding mode of an already coded macroblock that belongs to the same cluster in the same frame as well as the statistics of the coding modes of similar clusters in previous frames, are used for limiting the range of available coding modes within each cluster. The results show that the proposed algorithm is able to achieve an average of 47% time-saving when compared to the full search method and 21% when compared to the fast mode decision algorithm employed in the JM12.2 reference H.264 software encoder. In both cases, results yield only a small degradation in rate-distortion performance and a negligible loss in subjective quality.

Optical flow forms an important initial processing stage for many machine vision tasks. A framework is presented for the recovery of dense optical flows from image sequences containing large motions. Sparse feature correspondences are used to assign multiple optical flow hypotheses to each image pixel which are then independently refined to produce a further set of refined hypotheses. One final flow is selected for each pixel from these refined flows by seeking to minimize the local matching error. Dense optical flows from image sequences with small motions are successfully recovered. In image sequences with very large motions, a clear increase in optical flow accuracy is observed when compared to a hierarchical approach to optical flow estimation.

Compressive sensing (CS) has received a lot of interest due to its compression capability and lack of complexity on the sensor side. In this paper, we present a study of three sampling patterns and investigate their performance on CS reconstruction. We then propose a new image fusion algorithm in the compressive domain by using an improved sampling pattern. There are few studies regarding the applicability of CS to image fusion. The main purpose of this work is to explore the properties of compressive measurements through different sampling patterns and their potential use in image fusion. The study demonstrates that CS-based image fusion has a number of perceived advantages in comparison with image fusion in the multiresolution (MR) domain. The simulations show that the proposed CS-based image fusion algorithm provides promising results.

Traditional volumetric scene reconstruction algorithms involve the evaluation of many millions of voxels which is highly time consuming. This paper presents an efficient algorithm based of future frame prediction that can dramatically reduce the number of voxels to be evaluated in time varying scenes. The new prediction method, combining scene flow and morphological dilations, is evaluated against a simple model dilation method. Results show the proposed method outperforms a simple dilation method and has the potential to improve the efficiency of volumetric scene reconstruction algorithms while retaining quality given accurate optical flows. 1

This paper investigates the impact of pixel-level fusion of videos from visible (VIZ) and infrared (IR) surveillance cameras on object tracking performance, as compared to tracking in single modality videos. Tracking has been accomplished by means of a particle filter which fuses a colour cue and the structural similarity measure (SSIM). The highest tracking accuracy has been obtained in IR sequences, whereas the VIZ video showed the worst tracking performance due to higher levels of clutter. However, metrics for fusion assessment clearly point towards the supremacy of the multiresolutional methods, especially Dual Tree-Complex Wavelet Transform method. Thus, a new, tracking-oriented metric is needed that is able to accurately assess how fusion affects the performance of the tracker.

Advances in fusion of multi-sensor inputs have necessitated the creation of more sophisticated fused image assessment techniques. The current work extends previous studies investigating participant accuracy in tracking individuals in a video sequence. Participants were shown visible and IR videos individually and the two video inputs side-by-side, as well as averaged, discrete wavelet transform, and dual- tree complex wavelet transform fused videos. Two scenarios were shown to participants: one featured a camouflaged man walking down a pathway through foliage and across a clearing; the other featured several individuals moving around the clearing. The side-by-side scanpath data were analysed by studying how often participants looked at the visible and infrared sides, and analysing how accurately participants tracked the given target, and compared with previously analysed data. The results of this study are discussed in the context of wider applications to image assessment, and the potential for modelling human scanpath performance.

A new method for multimodal image fusion, based on statistical modelling of wavelet coefficients, is proposed in this paper. The algorithm draws from the Weighted Average scheme, but incorporates Laplacian bivariate parent-child statistical dependencies. The interscale dependency is brought in the form of shrinkage functions. The proposed method has been shown to perform very well with noisy datasets, outperforming other conventional methods in terms of fusion quality and noise reduction in the fused output.

The colour volumetric data which is constructed from a set of multi-view images is capable of providing realistic immersive experience. However it is not widely applicable due to its manifold increase in bandwidth. This paper presents a novel framework to achieve scalable volumetric compression. Based on wavelet transformation, data rearrangement algorithm is proposed to compact volumetric data leading to high efficiency of transformation. The colour data is also rearranged by using the characteristics of human eye sensitivity. Moreover, the pre-processing for adaptive resolution is proposed in this paper. The low resolution overcomes the limitation of the data transmission at low bit rate, whilst the fine resolution improves the synthesised images' quality. The results of our proposed schemes show significant improvement of the compression performance over the traditional 3D coding.

A novel multiscale image segmentation algorithm is presented, which is based on the dominant color and homogeneous texture features (HTF) that are adopted in the MPEG-7 standard. These features are efficiently combined to perform the automatic segmentation. First, the image is roughly segmented into textured and nontextured regions using Gabor decomposition. A multiscale segmentation is then applied to the resulting regions, according to the local texture feature. Finally, a precise boundary refinement procedure is employed to accurately determine the boundaries between textured and nontextured regions. A novel region merging algorithm is introduced with a simple and effective segment classification by using HTF to deal with the over-segmentation problem. Experiments show that our algorithm provides an improved performance compared with JSEG and a watershed algorithm.

This paper describes a novel object mining system for videos. An algorithm published in a previous paper by the authors is used to segment the video into shots and extract stable tracks from them. A grouping technique is introduced to combine these stable tracks into meaningful object clusters. These clusters are used in mining similar objects. Compared to other object mining systems, our approach mines more instances of similar objects in different shots. The proposed framework is applied to a full length feature film and improved results are shown.

In this paper an algorithm for real-time unsupervised segmentation and tracking of a moving object is proposed. This is performed within the compressed domain using motion information only. Initial object segmentation is done using iterative rejection, taking advantage of its computational efficiency. The system seeks to overcome its disadvantages, namely a delay in object macroblocks appearing after consistency checking and non-identification of macroblocks containing object boundaries, by taking a model based approach to object tracking. The output of iterative rejection is used to update the model after tracking has taken place in each frame. Experimental results on a number of MPEG-2 encoded sequences demonstrate its effectiveness in identifying and tracking an object of interest from a compressed video stream and that the system is better than purely using iterative rejection as a segmentation method.

This paper presents a new statistical image segmentation algorithm, in which the texture features are modeled by symmetric alpha-stable (SalphaS) distributions. These features are efficiently combined with the dominant color feature to perform automatic segmentation. First, the image is roughly segmented into textured and nontextured regions using the dual-tree complex wavelet transform (DT-CWT) with the sub-band coefficients modeled as SalphaS random variables. A mul-tiscale segmentation is then applied to the resulting regions, according to the local texture characteristics. Finally, a novel statistical region merging algorithm is introduced by measuring the Kullback-Leibler distance (KLD) between estimated SalphaS models for the neighboring segments. Experiments show that our algorithm achieves superior segmentation results in comparison with existing state-of-the-art image segmentation algorithms.

The purpose of the Applied Multi-dimensional Fusion Project is to investigate the benefits that data fusion and related techniques may bring to future military Intelligence Surveillance Target Acquisition and Reconnaissance systems. In the course of this work, it is intended to show the practical application of some of the best multi-dimensional fusion research in the UK. This paper highlights the work done in the area of multi-spectral synthetic data generation, super-resolution, joint fusion and blind image restoration, multi-resolution target detection and identification and assessment measures for fusion. The paper also delves into the future aspirations of the work to look further at the use of hyper-spectral data and hyper-spectral fusion. The paper presents a wide work base in multi-dimensional fusion that is brought together through the use of common synthetic data, posing real-life problems faced in the theatre of war. Work done to date has produced practical pertinent research products with direct applicability to the problems posed.

This article discusses a framework for model-based, context-dependent video coding based on exploitation of characteristics of the human visual system. The system utilizes variable-quality coding based on priority maps which are created using mostly context-dependent rules. The technique is demonstrated through two case studies of specific video context, namely open signed content and football sequences. Eye-tracking analysis is employed for identifying the characteristics of each context, which are subsequently exploited for coding purposes, either directly or through a gaze prediction model. The framework is shown to achieve a considerable improvement in coding efficiency.

Mobility tracking based on data from wireless cellular networks is a key challenge that has been recently investigated both from a theoretical and practical point of view. This paper proposes Monte Carlo techniques for mobility tracking in wireless communication networks by means of received signal strength indications. These techniques allow for accurate estimation of mobile station's (MS) position and speed. The command process of the MS is represented by a first-order Markov model which can take values from a finite set of acceleration levels. The wide range of acceleration changes is covered by a set of preliminary determined acceleration values. A particle filter and a Rao-Blackwellised particle filter are proposed and their performance is evaluated both over synthetic and real data. A comparison with an extended Kalman filter (EKF) is performed with respect to accuracy and computational complexity. With a small number of particles the RBPF gives more accurate results than the PF and the EKF. A posterior Cramer Rao lower bound (PCRLB) is calculated and it is compared with the filters' root- mean-square error performance.

In this paper, we present a novel multimodal image fusion algorithm in ICA domain. It uses segmentation to determine the most important regions in the input images and consequently fuses the ICA coefficients from given regions using the Piella fusion metric to maximise the quality of the fused image. The proposed method exhibits significantly higher performance than the basic ICA algorithm and improvement over other state-of-the-art algorithms

Image fusion is the process of combining images of differing modalities, such as visible and infrared (IR) images. Significant work has recently been carried out comparing methods of fused image assessment, with findings strongly suggesting that a task-centred approach would be beneficial to the assessment process. The current paper reports a pilot study analysing eye movements of participants involved in four tasks. The first and second tasks involved tracking a human figure wearing camouflage clothing walking through thick undergrowth at light and dark luminance levels, whilst the third and fourth task required tracking an individual in a crowd, again at two luminance levels. Participants were shown the original visible and IR images individually, pixel-averaged, contrast pyramid, and dual-tree complex wavelet fused video sequences. They viewed each display and sequence three times to compare inter-subject scanpath variability. This paper describes the initial analysis of the eye-tracking data gathered from the pilot study. These were also compared with computational metric assessment of the image sequences

A number of segmentation techniques are compared with regard to their usefulness for region-based image and video fusion. In order to achieve this, a new multi-sensor data set is introduced containing a variety of infra-red, visible and pixel fused images together with manually produced "ground truth" segmentations. This enables the objective comparison of joint and unimodal segmentation techniques. A clear advantage to using joint segmentation over unimodal segmentation, when dealing with sets of multi-modal images, is shown. The relevance of these results to region-based image fusion is confirmed with task-based analysis and a quantitative comparison of the fused images produced using the various segmentation algorithms

This paper addresses the problem of object tracking in video sequences. The use of a structural similarity measure for tracking is proposed. The measure reflects the distance between two images by comparing their structural and spatial characteristics and has shown to be robust to illumination and contrast changes. As a result it guarantees robustness of the tracking process under changes in the environment. The previously used Bhattacharyya distance is not robust to such changes. Additionally, when a tracker is run with the Bhattacharyya distance, histograms should be calculated in order to find the likelihood function of the measurements. With the new function there is no need to calculate histograms. A particle filter (PF) is implemented where this measure is used for computing the distance between the reference and current frame. The algorithm performance has been tested and evaluated over real-world video sequences, and has been shown to outperform methods based on colour and edge histograms

This paper addresses the problem of position localisation of mobile nodes in ad hoc wireless networks based on received signal strength indicator measurements. Node mobility is modelled as a linear system driven by a discrete command Markov process. Self-localisation of mobile nodes is performed via an interacting multiple model filter consisting of a bank of unscented Kalman filters (IMM-UKF). Estimation of the mobility state, which comprises the position, speed and acceleration of the mobile nodes is accomplished. The performance of the IMM- UKF filter is investigated and compared to a multiple model particle filter (MM PF) by Monte Carlo simulation

In this paper we present a novel fusion method in ICA domain. The method uses nonlinear shrinkage of the coefficients in ICA domain to determine the ICA coefficients to be used in the fused image reconstruction, so that the noise transferred from input images into the fused output is minimized. Experimental results have shown that it has improved performance in the presence of noise, compared to the basic ICA algorithm. Compared to standard multiresolutional fusion methods, the noise in the fused image is visually less annoying, while the important detail is still adequately represented. The proposed method exhibits considerably higher fusion performance, measured by Piella and Petrovic fusion metric, than the basic ICA algorithm and improvement over other state-of-the-art algorithms.

A novel algorithm for disparity/depth estimation from multi-view images is presented. A dynamic programming approach with window-based correlation and a novel cost function is proposed. The smoothness of disparity/depth map is embedded in dynamic programming approach, whilst the window-based correlation increases reliability. The enhancement methods are included, i.e. adaptive window size and shiftable window are used to increase reliability in homogenous areas and to increase sharpness at object boundaries. First, the algorithms estimate depth maps along a single camera axis. The algorithms exploits then combines the depth estimates from different axis to derive a suitable depth map for multi-view images. The proposed scheme outperforms existing approaches in parallel and in the non-parallel camera configurations

This paper describes a modified rate control algorithm for H.264 that can accommodate multiple priority levels given a region of interest (Rol). The modified method allows better control of the quality of the RoI and gradual variation of the quality in the rest of the video frame through a bit redistribution process that is based on a number of parameters, including characteristics of the Rol, user input and perceptual factors.

In this paper, we propose a novel volumetric representation for a sparse set of calibrated multi-view images of a non-Lambertian scene. The depth map of each reference view is registered into a volume and a simple algorithm to shape the volume is introduced. Particular colours are defined for each voxel to render a smooth and realistic image. Synthesized results demonstrate the good performance with the proposed scheme, both for parallel-camera and non-parallel-camera geometries.

This paper describes a novel method for content extraction and scene retrieval for video sequences based on local region descriptors. The local invariant features are obtained for all frames in a sequence and tracked throughout the shot to extract stable features. The scenes in a shot are represented by these stable features rather than features from one or more key frames. Compared to previous key frame based approaches, the proposed method is highly robust to camera and object motions and can withstand severe illumination changes. The proposed approach is applied to scene retrieval experiments and excellent performance is demonstrated.

In this paper, we present a novel region-based multimodal image fusion algorithm in the ICA domain. It uses segmentation to determine the most important regions in the input images and consequently fuses the ICA coefficients from the given regions. The proposed method exhibits considerably higher performance than the basic ICA algorithm and shows improvement over other state-of-the-art algorithms

This paper presents an adaptive in-band motion compensated temporal filtering (MCTF) scheme for 3D wavelet based scalable video coding. The proposed scheme solves the motion mismatch problem when motion vectors from the LL subband are inaccurately applied to the highpass subbands in decoding high spatial resolution video. Specifically, we compare the macroblock residue energy in the highpass frames obtained by using motion vectors from both the LL and highpass subbands, and then adaptively transmit different sets of motion vectors based on whether mismatch has occurred in the highpass subbands. Macroblocks in the higher temporal levels favour the selection of highpass subbands' motion vectors because the motion estimation process becomes less accurate as temporal level increases. The modes information, which specifies whether the LL subband motion vectors or the highpass subbands' motion vectors are used by the current macroblock, is coded by run-length coding. Experimental results show that the proposed scheme improves both the visual quality and PSNR for high resolution decoding with comparison to other in-band MCTF schemes. Furthermore, our scheme requires only modifications when performing MCTF in the highpass subbands, thus, the original strength of in-band MCTF for decoding low spatial resolution video is well preserved.

This paper describes a spatial error concealment method that uses edge related information for concealing missing macroblocks in a way that not only preserves existing edges but also avoids introducing new strong ones. The method relies on a novel switching algorithm which uses the directional entropy of neighboring edges for choosing between two interpolation methods, a directional along detected edges or a bilinear using the nearest neighboring pixels. Results show that the performance of the proposed method is subjectively and objectively (PSNR wise) better compared to both 'single interpolation' and to edge strength based switching methods

The latest video coding standard H.264 has been recently approved and has already been adopted for numerous applications including HD-DVD and satellite broadcast. To allow interconnectivity between different applications using H.264, transcoding will be a key factor. When requantizing a bit stream the incoming coding decisions are usually kept unchanged to reduce the complexity, but it can have a major impact on the coding efficiency. This paper proposes a novel algorithm for mode refinement of intra prediction in the case of requantization of H.264 intra frames. The proposed approach gives a comparable quality to a full search for a fraction of its complexity by exploiting the statistical properties of the mode distribution

The prevalence of image fusion---the combination of images of different modalities, such as visible and infrared radiation---has increased the demand for accurate methods of image-quality assessment. The current study used a signal-detection paradigm, identifying the presence or absence of a target in briefly presented images followed by an energy mask, which was compared with computational metric and subjective quality assessment results. In Study 1, 18 participants were presented with fused infrared-visible light images, with a soldier either present or not. Two independent variables, image-fusion method (averaging, contrast pyramid, dual-tree complex wavelet transform) and JPEG compression (no compression, low and high compression), were used in a repeated-measures design. Participants were presented with images and asked to state whether or not they detected the target. In addition, subjective ratings and metric results were obtained. This process was repeated in Study 2, using JPEG2000 compression. The results showed a significant effect for fusion but not compression in JPEG2000 images, while JPEG images showed significant effects for both fusion and compression. Subjective ratings differed, especially for JPEG2000 images, while metric results for both JPEG and JPEG2000 showed similar trends. These results indicate that objective and subjective ratings can differ significantly, and subjective ratings should, therefore, be used with care.

Delay sensitive video transmission over error prone networks can suffer from packet erasures when channel conditions are not favorable. Use of error concealment (EC) at the video decoder is necessary in such cases to prevent error induced artefacts making the affected video frames visibly intolerable. This paper proposes an EC method that incorporates enhanced temporal and spatial concealment elements, the use of which is controlled by a mode selection (MS) algorithm well matched to the characteristics of the temporal concealment approach. The performance of the individual enhancements and of the MS algorithm are compared with the respective features of the method employed in the H.264 joint model (JM) decoder and with other state of the art methods. The overall performance of the proposed method is shown to offer significant gains (up to 9 dB) compared to that of the JM decoder for a wide range of natural and animation image sequences without any considerable increase in complexity

Subpixel motion estimation plays an important role in compression efficiency within modern video codecs such as MPEG2, MPEG4, and H.264. Subpixel motion estimation is implemented within these standards using interpolated values at 1/2 or 1/4 pixel accuracy. Such interpolation gives a good reduction in residual energy for each predicted macroblock and, therefore, improves compression. However, this leads to a significant increase in computational complexity at the encoder. This is especially true for H.264 where the cost of an exhaustive set of macroblock segmentations need to be estimated in order to obtain an optimal mode for prediction. This paper presents a novel interpolation-free scheme for subpixel motion estimation using the result of the full pixel sum of absolute difference distribution of each motion compensated block applied to an H.264 encoder. This system produces reduced complexity motion estimation with a controllable tradeoff between compression performance and encoder speed. These methods facilitate the generation of a real time software H.264 encoder

This paper presents a novel framework to achieve scalable multi-view image coding. As open loop operation, the wavelet lifting scheme for geometric filtering has been exploited to overcome the limitation of SNR scalability and to attain view scalability. The essential key for achieving the spatial scalability is the in-band prediction. It removes correlations among subbands level-by-level via shift-invariant references obtained by overcomplete discrete wavelet transforms (ODWT). Additionally, the proposed disparity compensated view filtering is allowed to exploit the different filters and estimation parameters for each resolution level. The experiments show comparable results at full resolution and the significant improvement at coarser resolution over the conventional spatial prediction scheme.

Image fusion is the process of extracting meaningful visual information from two or more images and combining them to form one fused image. Image fusion is important within many different image processing fields from remote sensing to medical applications. Previously, real valued wavelet transforms have been used for image fusion. Although this technique has provided improvements over more naive methods, this transform suffers from the shift variance and lack of directionality associated with its wavelet bases. These problems have been overcome by the use of a reversible and discrete complex wavelet transform (the dual tree complex wavelet transform DT-CWT). However, the existing structure of this complex wavelet decomposition enforces a very strict choice of filters in order to achieve a necessary quarter shift in coefficient output. This paper therefore introduces an alternative structure to the DT-CWT that is more flexible in its potential choice of filters and can be implemented by the combination of four normally structured wavelet transforms. The use of these more common wavelet transforms enables this method to make use of existing optimised wavelet decomposition and recomposition methods, code and filter choice.

This paper develops error concealment methods for multiple description video coding (MDC) in order to adapt to error prone packet networks. The three-loop slice group MDC approach of D. Wang et al. (2005) is used. MDC is very suitable for multiple channel environments, and especially able to maintain acceptable quality when some of these channels fail completely, i.e. in an on-off MDC environment, without experiencing any drifting problem. Our MDC scheme coupled with the proposed concealment approaches proved to be suitable not only for the on-off MDC environment case (data from one channel fully lost), but also for the case where only some packets are lost from one or both channels. Copying video and using motion vectors from correct descriptions are combined together for concealment prior to applying traditional methods. Results are compared to the traditional error concealment method proposed in the H.264 reference software, showing significant improvements for both the balanced and unbalanced channel cases.

This paper describes a novel gaze-contingent display (GCD) using texture mapping and OpenGL. This new system has a number of key features: (a) it is platform independent, i.e. it runs on different computers and under different operating systems; (b) it is eyetracker independent, since it provides an interactive focus+context display that can be easily integrated with any eye-tracker that provides real-time 2-D gaze estimation; (c) it is flexible in that it provides for straightforward modification of the main GCD parameters, including size and shape of the window and its border; and (d) through the use of OpenGL extensions it can perform local real-time image analysis within the GCD window. The new GCD system implementation is described in detail and some performance figures are given. Several applications of this system are studied, including gaze-contingent multi-resolution displays, gaze-contingent multi-modality displays, and gaze-contingent image analysis.

Underdetermined blind source separation is a key application in audio where it is desirable to extract multiple sources from a stereo recording. A new variant on the stereo matching pursuit, the dual matching pursuit, is presented whereby independent matching pursuits are run on both channels of a stereo mixture of greater than two sources. By identifying correlating atoms from each decomposition, a histogram plot is applied to identify the position of each source in the stereo image and the atoms grouped to recover the original signals. To improve the atomic correlation between channels, a fixed overcomplete representation for each of the signal types present in a mixture is obtained by applying a learning algorithm to existing sources of that type and reducing the redundancy in the resulting basis set via a correlation-based algorithm. The resulting dictionaries are then used as a time-frequency basis for the independent matching pursuits. The results show improved separation quality compared to the dual matching pursuit with mathematical time-frequency dictionaries. The noise immunity of this method due to the use of overcomplete representations is also demonstrated showing that the system can withstand mixture signal-to-noise ratios down to 30 dB.

The ability to communicate remotely through the use of video as promised by wireless networks and already practised over fixed networks, is for deaf people as important as voice telephony is for hearing people. Sign languages are visual-spatial languages and as such demand good image quality for interaction and understanding. In this paper, based on analysis of the viewer's perceptual behavior and the video content involved we propose a sign language video coding system using foveated processing, which can lead to bit rate savings without compromising the comprehension of the coded sequence. We support this claim with the results of an initial comprehension assessment trial of such coded sequences by deaf users.

This paper develops an efficient parametric multiresolution region tracking algorithm which is applied to the task of object replacement in video sequences. The tracker relies on gradient-based techniques to provide efficient estimation of the target location and pose in each frame. The tracking algorithm improves upon similar efficient parametric algorithms by increasing the distance an object can move from frame to frame. Experimental results provide clear evidence of the improved performance over existing region tracking. The parameters that estimate the pose and location of the target are used to initialise the object replacement algorithm. The object replacement uses the pose estimates with texture mapping techniques to perform image warping of an arbitrary sized replacement image. The location estimates are used to accurately insert the replacement image into the current frame.

This paper proposes a novel scheme for multiple description video coding approach using slice group coding tool proposed in H.264. Independent motion compensation loops are maintained for two descriptions in the encoder. In each description, one slice group is encoded as main information, while the other slice group is encoded very coarsely, as redundancy, to keep basic information. This coarse slice group can be encoded using normal encoding with coarse quantizer, or using predicted motion vectors by the main slice group. Mode decision is made to select best encoding method. Results show that our scheme works very well and keeps subjective quality very good for middle and higher bitrate. Redundancy and side quality can be controlled by changing parameters of coarsely coded slice group, unlike some other MDC methods with fixed redundancy.

This paper addresses two issues related to motion estimation using the block matching algorithms (BMA): (1) determining the reliability of the motion vectors of each block, and (2) imposing smoothness constraint to the motion vector field. We introduce a new robust reliability measure to represent the confidence level of the motion vector from the cost function distribution and propose a novel algorithm that incorporates smoothness constraint into the motion vector field evaluation by implementing a priority queue structure based on the reliability measure. In this framework, a smooth motion vector field is evaluated in a single pass without going through iterations typical of many existing optical flow estimation algorithms. Hence it is fast and can easily be incorporated into real-time applications for video compression as well as image segmentation.

This paper proposes a low-complexity sub-pixel refinement to motion estimation based on full-search block matching algorithm (BMA) at integer-pixel accuracy. This algorithm eliminates the need to produce interpolated reference frames, which is may be too memory- and processor- intensive, for some real-time mobile applications. The algorithm assumes the BMA is done at pixel resolution and the (sum-of-absolute-differences) SADs of the candidate motion vector and its neighbouring vectors are available for each block. The proposed method than models the SAD distribution around the candidate motion vector and its neighbouring points. Actual minimum point at sub-pixel resolution is then computed according to the model used. 3 variations of the parabolic model are considered and simulations using the H.263 standard encoder on several test sequences reveal an improvement of 1.0 dB over integer-accuracy motion estimation. Albeit its simplicity, some test cases come close to the results obtained by actual interpolated reference frames.

The reassignment procedure has often been employed to improve readability of some time-frequency representations (TFRs). When processing noisy signals, the problem of sensitivity of the technique to noise is encountered. A simple modification of the reassignment method is proposed, based on a thresholding operation. Specifically, by preventing the reassignment of the distribution coefficients below the noise dependent threshold and replacing them with zeros, the enhanced signatures on the time-frequency plane are obtained. This method is compared with other techniques, such as the reassigned spectrogram (RSP) and the supervised reassigned spectrogram (SRSP). An experimental test of these algorithms as the instantaneous frequency (IF) estimators for a chirp signal have shown that our method improves the accuracy of the estimation for heavy noise.

Three-dimensional wavelet coding of volumetric medical images provides better coding performance compared to corresponding 2D methods by exploiting the inter-slice correlation that exists in such data. It introduces however latencies when it comes to transmitting specific parts of the volume. This paper presents an extension to 3D-SPlHT which allows 3D region of interest (ROI) coding. ROl coding enables faster reconstruction of diagnostically useful regions in volumetric datasets by assigning higher priority to them in the bitstream. It also introduces the possibility for increased compression performance, by allowing certain parts of the volume to be coded in a lossy manner while others are coded losslessly. The necessary modifications to 3D-SPIHT for ROI coding are described and methods for specifying a 3D ROI without adding a significant overhead are suggested. Results are presented highlighting the benefits of the ROl extension.

The segmentation of images into meaningful and homogenous regions is a key method for image analysis within applications such as content based retrieval. The watershed transform is a well established tool for the segmentation of images. However, watershed segmentation is often not effective for textured image regions that are perceptually homogeneous. In order to segment such regions properly, the concept of the "texture gradient" is introduced. Texture information and its gradient are extracted using a novel nondecimated form of a complex wavelet transform. A novel marker location algorithm is subsequently used to locate significant homogeneous textured or non textured regions. A marker driven watershed transform is then used to segment the identified regions properly. The combined algorithm produces effective texture and intensity based segmentation for application to content based image retrieval.

The fusion of images is the process of combining two or more images into a single image retaining important features from each. Fusion is an important technique within many disparate fields such as remote sensing, robotics and medical applications. Wavelet based fusion techniques have been reasonably effective in combining perceptually important image features. Shift invariance of the wavelet transform is important in ensuring robust subband fusion. Therefore, the novel application of the shift invariant and directionally selective Dual Tree Complex Wavelet Transform (DT-CWT) to image fusion is now introduced. This novel technique provides improved qualitative and quantitative results compared to previous wavelet fusion methods. 1

The watershed transform is a well established tool for the segmentation of images. However, watershed segmentation is often not effective for textured regions that are perceptually homogeneous. Such regions are usually inaccurately over-segmented with no reference to any texture changes. We now introduce a novel concept of “texture gradient” implemented using a non-decimated complex wavelet transform. A novel marker location algorithm is subsequently used to locate significant homogeneous textured or non textured regions. A marker driven watershed transform is then used to properly segment the identified regions. The combined algorithm produces effective texture and intensity based segmentation for the application to content based retrieval of images.

The widespread adoption of digital video techniques has generated a requirement for authenticity verification in applications such as criminal evidence, insurance claims and commercial databases. This work addresses problems that arise from a spatial digital watermarking technique developed to detect frame reordering and dropping scenarios. It discusses the differences between mutual frame types at different bit-rates. Many papers consider detection after MPEG-2 decoding as a naïve approach. However, this approach does offer significant advantages for a slight increase of computation load. This paper also establishes a link between the detector performance and the sequence content. The uniqueness of this work is the comparison of the test results using 18 different standard MPEG test-sequences. The functionality of the algorithm is demonstrated with a simulated attack.

This paper presents results in the image processing and visualization aspects of a virtual liver biopsy system (a system for simulating the medical procedure of liver biopsy). The creation of 3D models from 2D images of the organs involved is described, and segmentation requirements of this process are discussed. Endoscopic images of the liver that simulate the needle's point of view are created by means of combined volume and surface rendering. For this purpose ray casting is used with the ray start and end points being constrained within a surface rendered environment. Visualization of the needle insertion process from an exterior point of view is presented. A real-time sectional imaging component is also used in which the displayed 2D-image section of the 3D volume tracks the tip of the needle.

Simple wavelet and wavelet packet transforms have often been used for texture characterisation through the analysis of spatial-frequency content. However, most previous methods make no use of any statistical analysis of the transforms' subbands. A novel method is now presented for modelling the multivariate distributions of subband coefficients by considering spatially related coefficients. The Bhattacharya and divergence metrics are then used to produce an improved texture classification method for the application to content based image retrieval.

Content-based retrieval is ultimately dependent on the features used for the annotation of data and its efficiency is dependent on the invariance and robust properties of these features. For texture-based features an important form of invariance is rotational invariance. Novel rotationally invariant texture-based features are introduced that are extracted from a polar Fourier transform (PFT). The PFT is similar to the discrete Fourier transform in two dimensions but uses transform parameters radius and angle rather than the Cartesian co-ordinates. The PFT is discretised appropriately across the angular and radial frequency space with the transform magnitudes forming the rotationally invariant features. These features although rotationally invariant, capture the angular distribution together with the radial distribution of frequency within texture. Preliminary results show the method to give better results than rotationally variant and invariant Gabor filter schemes.

A new genetic algorithm-based optimisation technique for stereo matching using complex conjugate wavelet pyramids is proposed. Reliable disparity fields are estimated in the wavelet domain with low computational cost. The new cost function is composed of the differences in wavelet coefficient values, plus vertical discontinuity and ordering constraints. Within homogenous regions, smoothness constraints on the disparity field are also employed. A genetic algorithm is used, where previously estimated vectors at the former image hierarchy are used to predict the corresponding search space of chromosomes, and to correct each newly calculated set of disparity vectors. This significantly reduces computational complexity compared to other methods, whilst maintaining robust performance.

This paper proposes a reliable method for tracking the trajectory of video objects using the vector Kalman predictor. Video objects, within the scope of this paper, are defined as groups of image pixels coherent spatially as well as in their values of luminance. The extent to which the quality of the unsupervised region split and merge segmentation affects the accuracy of the tracker is discussed, alongside the improvements made in the segmentation process as a result of the feedback from the tracking algorithm. The overall low complexity of the system, and the time savings made in using a spiral search algorithm, provide this method with prospects of being implemented in real-time.

We examine the performance of a space-time coded (STC) OFDM system. As an example, two emerging WLAN standards (IEEE 802.11a and ETSI HIPERLAN/2) are modified to accommodate the STC technique. Performance results using statistical multi-element channel models are compared with those using state-of-the-art indoor MIMO channel measurements. We conclude that the gap between the ideal setting and our measured channel is 4, 5 and 6 dB for 1, 2 and 3 receive antennas respectively. We observe also that STC with (2Tx, 3Rx) over the measured channels doubles the capacity as compared to the equivalent mode of HIPERLAN/2.

The benefits of polarisation diversity as a fade countermeasure within second generation cellular networks are well known. Capacity enhancement by means of joint detection is also a well-regarded technique for interference-limited systems such as TD-CDMA (time division-code division multiple access) within the UMTS (Universal Mobile Telecommunication System) wireless standards. However, relatively little attention has been focussed towards the potential gains of combining these two technologies. This paper considers, by means of a series of wideband dual polar channel measurements from a field trials campaign taken within the lower UMTS band (1910-1915 MHz), the relative merits of this approach when applied to a multistage interference canceller. This paper shows that signal strength improvements of between 2.5 dB and 3.2 dB at the 1% cumulative level were observed by applying dual polarised antennas at the receiver.

Block-matching motion estimation (BMME) can be formulated as a 2-D constrained minimization problem. This problem can, therefore, be solved with reduced complexity using optimization techniques. This paper proposes a novel fast BMME algorithm called the simplex minimization search (SMS). The algorithm is based on the simplex minimization (SM) optimization method. The initialization procedure, termination criterion, and constraints on the independent variables of the search are designed to take advantage of the characteristics of the BMME problem and the properties of the block motion fields of typical video sequences. Simulation results show that the proposed algorithm outperforms other fast BMME algorithms, providing better prediction quality, a smoother motion field, and higher speed-up ratio. This paper also investigates the properties of the multiple-reference (MR) block motion field. Guided by the results of this investigation, the paper extends the SMS algorithm to the MR case. Three MR SMS algorithms are proposed, providing different degrees of compromise between prediction quality and computational complexity. Simulation results using 50 reference frames indicate that the proposed MR algorithms have a computational complexity comparable to that of single-reference full-search while still maintaining the prediction gain of MR motion estimation.

Matching pursuits over a basis of separable Gabor functions has been demonstrated to outperform DCT methods for low bit rate video coding. This paper introduces an error resilient implementation of the matching pursuits algorithm, based on the error resilient positional code. Coded video is transmitted using the simulated HIPERLAN/1 air interface standard, which recommends ARQ as a means of overcoming channel errors. This may be unsuitable for real time and broadcast applications. Therefore, a modified HIPERLAN/1 receiver is proposed in this paper, which does not use ARQ to retransmit erroneous packets but instead passes them to the video decoder to exploit error resilience. This strategy provides an acceptable reconstruction quality for average bit error rates equal to 1 in 1000 and is superior to a standard compliant system in the absence of ARQ. This confirms that wireless LAN standards should support a transparent mode for video applications.

This paper investigates the performance of different temporal concealment techniques when incorporated within a long-term motion compensated video codec. In particular, the paper finds a combination of techniques that best recovers the spatial-temporal components of a damaged long-term motion vector. It is demonstrated also that accurate recovery of the spatial components of the damaged motion vector is, in general, more important and more complex than the recovery of the temporal component. Such findings are explained in view of the properties of the long-term block motion field. The paper also, proposes a novel long-term multihypothesis temporal concealment technique where the damaged block is concealed using the average of a number of candidates from the several reference frames available at the decoder. The superior performance of this technique is demonstrated over a range of block error rates.

With the increased role of technology in video production, several types of complex video special effects editing have begun to appear. We consider wiping special effects editing in MPEG-2 compressed video without full frame decompression and motion estimation. We estimated the DCT coefficients and use these coefficients together with the existing motion vectors to produce these special effects editing in the compressed domain. The results show that both the objective and subjective quality of the edited video in the compressed domain closely follows the quality of the video edited in the uncompressed domain at the same bit rate.

To maximise the benefits from data compression, it would be advantageous to develop algorithms that do not require decompression to extract relevant information for post processing. In this paper, a novel technique for extracting variance is proposed for MPEG-2 compressed video using Parseval's theorem. Results show that the estimated variance closely matches with the actual variance. Furthermore, this technique is applied to identify scene changes in the compressed domain.

New rotationally invariant texture feature extraction methods are introduced that utilise the dual-tree complex wavelet transform (DT-CWT). The complex wavelet transform is a new technique that uses a dual tree of wavelet filters to obtain the real and imaginary parts of complex wavelet coefficients. When applied in two dimensions the DT-CWT produces shift invariant orientated subbands. Both isotropic and anisotropic rotationally invariant features can be extracted from the energies of these subbands. Using simple minimum distance classifiers, the classification performance of the proposed feature extraction methods were tested with rotated sample textures. The anisotropic features gave the best classification results for the rotated texture tests, outperforming a similar method using a real wavelet decomposition.

A hierarchical genetic algorithm for disparity estimation is presented. The goal, to estimate reliable disparity fields with low computational cost, is reached using a hierarchical genetic matching procedure. Firstly, each hierarchical image of the stereo pair is divided into sets of feature points and non-feature points. The image morphological gradient for feature points and the disparity Laplacian function for non-feature points are incorporated into the matching function to serve as an adaptive smoothness term. Meanwhile, the vertical-discontinuity constraint and the ordering constraint are also proposed to smooth out vertical disparity discontinuities and to obtain a more reliable disparity estimation. In the hierarchical genetic matching procedure, previously estimated vectors at the former image hierarchy are used to predict the corresponding searching space of chromosomes, and to correct each newly calculated set of disparity vectors. This significantly increases the accuracy of disparity estimation.

A new framework for fusion of 2D images based on their multiscale edges is described in this paper. The new method uses the multiscale edge representation of images proposed by Mallat and Hwang (1992). The input images are fused using their multiscale edges only. Two different algorithms for fusing the point representations and the chain representations of the multiscale edges (wavelet transform modulus maxima) are given. The chain representation has been found to provide numerous new alternatives for image fusion, since edge graph fusion techniques can be employed to combine the images. The new framework encompasses different levels, i.e. pixel and feature levels, of image fusion in the wavelet domain.

This paper investigates the properties of the multiple-reference block motion field. Guided by the results of this investigation, the paper proposes three fast multiple-reference block matching motion estimation algorithms. The proposed algorithms are extensions of the single-reference simplex minimisation search (SMS) algorithm. The algorithms provide different degrees of compromise between prediction quality and computational complexity. Simulation results using a multi-frame memory of 50 frames indicate that the proposed multiple-reference SMS algorithms have a computational complexity comparable to that of single-reference full-search while still maintaining the prediction gain of multiple-reference motion estimation.

With the increase of digital technology in video production, several types of complex video special effects editing have begun to appear in video clips. In this paper we consider fade-out and fade-in special effects editing in MPEG-2 compressed video without full frame decompression and motion estimation. We estimated the DCT coefficients and use these coefficients together with the existing motion vectors to produce these special effects editing in compressed domain. Results show that both objective and subjective quality of the edited video in compressed domain closely follows the quality of the edited video in uncompressed video at the same bit rate.

There is an increased need to extract key information automatically from video for the purposes of indexing, fast retrieval, and scene analysis. To support this vision, reliable scene change detection algorithms must be developed. Several algorithms have been proposed for both sudden and gradual scene change detection in uncompressed and compressed video. In this paper we present an algorithm for fade-in and fade-out scene change detection in both uncompressed and compressed video sequences using histograms. We use the properties of the fading operation and extract these features in the luminance histogram. Results show that the proposed algorithm can be used in both uncompressed and compressed video to detect fade regions with a high reliability and less computations.

Matching pursuits over a basis of separable Gabor functions has been demonstrated to outperform DCT methods for displaced frame difference coding for video compression. Unfortunately, apart from very low bit-rate applications, the algorithm involves an extremely high computational load. This paper contains an original contribution to the issues of dictionary selection and fast implementation for matching pursuits video coding. First, it is shown that the PSNR performance of existing matching pursuits codecs can be improved and the implementation cost reduced by a better selection of dictionary functions. Secondly, dictionary factorization is put forward to further reduce implementation costs. A reduction of the computational load by a factor of 20 is achieved compared to implementations reported to date. For a majority of test conditions, this reduction is supplemented by an improvement in reconstruction quality. Finally, a pruned full-search algorithm is introduced, which offers significant quality gains compared to the better-known heuristic fast-search algorithm, while keeping the computational cost low.

This paper presents a novel algorithm for wipe scene change detection in video sequences. In the proposed scheme, each image in the sequence is mapped to a reduced image. Then we use statistical features and structural properties of the images to identify wipe transition region. Finally, Hough transform is used to analyse the wiping pattern and the direction of wiping. Results show that the algorithm is capable of detecting all wipe regions accurately even when the video sequence contains other special effects.

Automatic identification of special effects is a prerequisite for video indexing and intelligent video encoding. In this paper we present an algorithm for fade and dissolve scene change detection in video sequences. We use statistical features of the images to identify these special effects in uncompressed video. DC-estimation is used to evaluate statistical features both in H.263 and MPEG-2 compressed video. Results show that these special effects can be identified accurately with the proposed scheme.

Content based indexing and retrieval of video is becoming increasingly important in many applications. Identifying scene changes and special effects in a video scene is an essential pre-requisite for automatic indexing. This paper presents a real time algorithm, which can detect abrupt scene changes in the compressed domain. It is based on the number of interpolated macroblocks (MBs) for a given B-frame as the main feature since it expresses a measure of how strong the previous and future I or P (I/P) frames are correlated. Experimental results show that this algorithm can detect most abrupt scene changes in MPEG-2 compressed video.

This paper analyzes various fast 2D-DCT algorithms regarding their suitability for VLIW processors. Operations for truncation or rounding which are usually neglected in proposals for fast algorithms have also been taken into consideration. Loeffler's algorithm with parallel multiplications was found to be most suitable due to its parallel structure.

The importance of texture analysis and classification in image processing is well known. However, many existing texture classification schemes suffer from a number of drawbacks. A large number of features are commonly used to represent each texture and an excessively large image area is often required for the texture analysis, both leading to high computational complexity. Furthermore, most existing schemes are highly orientation dependent and thus cannot correctly classify textures after rotation. In this paper, two novel feature extraction techniques for rotation invariant texture classification are presented. These schemes, using the wavelet transform and Gaussian Markov random field modelling, are shown to give a consistently high performance for rotated textures in the presence of noise. Moreover, they use just four features to represent each texture and require only a 16/spl times/16 image area for their analysis leading to a significantly lower computational complexity than most existing schemes.

The optimal features with which to discriminate between regions and, thus, segment an image often differ depending on the nature of the image. Many real images are made up of both smooth and textured regions and are best segmented using different features in different areas. A scheme that automatically selects the optimal features for each pixel using wavelet analysis is proposed, leading to a robust segmentation algorithm. An automatic method for determining the optimal number of regions for segmentation is also developed.