In this paper, we propose a new method for estimating sub-pixel motion via exploiting the principle of phase correlation in the Fourier domain. The method is based on linear weighting of the height of the main peak on the one hand and the difference between its two neighboring side-peaks on the other. Using both synthetic and real data we show that the proposed method outperforms many established approaches and achieves improved accuracy even in the presence of noisy samples.

Signal estimation for sparsely and irregularly sampled signals can be carried out using either noniterative methods, or iterative methods or methods that deal with irregular samples and their uncertainty, through normalized convolution. The latter is a general method for filtering incomplete or uncertain data and is based on the separation of both data and operator into a signal part and a certainty part. It has been proven that normalized convolution yields a local description which is optimal both in an algebraic and a least-squares sense. In this letter, we employ the normalized convolution concept to formulate a novel reconstruction method for irregularly sampled signals, utilizing an anisotropic, rotated applicability filter. Our experimental results demonstrate performance gains in a least-squares sense, retaining edge and contour information, especially in sparsely sampled areas on the image plane.

We present a novel approach for the compensation of temporal brightness variations (commonly referred to as flicker) in archived film sequences. The proposed method is motivated by fundamental principles of photographic image registration and provides a substantial level of adaptation to temporal but also spatial variations of picture brightness. Additionally, our scheme provides an efficient mechanism for the adaptive estimation of flicker compensation profile, which makes it suitable for the compensation of long duration film sequences while it addresses problems arising from scene motion and illumination using a novel motion-compensated graylevel tracing approach. We present experimental evidence which suggests that our method offers high levels of performance and compares favorably with competing state-of-the-art techniques for flicker compensation.

This paper proposes a novel 3D lossy compression algorithm tailored for 3D faces. We analyse the effects of compression on the face verification rate and measure recognition performances on the face recognition grand challenge database. Whilst preserving the spatial resolution enabling reconstruction of surface details, the proposed scheme achieves substantial compression to the extent that personal 3D biometric data could fit on a 2D barcode.

A novel spatio-temporal method is proposed for film dirt detection and recovery. Firstly, a more reliable confidence measurement of dirt is extracted for color films. False alarms caused by motion are filtered using consistency checks among several measurements. Then, candidate dirt is detected by filtering and thresholding this confidence measurement. Finally, bi-directional local motion compensation and ML3Dex filtering are taken for the recovery of dirt pixels. Experiments on real data demonstrate the efficiency and effectiveness of our method in terms of both detection and recovery of dirt.

A novel extension of phase correlation to subspace correlation is proposed, in which 2-D translation is decomposed into two 1-D motions thus only 1-D Fourier transform is used to estimate the corresponding motion. In each subspace, the first two highest peaks from 1-D correlation are linearly interpolated for subpixel accuracy. Experimental results have shown both the robustness and accuracy of our method.

We propose a new motion estimation scheme particularly suitable for broadcast-quality digital video applications due to its performance-complexity characteristics. The proposed scheme is based on the principle of gradient correlation and its computational efficiency is due to the fact that it operates in the frequency domain. The scheme involves the quad-tree decomposition of a frame thus providing a better level of adaptation to scene contents compared to fixed block size approaches. Quad-tree decompositions are obtained by using the motion compensated prediction error to control the partition of a parent block to four children quadrants. The partition criterion is applied iteratively until a target number of motion vectors or a target level of motion compensated prediction error is achieved or, ultimately, until no more than a single motion component can be identified. The partition criterion also guarantees a monotonic decrease of the motion compensated prediction error with an increasing number of iterations making our scheme suitable for progressive transmission and embedded coding applications. Our results show that our scheme outperforms fixed block size phase correlation as well as quad-tree motion estimation based on phase correlation in terms of rate-distortion characteristics yielding smoother motion vector fields as well as more compressible motion-compensated prediction residuals.

In this correspondence, a novel segmentation-assisted method for film-dirt detection is proposed. We exploit the fact that film dirt manifests in the spatial domain as a cluster of connected pixels whose intensity differs substantially from that of its neighborhood, and we employ a segmentation-based approach to identify this type of structure. A key feature of our approach is the computation of a measure of confidence attached to detected dirt regions, which can be utilized for performance fine tuning. Another important feature of our algorithm is the avoidance of the computational complexity associated with motion estimation. Our experimental framework benefits from the availability of manually derived as well as objective ground-truth data obtained using infrared scanning. Our results demonstrate that the proposed method compares favorably with standard spatial, temporal, and multistage median-filtering approaches and provides efficient and robust detection for a wide variety of test materials.

We propose a method for obtaining high-accuracy sub-pixel motion estimates using phase correlation. Our method is motivated by recently published analysis according to which the Fourier inverse of the normalized cross-power spectrum of pairs of images which have been mutually shifted by a fractional amount can be approximated by a two-dimensional sinc function. We use a modified version of such a function to obtain a sub-pixel estimate of motion by means of variable-separable fitting in the vicinity of the maximum peak of the phase correlation surface. We demonstrate that our method outperforms, in terms of sub-pixel accuracy, not only other surface fitting techniques but also the state-of-the-art in motion estimation using phase correlation including the technique that motivated our work in the first place. Furthermore our method performs particularly well in the presence of artificially induced additive white Gaussian noise and also offers better motion vector coherence in terms of zero-order entropy.

We present a novel approach for the compensation of temporal brightness variations (commonly referred to as flicker) in archived film sequences. The proposed method is motivated by fundamental principles of photographic image registration and provides a substantial level of adaptation to temporal but also spatial variations of picture brightness. Additionally our scheme provides an efficient mechanism for the appointment and dynamic update of reference frames which makes it suitable for the compensation of long duration film sequences while it addresses problems arising from scene motion using a novel motion-compensated greylevel tracing approach. We present experimental evidence which suggests that our method offers high levels of performance and compares favourably with competing state-of-the-art techniques for flicker compensation

To represent object movement, the conventional approach is to embed the motion information into parametric and/or statistical motion models. However, the inherent complexity of this description is ill suited for application requiring rapid and automatic, albeit significant, responses. Surveillance applications belong to this class and are emerging as the most active field of research in computer vision. For this purpose, we observe that the differential invariants obtained by dense optic flow are capable of accurately describing complex object motion without requiring setting up and initializing models. In this letter, we demonstrate the novel use of such motion descriptors for spatio-temporal object segmentation and delayering of sequences. Our results show the ability of this approach to describe simply and accurately differently moving objects and to be incorporated in segmentation processes that deliver a hierarchical description of object, producing evident improvements to the segmented objects while being computationally efficient

A novel segmentation-assisted method for film dirt detection is proposed. Since dirt manifests as a cluster of pixels whose intensity differs from that of its neighbourhood, we employ segmentation and assume that each small region as a dirt candidate. The assumption is validated by considering raw (non-motion compensated) differences between the current frame and each of the previous and next frames which provides a measure of a confidence. Our experiments show that our method compares favourably with standard spatial, temporal and multistage median filtering approaches and provides efficient and robust detection even for fast moving sequences. 1

We propose a quad-tree scheme for obtaining subpixel estimates of interframe motion in the frequency domain. Our scheme is based on phase correlation and uses key features of the phase correlation surface to control the partition of a parent block to four children quadrants. The partition criterion is applied iteratively until a target number of motion vectors or a target level of motion compensated prediction error is achieved or, ultimately, if no more than a single motion component per block can be identified. Our results show that our scheme provides a better level of adaptation to scene contents and outperforms fixed block size phase correlation in terms of total motion compensated prediction error for the same number of motion vectors and also in terms of number of motion vectors for the same level of motion compensated prediction error.

A new approach to variable size block matching is proposed, based on the binary partitioning of blocks. If a particular block does not allow for accurate motion compensation, then it is split into two using the horizontal or vertical line that achieves the maximum reduction in motion compensation error. This method causes partitioning to occur along motion boundaries, thus substantially reducing blocking artifacts. In addition, small blocks are placed in regions of complex motion, while large blocks cover regions of uniform motion. The proposed technique provides significant gains in picture quality of 1.5 to 3.0 dB, when compared to fixed size block matching at the same total rate.

We propose a quad-tree scheme for obtaining sub-pixel estimates of interframe motion in the frequency domain. Our scheme is based on phase correlation and uses motion compensated prediction error to control the partition of a parent block to four children quadrants. This criterion guarantees a monotonic decrease of the motion compensated prediction error with an increasing number of iterations making our scheme suitable for embedded coding applications. Our results show that our scheme provides a better level of adaptation to scene contents and outperforms fixed block size phase correlation in terms of total motion compensated prediction error for the same number of motion vectors and also in terms of number of motion vectors for the same level of motion compensated prediction error

A user-oriented multimodal interface (MMI) framework is proposed. Considering the complexities of media connotations and uncertainties of the user’s demands, content-based retrieval has intrinsic requirements for MMI for effective media-content interactions. Through integration of knowledge based conduction, learning of semantic concepts, natural language processing and analysis of users’ profiles, our framework can establish a solid basis for design and implementation of general CBR systems satisfying extensibility, condensability and inter-operability.

A highly accurate and computationally efficient method is presented suitable for the estimation of motion in video sequences. The method is based on the maximization of the spatial gradient cross-correlation function, which is computed in the frequency domain and therefore can be implemented by fast transformation algorithms. We present enhancements to the baseline gradient-correlation algorithm, which further improve performance, especially in the presence of manually induced additive Gaussian noise. We also present a comparative performance analysis, which demonstrates that the proposed method outperforms the state-of-the-art in frequency-domain motion estimation, in the shape of phase correlation.

A new method for the lossy compression of segmentation maps is presented. In the first stage of the proposed algorithm, region boundary segments are coded, providing a coarse polygonal approximation of regions, as well as a complete representation of their adjacency. The second stage consists of the progressive refinement of these segments, until either the desired degree of accuracy is reached, or the target bit-rate is achieved. Rate-distortion curves demonstrate good lossy performance in the range of 0.3 to 0.6 bits per contour point.

A new model for the correction of flicker in archived film sequences is formulated from first principles. The model takes into account the Hurter-Driffield density versus log exposure characteristic to estimate image intensity errors caused by exposure inconsistencies. The model is shown to be highly nonlinear and can accommodate typical grayscale manipulations performed during film-to-video transfers. Motivated by the above model, an algorithm for the estimation of a correction profile across the available grayscale is developed. This profile is obtained from histogrammed gray-level differences between a reference and a degraded frame and is robust with respect to intensity changes due to scene and camera motion. Our results show that the proposed algorithm is very effective toward removing flicker for a variety of archived sequences and compares favorably with other state-of-the-art techniques developed for the same purpose. Additionally, the parametric form of the profile makes it particularly suitable for metadata use and allows fast inversion of the restoration process and recovery of the original material.

In this paper a novel two-stage architecture for object-based segmentation of moving sequences is proposed using multiple features such as motion, intensity and texture. The first stage locates perceptually meaningful objects using a hierarchy of single-feature segmentation processes. The second stage refines the boundaries of located objects using a combination of features according to a set of appropriate rules. Experimental results show that the proposed approach yields intuitively correct as well as accurate segmentations of moving sequences, which compare favourably with established state-of-the art techniques in the literature. 1

A novel algorithm for the detection of cuts in video sequences is proposed. The algorithm uses phase correlation to obtain a measure of content similarity for temporally adjacent frames and responds very well to scene cuts. The algorithm is insensitive to the presence of global illumination changes and noise and outperforms established methods for cut detection. As the proposed scheme is implemented in the frequency domain, the availability of fast hardware makes the scheme attractive for interactive and on-line applications.

Twin-lens telecine flicker is considered as a special type of artefact in film-to-video conversions. It is introduced by telecine equipment using separate optic light paths to register each of the two fields of a line-interlaced video frame. Motion estimation and compensation algorithms suitable for the elimination of this type of artefact are developed. Their application achieves considerable improvement in picture quality both subjectively and in terms of measured mean-square error.

A numerical study of the performance of hybrid direct-sequence slow-frequency-hopped spread-spectrum random-access schemes using Reed-Solomon forward-error control coding is performed. The performance measures of interest are the maximum numbers of simultaneous transmitters that can be tolerated in the vicinity of a receiver at a desirable level of the bit (or symbol) error probability, the normalized throughput, and the packet error probability. For the case where all simultaneous signals have the same received power levels, a critical comparison of the performance of direct-sequence, frequency-hopped, and hybrid spread-spectrum systems with the same overall bandwidth expansion is conducted with respect to the above performance measures, and ranges of parameters, for which each system outperforms the others, are identified. It is also established that hybrid spread-spectrum random-access schemes have considerably higher throughput than the uncoded nonspread-spectrum ones, for the desired range of values of the packet error probability.>