The paper presents new developments in harmonic analysis associated with the motion transformations embedded in digital signals. In this context, harmonic analysis provides motion analysis with a complete theoretical construction of perfectly matching concepts and a related toolbox leading to fast algorithms. This theory can be built from only two assumptions: an associative structure for the local motion transformations expressed as Lie group and a principle of optimality for the global evolution expressed as a variational extremal. Motion analysis means not only detection, estimation, interpolation, and tracking but also propagators motion-compensated filtering, signal decomposition, and selective reconstruction. The optimality principle defines the trajectory and provides the appropriate equations of motion, the selective tracking equations, the selective constants of motion to be tracked, and all the symmetries to be imposed on the system. The harmonic analysis provides new special functions, orthogonal bases, PDEs, ODEs and integral equations. The tools to be developed rely on group representations, continuous and discrete wavelets, the estimation theory (prediction, smoothing and interpolation) and filtering theory (Kalman filters, motion-based convolutions, integral transforms). All the algorithms are supported by fast and parallelizable implementations based on the FFT and dynamic programming.

This paper presents a novel motion parameter estimation (ME) algorithm based on the spatio-temporal continuous wavelet transform (CWT). The multidimensional nature of the CWT allows for the definition of a multitude of energy densities by integrating over a subset of the CWT parameter space. Three energy densities are used to estimate motion parameters by sequentially optimizing a state vector composed of velocity, position, and size parameters. This optimization is performed on a frame-by-frame basis allowing the algorithm to track moving objects. The ME algorithm is designed to address real world challenges encountered in the defense industry and traffic monitoring scenarios, such as attaining robust performance in noise and handling obscuration and crossing object trajectories.

This paper deals with the estimation of deformational parameters in discrete spatio-temporal signals. The parameters of concern correspond to time-varying scales. As such they can be the coefficients of either a Taylor expansion of the scale or a given deformational transformation. At first sight there are just a few deformational transformations that provide continuous wavelet transforms. The approach presented in this paper associates deformational transformations to motion transformations taking place in higher dimensional spaces and projected on the sensor plane. Then finding continuous wavelet transforms becomes much easier since numerous continuous wavelet transforms have already been defined for motion analysis. It is also known that spatio-temporal continuous wavelet transforms provide minimum-mean-squared-error estimates of motion parameters. Any deformational transformation of features embedded in a spatio-temporal signal may always be related to the projection on the sensor plane of the motion of a rigid object taking place in a higher dimensional space. This reasoning applies conversely. The associated rigid motion may be actual or virtual may take place either on a flat space or on a curved space immersed in higher dimensions. Continuous wavelet transforms for the estimation of deformational parameters may be then deduced from those already existing in motion analysis.

This paper addresses the problem of estimating, analyzing and tracking objects moving with spatio-temporal rotational motion (spin or orbit). It is assumed that the digital signals of interest are acquired from a camera and structured as digital image sequences. The trajectories in the signal are two-dimensional spatial projections in time of motion taking place in a three-dimensional space. The purpose of this work is to focus on the rotational motion, i.e. estimate the angular velocity. In natural scenes, rotational motion usually composes with translational or accelerated motion on a trajectory. This paper shows that trajectory parameters and rotational motion can be efficiently estimated and tracked either simultaneously or separately. The final goal of this work is to provide selective reconstructions of moving objects of interest. This paper constructs new continuous wavelet transforms that can be tuned to both translational and rotational motion. The parameters of analysis that are taken into account in these rotational wavelet transforms are space and time position, velocity, spatial scale, angular orientation and angular velocity. The continuous wavelet functions are finally discretized for signal processing. The link between rotational motion, symmetry and critical sampling is also presented. Applications are presented with tracking and estimation.

This paper addresses the problem of estimating and analyzing accelerated motion in spatio-temporal discrete signals. It is assumed that the digital signals of interest are acquired from imaging sensors and structured as digital image sequences. The motion trajectories in the signal are two-dimensional spatial projections in time of three-dimensional motions. Consequently, they contain all the orders of acceleration. The purpose of this work is to estimate the trajectory and the motion parameters of selected moving objects in the scene. The final goal is to provide selective reconstructions of accelerated objects of interest. This paper presents the construction of new continuous wavelet transforms that can be tuned to any order of accelerations, we demonstrate their existence and provide the related admissibility conditions. The parameters for analysis that are taken into account in these accelerated wavelet transforms are spatial and temporal translations, velocity, acceleration (second or nth order), spatial scale and spatial rotation. The continuous wavelet functions are finally discretized for signal processing.

The purpose of this paper is to develop a motion based segmentation for digital image sequences that is based on the continuous wavelet transform. The continuous wavelet transform allows estimating the motion parameters on all the moving discontinuities, edges and boundaries in the image sequence. This technique provides all the information of motion parameter estimates and edge locations at once without going back and forth refining the segmentation and the motion parameter estimation. Also, this is achieved without involving any point/block corresponding techniques in our algorithm. The edges and the motion parameter estimates are calculated locally on small windows or pixels in the image planes by maximizing the square of the modulus of the wavelet transform. A clustering procedure allows separating all the detected edges into clusters of homogeneous motion. Building a ridge skeleton on the reconstructed edges in each cluster provides the ultimate motion-based segments or partition. The algorithm was simulated using real traffic image sequences acquired by a mobile camera and proved to be accurate and robust.

This paper addresses the problem of estimating, analyzing and tracking objects moving with spatio-temporal rotational motion (i.e. the angular velocity of spinning or orbiting motions). It is assumed that the digital signals of interest are acquired from a camera and structured as digital image sequences. The trajectories in the signal are two-dimensional spatial projections in time of motion taking place in a three-dimensional space. The purpose of this work is to focus on the rotational motion, i.e. estimate the angular velocity and acceleration. In natural scenes, rotational motion usually evolves on a trajectory and then composes with translational or accelerated motion. This paper shows that the trajectory parameters and the rotational parameters can be efficiently estimated and tracked either simultaneously or separately. The final goal of this work is to provide selective reconstructions of moving objects of interest. This paper constructs new continuous wavelet transforms that can be tuned to both translational and rotational motion. The parameters of analysis that are taken into account in these rotational wavelet transforms are space and time position, velocity, spatial scale, angular orientation and angular velocity. The continuous wavelet functions are finally discretized for signal processing. The link between rotational motion, symmetry and critical sampling is also presented. Simulations have been performed with estimation, detection and tracking on natural scenes.

This paper addresses the problem of detecting and tracking moving objects in digital image sequences. The main goal is to detect and select mobile objects in a scene, construct the trajectories, and eventually reconstruct the target objects or their signatures. It is assumed that the image sequences are acquired from imaging sensors. The method is based on spatio-temporal continuous wavelet transforms, discretized for digital signal analysis. It turns out that the wavelet transform can be used efficiently in a Kalman filtering framework to perform detection and tracking. Several families of wavelets are considered for motion analysis according to the specific spatio-temporal transformation. Their construction is based on mechanical parameters describing uniform motion, translation, rotation, acceleration, and deformation. The main idea is that each kind of motion generates a specific signal transformation, which is analyzed by a suitable family of continuous wavelets. The analysis is therefore associated with a set of operators that describe the signal transformations at hand. These operators are then associated with a set of selectivity criteria. This leads to a set of filters that are tuned to the moving objects of interest.

This paper presents a global optimization of the transmission chain for video on asynchronous transfer mode (ATM) channels. This optimum control includes compression, transmission, and decoding. First, the gain in statistical multiplexing is put forward to demonstrate that transmitting at variable rates on asynchronous multiplexing links is more efficient than exploiting the constant rate on synchronous links. The joint optimization relies on both the entropy rate and the end-to-end image quality. This requires the characterization of the video sources as entropy generators, and the development of the entropy rate-distortion functions in the coder and the transmission channel. Quantizers and variable-length entropy coders in coding, traffic, and queues in transmission multiplexing each lead to performance functions expressing quality in terms of entropy rate. The objective measures of quality are, respectively, the PSNR in terms of the output data rate and the cell loss in terms of the network loads. The main advantage of transmitting on variable bit-rate channels is to permit the generation of image sequences at constant subjective quality on the coding side, and the saving of transmission bandwidth through a gain in statistical multiplexing on the network side. Mirrored control actions are described for coding and multiplexing; they lead to a unique global optimum of the transmission chain. Since the decoders are generally slaved to the coding and transmission performances, they are restricted to perform independent optimum signal reconstruction.

This paper deals with new advances made in the field of discrete spatio-temporal filters applied to digital image sequences. Within time-varying images, the temporal correlation of the information is folded within the spatio-temporal domain by motions originating from both camera and object displacements. The spatio-temporal video information can be therefore reformulated in terms of motion trajectories and intensity variations along these trajectories. To yield that signal description, the spatio-temporal scenes will be segmented according to motion. Motion-compensated temporal filters have been used as convolutional filters applied along the assumed motion trajectories. Spectral interpretations show the efficiency of motion-compensated filtering for video signals. As a matter of fact, the whole signal analysis performed in this paper also includes a spatial filtering to achieve a complete spatio-temporal (2-D+T) decomposition. Motion-compensated filtering leads to multiresolution applications. It leads to optimum and adaptive signal-to-noise decomposition procedures based on the temporal correlative content. Such properties allow enhancing tasks like temporal interpolation, image sequence smoothing, and restoration. Simulation results are presented in this paper to illustrate the field of image sequence coding.

The paper presents new approaches in the field of motion-compensated spatio-temporal filters applied to digital image sequences. In time-varying imagery, the temporal correlation information of pixel intensities is folded by motions which may originate from both camera and object displacements. Motion-compensated filters are defined as temporal filters applied along assumed motion trajectories. As a matter of fact, the paper deals with three-dimensional spatio-temporal filters and aims at generalizing the motion-compensated temporal filtering process as the product of two distinct operators. The first operator depends only on the estimated motion parameters derived from both motion-based image segmentations and parametric affine modelings of regions in motion. The second operator analyzes only the correlations of image-by-image intensities measured along the assumed motion trajectories. Multiresolution filters or wavelets may be consequently applied along the motion trajectories to produce optimum and adaptive resulting procedures for purposes like spatio-temporal prediction, interpolation and smoothing. In the paper, applications are provided to cover the field of image sequence coding and interpolation.

This paper describes source models of bit rate for digital TV codecs based on the Discrete Cosine Transformation as decorrelation technique. These models are based on cyclostationary processes. Properties of cyclostationary processes are such that the process is preserved over the whole TV transmission. Therefore, the influence of image Statistics is presented on the coders, the transmission networks, and, the decoders. An emphasis is given on variable bit rate codecs transmitting on ATM networks.