Motion estimation methods can be broadly classified as being spatiotemporal or frequency domain in nature. The Gabor representation is an analysis framework providing localized frequency information. When applied to image sequences, the 3-D Gabor representation displays spatiotemporal/spatiotemporal-frequency (st/stf) information, enabling the application of robust frequency domain methods with adjustable spatiotemporal resolution. In this work, the 3-D Gabor representation is applied to motion analysis. We demonstrate that piecewise uniform translational motion can be estimated by using a uniform translation motion model in the st/stf domain. The resulting motion estimation method exhibits both good spatiotemporal resolution and substantial noise resistance compared to existing spatiotemporal methods. To form the basis of this model, we derive the signature of the translational motion in the 3-D Gabor domain. Finally, to obtain higher spatiotemporal resolution for more complex motions, a dense motion field estimation method is developed to find a motion estimate for every pixel in the sequence.

In this paper, detection and estimation of rotational motion in the 3-D Gabor domain are investigated. By extending frequency domain methods to the joint spatiotemporal/spatio-temporal-frequency (st/stf) analysis framework, we propose a technique to estimate multiple localized rotational motions in image sequences. Based on a rotational motion model derived in the 3-D Gabor domain, an algorithm for estimating localized uniform rotational motions is constructed and discussed.

This paper presents two advancements in motion estimation based on the 3-D Gabor representation. First, we introduce a new searching strategy and solve the estimation bias problem encountered in the current approach to piecewise translational motion estimation; second, in order to achieve better spatiotemporal resolution, we constructed a new algorithm that can provide dense motion field estimation.

Compression is essential for the efficient storage, transmission, and processing of volumetric image data. While volumetric image compression has not been pursued as actively as still image and video compression, a variety of approaches have been proposed. Many of these approaches have been inspired by existing image or video compression techniques. Local 2-D and 3-D decompositions (most notably local scale (wavelet) transforms) have proven to be particularly useful. We examine the state-of-the-art in this field, and consider a 3-D extension of a previously developed local frequency decomposition (the derivative of Gaussian transform) for the representation and coding of volumetric data.

The derivative of Gaussian transform is a spatial/spatial frequency representation of an image that has been used in image and video compression algorithms. Both the forward and inverse transforms require O(N/sup 3/) operations. We consider truncating the length N basis functions to produce a sparse basis matrix and thus to reduce the computational load of the reconstruction. We examine the effects of truncation on the basis function energy and evaluate the change in a number of basis quality measures as the basis function length is decreased.

We investigate the improvements that can be obtained in several conventional video-processing algorithms through the incorporation of three-dimensional (3-D) (spatiotemporal) segmentation information. Four classes of image sequence processing techniques are considered: low-pass filtering, high-pass filtering, high-frequency emphasis, and 3-D Sobel filtering. It is demonstrated that segmentation information can improve the performance of these techniques substantially so that this approach may be promising for other applications (e.g., deinterlacing and resolution conversion) as well.

We show how to extract plain text from PostScript files. A textual scan is inadequate because PostScript interpreters can generate characters on the page that do not appear in the source file. Furthermore, word and line breaks are implicit in the graphical rendition, and must be inferred from the positioning of word fragments. We present a robust technique for extracting text and recognizing words and paragraphs. The method uses a standard PostScript interpreter but redefines several PostScript operators, and simple heuristics are employed to locate word and line breaks. The scheme has been used to create a full-text index, and plain-text versions, of 40,000 technical reports (34 Gbytes of PostScript). Other text-extraction systems are reviewed: none offer the same combination of robustness and simplicity. © 1998 John Wiley & Sons, Ltd.

There are many benefits to be gained in image processing and compression by the use of analyzing functions which are local in both space and spatial frequency. It is often assumed that these benefits are in some way proportional to the degree of joint locality of the functions being used. Within the limits imposed by the uncertainty principle, there can be great variation in this joint locality across different local function families. While there is no generally accepted joint locality metric appropriate for visual applications, Gabor's joint uncertainty is often cited to justify the use of a set of functions. It has been shown that complex Gabor functions optimize this metric. There is some debate however regarding which, of the restricted class of real functions, has the lowest joint uncertainty. In this paper we examine three families of real functions and directly evaluate the Gabor metric for joint uncertainty. In contrast to previous attempts to prove the optimality of any one function, this analysis provides an explicit numerical basis for comparison of these real functions.

It has been shown that motion can be characterized in the spatiotemporal-frequency (Fourier) domain, and that this phenomenon can be exploited for the computation of optical flow. However in practical (natural) sequences, the signatures of the different motions cannot be resolved in the Fourier domain, nor associated with their respective regions in the image sequence. In this paper we consider the application of a spatiotemporal/spatiotemporal frequency representation (a 3-D version of the widely used Gabor transform) to the computation of optical flow in natural scenes.

This work presents a novel bond percolation-based approach to determine the clique potential parameters of a Gibbs-Markov model as a function of local characteristics of the underlying image. Using the renormalization group transformation, a multiscale description of the clique potential parameters is formed and used to obtain multiresolution image segmentation.

The article describes a new image transform that decomposes an image using a set of Gaussian derivatives. The basis functions themselves have been shown to effectively model the measured receptive fields of simple cells in the mammalian visual cortex. Based on these functions, it can be expected that this transform can provide a mechanism for exploiting the properties of the human visual system in image processing algorithms.

A new bond percolation-based approach is presented to determine the clique potential parameters of a Gibbs-Markov random field (GMRF) model used in image segmentation. Previously, experimentally determined fixed values were used for these parameters independent of the underlying image. Using the proposed approach, these parameters are now derived as a function of local characteristics of the image under consideration. An additional salient feature of this method is its suitability for a renormalization group approach to multi-scale description of the clique potential parameters.>

One of the most commonly used image enhancement techniques is noise removal. The goal of a noise removal method is to remove noise while preserving the image structure. In this paper a segmentation-based image enhancement method is presented which confines image smoothing to the region interiors. Since the structure in the image is captured by segmentation, confining image smoothing to the region interiors does not cause any significant loss of this structure. In fact, region confined averaging enhances the contrast between boundaries, and hence sharpens the image at these boundaries. The result is an image with reduced noise, but that appears sharper than the original.>

A method for rendering lightning using conventional raytracing techniques is discussed. The approach taken is directed at producing aesthetic images for animation, rather than providing a realistic physically based model for rendering. A particle system is used to generate the path of the lightning channel, and subsequently to animate the lightning. A technique, using implicit surfaces, is introduced for illuminating objects struck by lightning.

The authors describe a human visual perception approach and report some results for this problem. The approach is based on the differential quantization of images, in which smooth approximations are subtracted from the image prior to quantization. They consider two such approximations. The first one is an approximation by splines obtained from a sparse and fixed subsampled array of the image. The second one segments the image into piecewise constant regions on the basis of the local activity of the image. Both these approximations result in remainders of residual images where large errors are localized in portions of the image of high activity. Because of visual masking the remainder image can now be coarsely quantized without visual impairment to the reconstructed image. The coarsely quantized remainder is now encoded in an error free manner. In such a perceptually based encoding method the mean square error is now dependent on the activity of the image.>

A new segmentation algorithm for still black and white images is introduced. This algorithm forms the basis of a region-oriented sequence coding technique, currently under development. The algorithm models the human mechanism of selecting regions both by their interior characteristics and their boundaries. This is carried out in two different stages: with a preprocessing that takes into account only gray level information, and with a stochastic model for segmented images that uses both region interior and boundary information. In the stochastic model, the gray level information within the regions is modeled by stationary Gaussian processes, and the boundary information by a Gibbs-Markov random field (GMRF). The segmentation is carried out by finding the most likely realization of the joint process (maximum a posteriori criterion), given the preprocessed image. For decreasing the computational load while avoiding local maxima in the probability function, suboptimal versions of the algorithm are proposed.>

A method is described for reducing the information contained in an image sequence, while retaining the information necessary for the interpretation of the sequence by a human observer. The method consists of first locating the redundant information, reducing the degree of redundancy, and coding the result. The sequence is treated as a single 3D data volume, the voxels of which are grouped into several regions, obtained by a 3D split and merge algorithm. The information is coded by representing the borders of the regions using a pyramidal structure in the x, y, t space. The coefficients of the approximating polynomials are coded in a straightforward manner. For 256*256 pixel, 25 frame/s image sequences, compressions allowing transmission rates near 64 kbit/s are obtained.>

An orthogonal transform using polar separable filters is introduced. The conditions for perfect reconstruction properties to hold will be discussed. A method for constructing orthogonal pairs of directional operators is derived. An example of a two band orthogonal perfect reconstruction fan filter pair is presented. An orthogonal directional four band filter bank is shown

A compression technique based on an expansion is presented. The elementary functions of the expansion form a class of pyramidal Gabor functions covering the frequency domain in octave bands. The image sequence is coded by differentially coding selected coefficients of this expansion. Simulation results show sequences reconstructed with good quality for a bit rate less than 64 Kbit/s.

The generic issue of clustering/grouping is addressed. Recent research, both in computer and human vision, suggests the use of joint spatial/spatial-frequency (s/sf) representations. The spectrogram, the difference of Gaussians representation, the Gabor representation, and the Wigner distribution are discussed and compared. It is noted that the Wigner distribution gives superior joint resolution. Experimental results in the area of texture segmentation and Gestalt grouping using the Wigner distribution are presented, proving the feasibility of using s/sf representations for low-level (early, preattentive) vision.>