This paper presents an approach of shape retrieval using linear regression. This approach extracts object edges from decomposed images in different scale levels and approximates them by the linear regression algorithm Matching Pursuit (MP). MP uses linear combination of circular Gaussian basis functions to reconstruct target edges. Each Gaussian basis function contains several features: scale, position and amplitude, which are used to compute distance of two contours. Shapes are retrieved according to the distances between query image and all images in a database.

Artists pictures rarely have photo-realistic detail. Tools to create pictures from digital photographs might, therefore, include methods for removing detail. These tools such as Gaussian and anisotropic diffusion filters and connected-set morphological filters (sieves) remove detail whilst maintaining scale-space causality, in other words new detail is not created using these operators. Non-photorealistic rendering is, therefore, a potential application of these vision techniques. It is shown that certain scale-space filters preserve the appropriate edges of retained segments of interest. The resulting images have fewer extrema and are perceptually simpler than the original. A second artistic goal is to accentuate the centre of attention by reducing detail away from the centre. The process also removes the detail providing perceptual cues about photographic texture. This allows the ‘eye ’ to readily accept alternative, artistic, textures introduced to further create an artistic impression. Moreover, the edges bounding segments accurately represent shapes in the original image and so provide a starting point for sketches. 1

Two dimensional (2D) shape and appearance models are applied to the problem of creating a near-videorealistic talking head. A speech corpus of a talker uttering a set of phonetically balanced training sentences is analysed using a generative model of the human face. Segments of original parameter trajectories, corresponding to the synthesis unit (e.g.~triphone), are extracted from a codebook, then normalised, blended, concatenated and smoothed before being applied to the model to give natural, realistic animations of novel utterances. The system provides a 2D image sequence corresponding to the face of a talker. It is also used to animate the face of a 3D avatar by displacing the mesh according to movements of points in the shape model and dynamically texturing the face polygons using the appearance model.

We present work towards videorealistic synthetic visual speech using non-rigid appearance models. These models are used to track a talking face enunciating a set of training sentences. The resultant parameter trajectories are used in a concatenative synthesis scheme, where samples of original data are extracted from a corpus and concatenated to form new unseen sequences. Here we explore the effect on the synthesiser output of blending several synthesis units considered similar to the desired unit. We present preliminary subjective and objective results used to judge the realism of the system.

In this paper we present initial work towards a video-realistic visual speech synthesiser based on statistical models of shape and appearance. A synthesised image sequence corresponding to an utterance is formed by concatenation of synthesis units (in this case phonemes) from a pre-recorded corpus of training data. A smoothing spline is applied to the concatenated parameters to ensure smooth transitions between frames and the resultant parameters applied to the model—early results look promising.

The multimodal nature of speech is often ignored in human-computer interaction, but lip deformations and other body motion, such as those of the head, convey additional information. We integrate speech cues from many sources and this improves intelligibility, especially when the acoustic signal is degraded. The paper shows how this additional, often complementary, visual speech information can be used for speech recognition. Three methods for parameterizing lip image sequences for recognition using hidden Markov models are compared. Two of these are top-down approaches that fit a model of the inner and outer lip contours and derive lipreading features from a principal component analysis of shape or shape and appearance, respectively. The third, bottom-up, method uses a nonlinear scale-space analysis to form features directly from the pixel intensity. All methods are compared on a multitalker visual speech recognition task of isolated letters.

A new way of representing histograms is presented. The representation is based on a hierarchical scale-space decomposition that forms a scale tree. The tree is a mapping of the original histogram and provides a useful starting point for the visualization and analysis of high-dimensional histograms that are commonplace in image retrieval systems.

Modern neural and adaptive systems often have complicated error performance surfaces with many local extrema. Visualising and understanding these surfaces is critical to effective tuning of these systems but almost all visualisation methods are confined to two dimensions. Here we show how to use a morphological scale-space transform to convert these multi-dimensional complex error surfaces into two-dimensional trees where the leaf nodes are local minima and other nodes represent decision points such as saddle points and points of inflection.

Images are re-mapped as scale-space trees. The minimal data structure is then augmented by "complement nodes" to increase the practical value of the representation. It is then shown how the resulting ctree can be used to remove noise from images, provide a hierarchical way to estimate a dense disparity map from a stereo pair and to provide a basic segmentation of images for image retrieval purposes.

The method known as SSD, sum of squared difference, is a popular and moderately robust method for dense stereo matching, but it has several known limitations. Various refinements have been proposed, including actively changing the window shape during processing and multi-resolution analysis. We use the multi-scale tree representation of the image to generate window shapes that are well matched to image objects. We use the structure of the tree to refine our disparity search, creating a disparity tree. This tree is a close representation of the objects in the scene, and is a step towards creating a symbolic representation of objects within a scene. We find that the new method performs particularly well in regions of low texture: operating conditions that cause conventional methods to fail.

A useful representation of an image would be an object tree in which nodes represent objects, or parts of objects, and which includes at least one node that, together with its children, represents each object: a grandmother node. It is shown that scale-trees, obtained from greyscale images, approximate such a tree. It is then shown how they may be modified using other attributes to more closely become object trees. The result is a data structure that pro-vides “handles ” for every element of the image that can be used for manipu-lating the image. This segmentation has potential for object recognition. 1

In images with low texture the performance of conventional dense stereo can be poor. The usual solution to this is to use a large window but this itself can be problematic as the large window can blur important features and hence lead to errors in the disparity estimate. Here it is shown that, not only do connected set morphology operators overcome this problem, they perform best in regions of low texture. A further observation is that, since the operators give a heirarchical decompostion, there is a possibility of not only using these operators to choose a new window, but also to motivate a new matching method. 1

Systems that attempt to recover the spoken word from image sequences usually require complicated models of the mouth and its motions. Here we describe a new approach based on a fast mathematical morphology transform called the sieve. We form statistics of scale measurements in one and two dimensions and these are used as a feature vector for standard Hidden Markov Models (HMMs).

Sieves decompose one dimensional bounded functions (d/sub m/)/sub m=1//sup R/ that represent the information in a manner that is analogous to the pyramid of wavelets obtained by linear decomposition. Sieves based on sequences of increasing scale open-closings with flat structuring elements (M and N filters) map f to {d} and the recomposition, consisting of adding up all the granule functions, maps {d} to f. Experiments show that a more general property exists such that {d/spl circ/} maps to f/spl circ/ and back to {d/spl circ/} where the granule functions {d/spl circ/} are obtained from {d} by applying any operator /spl alpha/ consisting of changing the amplitudes of some granules, including zero, without changing their signs. In other words, the set of granule function vectors produced by the decomposition is closed under the operation /spl alpha/. An analytical proof of this property is presented. This property means that filters are useful in the context of feature recognition and, in addition, opens the way for an analysis of the noise resistance of sieves.

Decomposition by extrema is put into the context of linear vision systems and scale-space. It is proved that discrete one-dimensional, M- and N-sieves neither introduce new edges as the scale increases nor create new extrema. They share this property with diffusion based filters. They are robust and preserve edges of large scale features.

A cascade of increasing scale, 1-D, recursive median filters produces a sieve, termed an R-sieve, has a number of properties important to image processing. In particular, it (1) Simplifies signals without introducing new extrema or edges, that is, it preserves scale-space. It shares this property with Gaussian filters, but has the advantage of being significantly more robust. (2) The differences between successive stages of the sieve yield a transform, to the granularity domain. Patterns and shapes can be recognized in this domain using idempotent matched sieves and the result transformed back to the spatial domain. The R-sieve is very fast to compute and has a close relationship to 1-D alternating sequential filters with flat structuring elements. They are useful for machine vision applications.

Decomposition by extrema is put into the context of linear vision systems and scale-space. One dimensional discrete M- and N-sieves neither introduce new edges as the scale increases nor create new extrema. They share this property with diffusion based filters. Furthermore M- and N-sieve algorithms are extremely fast with order complexity n. Used to decompose an image, the resulting granularity is appropriate for pattern recognition.>