In cross-modal inference, we estimate complete fields from noisy and missing observations of one sensory modality using structure found in another sensory modality. This inference problem occurs in several areas including texture reconstruction and reconstruction of geophysical fields. We propose a method for cross-modal inference that simultaneously learns shape recipes between two modalities and estimates missing information by using a prior on image structure gleaned from the alternate modality. In the absence of a physical basis for representing image priors, we use a statistical one that represents correlations in differential features. This is done efficiently using a perturbation sampling scheme. Using just one example of the alternate modality, we produce a factorized ensemble representation of feature correlations that yields efficient solutions to large-sized spatial inference problems. We demonstrate the utility of this approach on cross-modal inference with depth and spectral data.

A reactive and concurrent control framework for viewpoint control is developed. The viewpoint control task is decomposed into three control objectives namely; obstacle avoidance, visibility and precision. A moving object is tracked in two panoramic sensors using color, and a scalar uncertainty metric of the object position estimate is introduced. Individual control objectives are accomplished by planning paths using harmonic functions and the task is accomplished using a new subject-to composition operator. It is shown that the system is stable under this composition. The system is demonstrated for tracking a human subject using a fixed panoramic sensor and another panoramic sensor mounted on a mobile platform.

A system to retrieve images using a description of the image intensity surface is presented. Gaussian derivative filters at several scales are applied to the image and low order 2D differential invariants are computed. The resulting multi-scale representation is indexed for rapid retrieval. Queries are designed by the users from an example image by selecting appropriate regions. The invariant vectors corresponding to these regions are matched with the database counterparts both in feature and coordinate space. This yields a match score per image. Images are sorted by the match score and displayed. Experiments conducted with over 1500 images of objects embedded in arbitrary backgrounds are described. It is observed that images similar in appearance and whose viewpoint is within small view variations of the query can be retrieved with an average precision of 50%.

The retrieval of images based on their visual similarity to an example image is an important and fascinating area of research. Here, a method to characterize visual appearance for determining global similarity in images is described. Images are filtered with Gaussian derivatives and geometric features are computed from the filtered images. The geometric features used here are curvature and phase. Two images may be said to be similar if they have similar distributions of such features. Global similarity may, therefore, be deduced by comparing histograms of these features. This allows for rapid retrieval and examples from collection of gray-level and trademark images are shown.

A system to retrieve images using a syntactic description of appearance is presented.A multi-scale invariant wxtor representation is obtained by first filtering images in the database with Gaussian derivative filters at several acalea and then computing low order differential invariants.The multi-scale representation is indexed for rapid retrieval.Queries are designed by the users horn an example image by eelecting appropriate regions.The invariant xnxtors corrmponding to these regions are matched with those in the database both in feature space as well M in coordinate space and a match score is obtained for each image.The results are then displayed to the user sorted by the match score.I?kom experiments conducted with ovsr 1500 images it is shown that imagee similar in appearance and whose viewpoint is within 25 degrees of the query image can be retrieved with an average precision of 57%

A model registration system capable of tracking an object through distinct aspects in real-time is presented. The system integrates tracking, pose determination, and aspect graph indexing. The tracking combines steerable filters with normalized cross-correlation, compensates for rotation in 2D and is adaptive. Robust statistical methods are used in the pose estimation to detect and remove mismatches. The aspect graph is used to determine when features will disappear or become difficult to trade and to predict when and where new features will become trackable. The overall system is stable and is amenable to real-time performance.

A practical real-time system for passive obstacle detection and avoidance is presented. Range information is obtained from stereo images by first computing a disparity picture from the image pair and extracting points above the ground plane. Then these points are projected onto the ground plane and an Instantaneous Obstacle Map (IOM) is obtained. The IOM is transformed into a one dimensional steering vector that represents the hindrance associated with steering in a particular direction and then a one dimensional search is performed on the steering vector for an angle with least hindrance. The steering direction and hindrance value are used to set the speed of the vehicle. This system has been implemented on the Mobile Perception Lab (MPL) at University of Massachusetts at Amherst with considerable success, running at 2 Hz for 256/spl times/240 sized images.>