Many image deblurring algorithms perform blur kernel estimation and image deblurring by assuming the blur type and distribution are already known. However, in practice such information is not known in advance and must be estimated using local blur measures. In this paper, we revisit the image partial blur detection and classification problem and propose several new or enhanced local blur measures using different types of image information including color, gradient and spectral information. The proposed measures demonstrate stronger discriminative power, better across-image stability or higher computational efficiency than previous ones. By learning the optimal combination of these measures with SVM classifiers, we obtain a patch-based image partial blur detector and classifier. Experiments on a large dataset of real images show the proposed approach has superior performance to the state-of-the-art approach.

Many different automatic color correction approaches have been proposed by different research communities in the past decade. However, these approaches are seldom compared, so their relative performance and applicability are unclear. For multi-view image and video stitching applications, an ideal color correction approach should be effective at transferring the color palette of the source image to the target image, and meanwhile be able to extend the transferred color from the overlapped area to the full target image without creating visual artifacts. In this paper we evaluate the performance of color correction approaches for automatic multi-view image and video stitching. We consider nine color correction algorithms from the literature applied to 40 synthetic image pairs and 30 real mosaic image pairs selected from different applications. Experimental results show that both parametric and non-parametric approaches have members that are effective at transferring colors, while parametric approaches are generally better than non-parametric approaches in extendability.

Navigating through unknown outdoor environments requires a robot to be able to see and model the far field terrain. In recent years this problem of seeing beyond reliable stereo readings into the far field has gained attention. Many proposed solutions involve using near field obstacle and ground plane regions labeled using stereo, to learn models which classify far field image regions. In this work we offer an alternative which exploits coherent image regions as determined by image segmentation to propagate obstacle and ground labels from the near and mid field to the image far field. Rather than relying on local features to classify individual pixels we model and compare appearance across the whole segment. New labels are determined by proximity in both image space and appearance space. Since both traversable and non-traversable surfaces can vary in appearance across the image, our approach has the advantage that each labeled segment acts as a distinct appearance model, which allows us to label similar neighbours. We evaluate our system using a publicly available dataset and compare its performance to a typical learning-based near-to-far labeling scheme.

Autonomous robot navigation in unstructured outdoor environments is a challenging and largely unsolved area of active research. The navigation task requires identifying safe, traversable paths that allow the robot to progress towards a goal while avoiding obstacles. Machine learning techniques are well adapted to this task, accomplishing near-to-far learning by training appearance-based models using near-field stereo readings in order to predict safe terrain and obstacles in the far field. However, these methods are subject to degraded performance when training data sets exhibit class imbalance, or skew, where data instances of one class outnumber those in another. In such scenarios, classifiers can be overwhelmed by the majority class, and will tend to ignore the minority class. In this paper, we show that typical outdoor terrain scenarios are associated with training data imbalance, and examine the impact of using undersampling, oversampling, SMOTE, and biased penalties techniques to correct for imbalance in stereo-derived training data. We conduct a statistically significant, repeated measures empirical evaluation and demonstrate improved far-field terrain prediction performance when using such methods for handling class imbalance versus taking no corrective action at all.

Many terrains in outdoor robot navigation problems have paths that are distinct and continuous compared to the non-traversable regions. In image space these paths correspond to continuous segments that can be thought of as clusters embedded in image feature space. These segments very often translate directly to traversable ground plane. In this paper we build the intuition for semi-supervised methods in path identification and present a Markov random walk based approach that requires very few labeled points. The method creates a nearest neighbor graph representation of the current image frame using features deemed suitable for the task and propagates labels based on the concept of absorbing Markov chains. We extend this formalism to the task of dynamically identifying traversable and non-traversable regions in the incoming image frames. We present results on actual terrains corresponding to test courses used by the LAGR test team. The results demonstrate that with minimal initial supervision the robot can navigate to the goal. We also conduct comparisons of our path labeling technique against other machine learning techniques including nonlinear support vector machines on hand labeled data. The results demonstrate that our semi-supervised approach is proficient in the domain of path traversal in unstructured domains.

Segmentation, or partitioning images into internally homogeneous regions, is an important first step in many Computer Vision tasks. In this paper, we attack the segmentation problem using an ensemble of low cost image segmentations. These segmentations are reconciled by applying recent techniques from the consensus clustering literature which exploit a Non-negative Matrix Factorization (NMF) framework. We describe extensions to these methods that scale them for large images and also incorporate smoothness constraints. This framework allows us to uniformly and easily combine segmentations from different algorithms or feature modalities. We then demonstrate that popular bottom up image segmentation algorithms, Mean Shift and Efficient Graph Based segmentation, perform no better than our simple combination of multiple image segmentations derived from k-means clustering (of various feature spaces) or from “naive” RGB quantizations. The algorithms are evaluated on the Berkeley image segmentation dataset.

In vision-based autonomous spacecraft docking multiple views of scene structure captured with the same camera and scene geometry are available under different lighting conditions. These ¿multiple exposure¿ images must be processed to localize visual features to compute the pose of the target object. This paper describes a novel multi-channel edge detection algorithm that localizes the structure of the target object by merging the gradient information of these multiple exposure images using tensor voting. This approach reduces the effect of illumination variation, including the effect of shadow edges, over the use of a single image and over simple combinations of single-channel edge maps. Compared to a recently proposed multi-channel edge detection approach using GMM modeling, the proposed approach can generate edge maps of comparable quality in much less time.

The cheirality constraint, which requires that reconstructed point correspondences lie in front of the cameras, has not typically been integrated into traditional RANSAC-based pose estimators. We have developed a new RANSAC-based relative pose estimator which incorporates the cheirality constraint not only to preempt invalid epipolar geometry hypotheses, but also as a criterion in identifying inliers for image feature correspondences. Because the application of the cheirality constraint is tightly related to the estimation of epipolar geometry, integrating them inside RANSAC can prevent inliers being falsely identified as cheirality outliers. The result is a more consistent and stable estimation which leaves denser feature correspondences for subsequent processing. Experimental comparison between the usual RANSAC-based approach and the proposed approach is performed.

Autonomous robot navigation in unstructured outdoor environments is a challenging area of active research. The navigation task requires identifying safe, traversable paths which allow the robot to progress toward a goal while avoiding obstacles. Machine learning techniques - in particular, classifier ensembles - are well adapted to this task, accomplishing near-to-far learning by augmenting near-field stereo readings in order to identify safe terrain and obstacles in the far field. Composition of the ensemble and subsequent combination of model outputs in this dynamic problem domain remain open questions. Recently, Ensemble selection has been proposed as a mechanism for selecting and combining models from an existing model library and shown to perform well in static domains. We propose the adaptation of this technique to the time-evolving data associated with the outdoor robot navigation domain. Important research questions as to the composition of the model library, as well as how to combine selected modelspsila outputs, are addressed in a two-factor experimental evaluation. We evaluate the performance of our technique on six fully labeled datasets, and show that our technique outperforms memoryless baseline techniques that do not leverage past experience.

An approach to stereo based local path planning in unstructured environments is presented. The approach differs from previous stereo based and image based planning systems (e.g. top-down occupancy grid planners, autonomous highway driving algorithms, and view-sequenced route representation), in that it uses specialized cost functions to find paths through an occupancy grid representation of the world directly in the image plane and forgoes a projection of cost information from the image plane down onto a top-down 2D Cartesian cost map. We discuss three cost metrics for path selection in image space. We present a basic image based planning system, discuss its susceptibility to rotational and translational oscillation, and present and implement two extensions to the basic system that overcome these limitations - a cylindrical based image system and a hierarchical planning system. All three systems are implemented in an autonomous robot and are tested against a standard top-down 2D Cartesian planning system on three outdoor courses of varying difficulty. We find that the basic image based planning system fails under certain conditions; however, the cylindrical based system is well suited to the task of local path planning and for use as a high resolution local planning component of a hierarchical planning system.

Autonomous robot navigation in unstructured outdoor environments is a challenging area of active research. The navigation task requires identifying safe, traversable paths which allow the robot to progress toward a goal while avoiding obstacles. One approach is to apply Machine Learning techniques that accomplish near to far learning by augmenting near-field Stereo to identify safe terrain and obstacles in the far field. Some mechanism for applying past learned experience to the active navigation task is crucial for effective far-field classification. We introduce a new method for long-term learning in the robot navigation task by selecting a subset of previously learned linear binary classifiers. We then combine their output to produce a final classification for a new image. Techniques for efficient selection of models, as well as the combination of their output, are addressed. We evaluate the performance of our technique on three fully labeled datasets, and show that our technique outperforms several baseline techniques that do not leverage past experience.

The processing power and network bandwidth required for true immersive telepresence applications are only now beginning to be available. We draw from our experience developing stereo based tele-immersion prototypes to present the main issues arising when building these systems. Tele-immersion is a new medium that enables a user to share a virtual space with remote participants. The user is immersed in a rendered three-dimensional (3-D) world that is transmitted from a remote site. To acquire this 3-D description, we apply binocular and trinocular stereo techniques which provide a view-independent scene description. Slow processing cycles or long network latencies interfere with the users' ability to communicate, so the dense stereo range data must be computed and transmitted at high frame rates. Moreover, reconstructed 3-D views of the remote scene must be as accurate as possible to achieve a sense of presence. We address both issues of speed and accuracy using a variety of techniques including the power of supercomputing clusters and a method for combining motion and stereo in order to increase speed and robustness. We present the latest prototype acquiring a room-size environment in real time using a supercomputing cluster, and we discuss its strengths and current weaknesses.

In an immersive tele-presence environment a 3D remote real scene is projected from the viewpoint of the local user. This 3D world is acquired through stereo reconstruction at the remote site. In this paper we start a performance analysis of stereo algorithms with respect to the task of immersive visualization. As opposed to usual monocular image based rendering, we are also interested in the depth error in novel views because our rendering is stereoscopic. We describe an evaluation test-bed which provides a world-wide first available set of registered dense "ground-truth" laser data and image data from multiple views. We establish metrics for novel depth views that reflect discrepancies both in the image and in 3D-space. It is well known that stereo performance is affected by both erroneous matching as well as incorrect depth triangulation. We experimentally study the effects of occlusion and low texture on the distributions of the error metrics. Then, we algebraically predict the behavior of depth and novel projection error as a function of the camera set-up and the error in the disparity. These are first steps towards building a laboratory for psychophysical judgement of depth estimates which is the ultimate performance test of tele-presence stereo.

Tele-immersion is a technology that augments your space with real-time 3D projections of remote spaces thus facilitating the interaction of people from different places in virtually the same environment. Tele-immersion combines 3D scene recovery from computer vision, and rendering and interaction from computer graphics. We describe the real-time 3D scene acquisition using a new algorithm for trinocular stereo. We extend this method in time by combining motion and stereo in order to increase speed and robustness.

The constraint of a third camera in stereo vision is a useful tool for reducing ambiguity in matching. Most of the systems using trinocular stereo to date however, have used configurations where the image planes of all three cameras are coplanar, or can be rectified to be so. We explore the computation of dense trinocular disparity maps for non-planar camera configurations which arise when cameras surround the object to be modeled. Our approach rectifies the cameras as two independent stereo pairs. We start with an exhaustive lookup scheme and then consider retaining only a list of N disparities per pixel with maximal correlation values for the right and left pairs. Experimental results and comparisons demonstrate that both methods reduce outliers over binocular stereo, and that the N-hypothesis system trades large lookup tables for somewhat lower density of valid matches.

The problem of robot visual navigation is bound up with the problem of selecting and extracting salient features or landmarks which can guide the robot safely through its environment. In addition for vision based manipulation tasks for mobile robots we must register the position and orientation of objects to be manipulated. In general identifying the ground plane and distinguishing non-planar obstacles is important for indoor navigation. The large scale industrial robots we study are used in such tasks as forest management or hazardous waste clean up. For these machines identification and registration of vertical cylinders representing trees to be avoided or 45 gal. drums to be manipulated is critical. Instead of laborious reconstruction of surfaces to find these critical plane and cylinder shape primitives we propose that the subspace defined by the disparity gradients of visible surfaces can be used to extract them efficiently. We demonstrate the use of these features, which are easily extracted in real time using a high speed pipeline processor.

How can we describe and measure real robotic systems? Implemented robotic systems for intelligent tasks are are often extremely complicated, depending not only on 'interesting' theoretical parameters and models, but on many assumptions and constants which may be set almost arbitrarily. In order to evaluate and compare systems which employ diverse components and methods, we are obliged to employ empirical, performance based methods. We model task implementations as particular parameterizations of the task goals they represent. Through fractional factorial experiments and analysis of variance we establish the statistically significant parameters and parameter interactions for a pair of part-orienting tasks. Sensitivity analysis determines optimal parameter values and reveals relationships among parameters as they affect task performance.

Rigorous analysis and evaluation of real implemented robotic systems for intelligent tasks are rarely performed. Such systems are often extremely complicated, depending not only on 'interesting' theoretical parameters and models, but on many assumptions and constants which may be set almost arbitrarily. We view all task implementations as particular parameterizations of the task goals they represent. Through fractional factorial experiments we establish the statistically significant parameters and parameter interactions for a 'sensorless' model-based push-orienting task. This type of analysis is a necessary step to understanding integrated intelligent systems. It reveals aspects of system implementations which cannot easily be predicted in advance, and gives a clear picture of the task requirements, given the strengths and weaknesses of the observed system.

To make sensors in general, and vision in particular, work for robotic systems we must find ways to extract timely, relevant, accurate information. Building such sensors will require employing any special devices, constraints or calibration that can help us meet these goals. This paper describes in detail issues in developing a calibrated stereo vision system for robotic part manipulation. A robot arm is used to generate calibration data. A highspeed pipeline processor generates correspondences at 4 Hz, using an image warping device to rectify the right image according to the epipolar transformation at frame rates. A highly simplified localization system determines part position to within acceptable tolerances for the robot manipulator within 5-7 seconds. The result is a complete vision/manipulation system for part manipulation.