Robotic grasping is fundamental to many real-world applications, and new approaches must be systematically evaluated. However, in most cases, the performance of a specific approach is assessed by simply counting the number of successful attempts in a given task, and this success rate is then compared to those of other solutions, without taking into account the random variability across different experiments (e.g. due to sensor noise or variations in object placement). In order to address this issue, we classify the observed performance into qualitatively ordered outcomes, thereby stratifying the results. We then show how to analyze these results in a statistical framework, which accounts for the variability between experiments. The advantages of our approach are demonstrated in the practical comparison of four grasp planning algorithms. In particular, we show that the proposed approach allows us to carry out several distinct evaluations from a single set of experiments, without having to repeat the data collection process. We demonstrate that differences between the algorithms, which would not be apparent from overall success rates, can be identified and evaluated.

Depth and surface normal estimation are crucial components in understanding 3D scene geometry from calibrated stereo images. In this paper, we propose visibility and disparity magnitude constraints for slanted patches in the scene. These constraints can be used to associate geometrically feasible planes with each point in the disparity space. The new constraints are validated in the PatchMatch Stereo framework. We use these new constraints not only for initialization, but also in the local plane refinement step of this iterative algorithm. The proposed constraints increase the probability of estimating correct plane parameters, and lead to an improved 3D reconstruction of the scene. Furthermore, the proposed constrained initialization reduces the number of iterations before convergence to the optimal plane parameters. In addition, as most stereo image pairs are not perfectly rectified, we modify the view propagation process by assigning the plane parameters to the neighbors of the candidate pixel. To update the plane parameters in the plane refinement step, we use a gradient free non-linear optimizer. The benefits of the new initialization, propagation, and refinement schemes are demonstrated.

This paper describes the construction of a layered scene model, based on a single hazy image that has sufficient depth variation. A depth map and radiance image are estimated by standard dehazing methods. The radiance image is then segmented into a small number of clusters, and a corresponding scene plane is estimated for each. This provides the basic structure of a layered scene model, without the need for multiple views, or image correspondences. We show that problems of gap filling and depth blending can be addressed systematically, with respect to the layered depth structure. The final models, which resemble cardboard 'pop-ups', are visually convincing. An implementation is described, and subjective depth preferences are tested in a psychophysical experiment.

Visual discomfort is a significant obstacle to the wider use of stereoscopic 3D displays. Many studies have identified the most common causes of discomfort, and a rich body of literature has emerged in recent years with proposed technological and algorithmic solutions. In this paper, we present the first comprehensive review of available image processing methods for reducing discomfort in stereoscopic images and videos. This review covers improved acquisition, disparity re-mapping, adaptive blur, crosstalk cancellation and motion adaptation, as well as improvements in display technology.

This work introduces a new statistical model of visibility in point-sampled scenes, such as those constructed from multiple lidar or depth-camera scans. A visibility density is defined along each optical ray, which gives the probability that a non-occluded scene-point exists at any particular location, with respect to a given camera. The new approach avoids any commitment to a surface-mesh, in order to develop a more data-driven probabilistic model. Furthermore, this approach naturally allows for the existence of gaps and uncertainty in the point-cloud (see fig. 2). The new model has potential applications to multi-view stereo problems, in which visibility is an essential component of the photometric reprojection error [3]. There are other potential applications to the graphical rendering of point-cloud data [2].

This paper addresses the problem of range-stereo fusion, for the construction of high-resolution depth maps. In particular, we combine low-resolution depth data with high-resolution stereo data, in a maximum a posteriori (MAP) formulation. Unlike existing schemes that build on MRF optimizers, we infer the disparity map from a series of local energy minimization problems that are solved hierarchically, by growing sparse initial disparities obtained from the depth data. The accuracy of the method is not compromised, owing to three properties of the data-term in the energy function. First, it incorporates a new correlation function that is capable of providing refined correlations and disparities, via subpixel correction. Second, the correlation scores rely on an adaptive cost aggregation step, based on the depth data. Third, the stereo and depth likelihoods are adaptively fused, based on the scene texture and camera geometry. These properties lead to a more selective growing process which, unlike previous seed-growing methods, avoids the tendency to propagate incorrect disparities. The proposed method gives rise to an intrinsically efficient algorithm, which runs at 3FPS on 2.0 MP images on a standard desktop computer. The strong performance of the new method is established both by quantitative comparisons with state-of-the-art methods, and by qualitative comparisons using real depth-stereo data-sets.

Current approaches to large-scale visual reconstruction would benefit from a statistical model of multiple-view projection. In particular, global constraints based on sceneclutter and occlusion are required. This work presents a new statistical model, starting from the simplest example of a random scene, viewed by two cameras. It has previously been shown that, if the scene is modelled as a Poisson process of identical objects, then the distance to a visible object follows an exponential distribution. But this is not qualitatively realistic, because the mode of the exponential is at zero, implying that the optical centre is fully amid the clutter. Real range-data, in contrast, follows a two-tailed distribution along each ray. A more realistic visibility density is proposed, in which the optical centre is displaced from the Poisson scene by a Gaussian shift. This means that the distance to an un-occluded object is now distributed according to the convolution of the exponential and Gaussian distributions, which has a variable mode. This distribution is re-parameterized along the corresponding epipolar line in another view, via the appropriate Jacobian. The resulting correspondence density gives the prior probability of a binocular match. The parameters of the density are estimated from a data-set of outdoor laser range-scans. This makes it possible to generate full 3D Poisson models that are statistically consistent with the real data. These synthetic scenes are used to further investigate the correspondence density, by Monte Carlo simulation.

An approximately Euclidean representation of the visible scene can be obtained directly from a range, or `time-of-flight', camera. An uncalibrated binocular system, in contrast, gives only a projective reconstruction of the scene. This paper analyzes the geometric mapping between the two representations, without requiring an intermediate calibration of the binocular system. The mapping can be found by either of two new methods, one of which requires point-correspondences between the range and colour cameras, and one of which does not. It is shown that these methods can be used to reproject the range data into the binocular images, which makes it possible to associate high-resolution colour and texture with each point in the Euclidean representation.

The receptive fields of simple cells in the visual cortex can be understood as linear filters. These filters can be modeled by Gabor functions or gaussian derivatives. Gabor functions can also be combined in an energy model of the complex cell response. This letter proposes an alternative model of the complex cell, based on gaussian derivatives. It is most important to account for the insensitivity of the complex response to small shifts of the image. The new model uses a linear combination of the first few derivative filters, at a single position, to approximate the first derivative filter, at a series of adjacent positions. The maximum response, over all positions, gives a signal that is insensitive to small shifts of the image. This model, unlike previous approaches, is based on the scale space theory of visual processing. In particular, the complex cell is built from filters that respond to the 2D differential structure of the image. The computational aspects of the new model are studied in one and two dimensions, using the steerability of the gaussian derivatives. The response of the model to basic images, such as edges and gratings, is derived formally. The response to natural images is also evaluated, using statistical measures of shift insensitivity. The neural implementation and predictions of the model are discussed.

The human visual system obeys Listing's law, which means that the cyclorotation of the eye (around the line of sight) can be predicted from the direction of the fixation point. It is shown here that Listing's law can conveniently be formulated in terms of rotation matrices. The function that defines the observed cyclorotation is derived in this representation. Two polynomial approximations of the function are developed, and the accuracy of each model is evaluated by numerical integration over a range of gaze directions. The error of the simplest approximation for typical eye movements is less than half a degree. It is shown that, given a set of calibrated images, the effect of Listing's law can be simulated in a way that is physically consistent with the original camera. This condition is important for robotic models of human vision, which typically do not reproduce the mechanics of the oculomotor system.

This paper describes the acquisition and content of a new multi-modal database. Some tools for making use of the data streams are also presented. The Computational Audio-Visual Analysis (CAVA) database is a unique collection of three synchronised data streams obtained from a binaural microphone pair, a stereoscopic camera pair and a head tracking device. All recordings are made from the perspective of a person; i.e. what would a human with natural head movements see and hear in a given environment. The database is intended to facilitate research into humans' ability to optimise their multi-modal sensory input and fills a gap by providing data that enables human centred audio-visual scene analysis. It also enables 3D localisation using either audio, visual, or audio-visual cues. A total of 50 sessions, with varying degrees of visual and auditory complexity, were recorded. These range from seeing and hearing a single speaker moving in and out of field of view, to moving around a 'cocktail party' style situation, mingling and joining different small groups of people chatting.

International audience