This paper addresses the problem of vision-based robot control where a reference image defines the equilibrium. Specifically, we consider the class of intensity-based nonmetric solutions, which provide for high accuracy, versatility, and robustness. Existing techniques within that class present either a fully coupled control error dynamics or at best only achieve decoupling of the translational part, i.e., they can only obtain a lower triangular system. These couplings in the system dynamics increase analysis complexity and may degrade system performance. This work proposes a new nonlinear observer for also decoupling the rotational part, i.e., for diagonally decoupling the entire control error dynamics. Theoretical proofs of stability and of those decoupling properties are provided. Improved performances are also experimentally confirmed using synthetic and real data, planar and nonplanar objects, simulating and applying a camera-mounted 6-DoF robotic arm.

This paper considers the problem of vision-based robot stabilization where the equilibrium state is defined via a reference image. Differently from most solutions, this study directly exploits the pixel intensities with no feature extraction or matching and uses only nonmetric information of the observed scene. Intensity-based techniques provide higher accuracy, whereas not requiring metric information increases their versatility. In this context, this paper further exploits the epipolar geometry and its intrinsic degeneracies. Such degeneracies always occur when that stabilization is sufficiently close to the equilibrium, regardless of the object shape. This remarkable fact allows the development of new vision-based control strategies with varying degrees of computational complexity and of prior knowledge. Importantly, they are arranged hierarchically from the simplest to the state-of-the-art ones, all in a unified framework. Three new local methods are then presented, and their closed-loop performances are experimentally assessed using both planar and nonplanar objects, under small and large displacements, simulating and employing a six-degree-of-freedom robotic arm.

This article addresses the problem of direct vision-based robot control where the equilibrium state is defined via a reference image. Direct methods refer to intensity-based nonmetric techniques to perform that stabilization. Intensity-based strategies provide for higher accuracy, whereas not requiring any metric information improves their versatility. However, existing direct techniques either have a coupled error dynamics, or are designed for planar objects only. This paper proposes a new direct technique that decouples the translational motion from the rotational one for the general case of both planar and nonplanar targets under general translational and rotational displacements. Furthermore, for the important case of a fronto-parallel planar object, the proposed technique leads to a fully diagonal interaction matrix. The equilibrium state is made locally exponentially stable for all those cases. These improvements are theoretically proven and experimentally demonstrated using a 6-DoF robotic arm.

This paper addresses the problem of stabilizing a robot at a pose specified via a reference image. Specifically, this paper focuses on six degrees-of-freedom visual servoing techniques that require neither metric information of the observed object nor precise camera and/or robot calibration parameters. Not requiring them improves the flexibility and robustness of servoing tasks. However, existing techniques within the focused class need prior knowledge of the object shape and/or of the camera motion. We present a new visual servoing technique that requires none of the aforementioned information. The proposed technique directly exploits 1) the projective parameters that relate the current image with the reference one and 2) the pixel intensities to obtain these parameters. The level of versatility and accuracy of servoing tasks are, thus, further improved. We also show that the proposed nonmetric scheme allows for path planning. In this way, the domain of convergence is greatly enlarged as well. Theoretical proofs and experimental results demonstrate that visual servoing can, indeed, be highly accurate and robust, despite unknown objects and imaging conditions. This naturally encompasses the cases of color images and illumination changes.

This paper concerns the problem of pose estimation for an inertial-visual sensor. It is well known that IMU bias, and calibration errors between camera and IMU frames can impair the achievement of high-quality estimates through the fusion of visual and inertial data. The main contribution of this work is the design of new observers to estimate pose, IMU bias and camera-to-IMU rotation. The observers design relies on an extension of the so-called passive complementary filter on SO(3). Stability of the observers is established using Lyapunov functions under adequate observability conditions. Experimental results are presented to assess this approach.

To date, there exist only few works on the use of color images for visual servoing. Perhaps, this is due to the difficulties usually found to cope with illumination changes in these images. This paper presents new parametric models and optimization methods for robustly and directly registering color images. Direct methods refer to those that exploit the pixel intensities, without resorting to image features. We then show how a robust and generic visual servoing scheme can be constructed using the obtained optimal parameters. The proposed models ensure robustness to arbitrary illumination changes in color images, do not require prior knowledge (including the spectral ones) of the object, illuminants or camera, and naturally encompass gray-level images. Furthermore, the exploitation of all information within the images, even from areas where no features exist, allow the algorithm to achieve high levels of accuracy. Various results are reported to show that visual servoing can indeed be highly accurate and robust despite unknown objects and unknown imaging conditions.

The majority of visual simultaneous localization and mapping (SLAM) approaches consider feature correspondences as an input to the joint process of estimating the camera pose and the scene structure. In this paper, we propose a new approach for simultaneously obtaining the correspondences, the camera pose, the scene structure, and the illumination changes, all directly using image intensities as observations. Exploitation of all possible image information leads to more accurate estimates and avoids the inherent difficulties of reliably associating features. We also show here that, in this case, structural constraints can be enforced within the procedure as well (instead ofa posteriori), namely the cheirality, the rigidity, and those related to the lighting variations. We formulate the visual SLAM problem as a nonlinear image alignment task. The proposed parameters to perform this task are optimally computed by an efficient second-order approximation method for fast processing and avoidance of irrelevant minima. Furthermore, a new solution to the visual SLAM initialization problem is described whereby no assumptions are made about either the scene or the camera motion. Experimental results are provided for a variety of scenes, including urban and outdoor ones, under general camera motion and different types of perturbations.

In this paper, we investigate how to improve the robustness of visual tracking methods with respect to generic lighting changes. We propose a new approach to the direct image alignment of either Lambertian or non-Lambertian objects under shadows, inter-reflections, glints as well as ambient, diffuse and specular reflections which may vary in power, type, number and space. The method is based on a proposed model of illumination changes together with an appropriate geometric model of image motion. The parameters related to these models are obtained through an efficient second-order optimization technique which minimizes directly the intensity discrepancies. Comparison results with existing direct methods show significant improvements in the tracking performance. Extensive experiments confirm the robustness and reliability of our method.

Traditionally in monocular SLAM, interest features are extracted and matched in successive images. Outliers are rejected a posteriori during a pose estimation process, and then the structure of the scene is reconstructed. In this paper, we propose a new approach for computing robustly and simultaneously the 3D camera displacement, the scene structure and the illumination changes directly from image intensity discrepancies. In this way, instead of depending on particular features, all possible image information is exploited. That problem is solved by using an efficient second-order optimization procedure and thus, high convergence rates and large domains of convergence are obtained. Furthermore, a new solution to the visual SLAM initialization problem is given whereby no assumptions are made either about the scene or the camera motion. The proposed approach is validated on experimental and simulated data. Comparisons with existing methods show significant performance improvements.

Existing visual servoing techniques which do not need metric information require, on the other hand, prior knowledge about the object's shape and/or the camera's motion. In this paper, we propose a new visual servoing technique which does not require any of them. The method is direct in the sense that: the intensity value of all pixels is used (i.e. we avoid the feature extraction step which introduces errors); and that the proposed control error as well as the control law are fully based on image data (i.e. metric measures are neither required nor estimated). Besides not relying on prior information, the scheme is robust to large errors in the camera's internal parameters. We provide the theoretical proofs that the proposed task function is locally isomorphic to the camera pose, that the approach is motion- and shape-independent, and also that the derived control law ensures local asymptotic stability. Furthermore, the proposed control error allows for simple, smooth, physically valid, singularity-free path planning, which leads to a large domain of convergence for the servoing. The approach is validated through various results using objects of different shapes, large initial displacements as well as large errors in the camera's internal parameters.

This work proposes a new vision-based framework to control a robot within model-free large-scale scenes, where the desired pose has never been attained beforehand. Thus, the desired image is not available. It is important to remark that existing visual servoing techniques cannot be applied in this context. The rigid, unknown scene (i.e. the metric model is also not available) is represented as a collection of planar regions, which may leave the field-of-view continuously as the robot moves toward its distant goal. Hence, a novel approach to detect new planes that enter the field-of-view, which is robust to large camera calibration errors, is then deployed here. In fact, it is well-known that representing the scene as composed by planes, the estimation processes are improved in terms of accuracy, stability, and rate of convergence. This extended 3D vision-based control technique is also based on an efficient second-order method for plane-based tracking and pose reconstruction. The framework is validated by using simulated data with artificially created scenes as well as with real images, and accurate navigation tasks are shown

This work presents a method for segmenting image patches which correspond to planar regions in the scene. The method consists of an efficient and robust solution for detecting multiple planar regions in a global optimal sense. Moreover, in contrast with existing techniques which also work on intensity images, neither assumptions about the scene are made nor heuristic hypotheses are formulated. More specifically, the proposed method is based on a systematic, progressive voting procedure from the solution of a linear system, which exploits the two-view geometry. Hence, besides avoiding intermediary depth maps, the progressive mechanism together with such a convergence mapping drastically reduce the computational and storage complexities of the approach. Results from both synthetic and real-world scenes in different scenarios and under various kinds of strong noise confirm its effectiveness and robustness against large camera calibration errors and to the presence of outliers

Addresses the issue of automatic hovering of an outdoor autonomous airship using image-based visual servoing. The hovering controller is designed using a full dynamic model of the airship, in a PD error feedback scheme, taking the visual signals as output and extracted from an on-board camera. The behavior and stability of the airship motion during the task execution and subjected to the wind disturbance is studied. The approach is finally validated in simulation using an accurate airship model.