Local minima are a well-known drawback of image-based visual servoing systems. Up to now, there were no formal guarantees on their number, or even their existence, according to the considered configuration. In this work, a formal approach is presented for the exhaustive computation of all minima and unstable equilibria for a class of six well-known image-based visual servoing controllers. This approach relies on a new polynomial formulation of the equilibrium condition that avoids using the camera pose. By using modern computational algebraic geometry methods and an ad hoc symmetry breaking strategy, the formal resolution of this new equilibrium condition is rendered computationally feasible. The proposed methodology is applied to compute the equilibria of several classical visual servoing tasks, with planar and nonplanar configurations of four and five points. The effects of local minima and saddle points on the dynamics of the system are finally illustrated through intensive simulation results, as well as the effects of image noise and uncertainties on depths.

Solving Direct Shooting Model Predictive Control (MPC) optimization problems online can be computationally expensive if a large horizon is used while also maintaining a dense time sampling. In these cases, it is accepted that tradeoffs between computational load and performances should be sought in order to meet real-time feasibility requirements. However, making the problem more tractable for the hardware should not necessarily imply a decrease in performances. One technique that has been proposed in the literature makes use of control input parameterizations to decrease the numerical complexity of nonlinear MPC problems without necessarily affecting the performances significantly. In this paper, we review the use of parameterizations and propose a simple Sequential Quadratic Programming algorithm for nonlinear MPC. We then benchmark the performances of the solver in simulation, showing that parameterizations allow to attain good performances with (significantly) lower computation times than state-of-the-art solvers.

This article formalizes, under a single common multisensor-based predictive control framework, five different types of parking maneuvers: perpendicular, diagonal for both forward and backward motions, and parallel for backward motions. Since, from a practical point of view, forward parallel parking is usually not advisable, it is not addressed in this work. By moving the effort from motion planning to control, the parking tasks can be completely defined solely from the detected empty parking spots. Additionally, the classical compromise between completeness and computational efficiency when compared to exploration-based path planning techniques is eliminated. The results of a few individual cases are presented and compared against a state-of-the-art path planning approach to illustrate the behavior and performance of the proposed framework as well as results from exhaustive simulations to assess its convergence. As shown in the convergence analyses, the presented approach allows us to park from virtually any sensible initial pose. Finally, real experimentation using a robotized Renault ZOE shows the validity and robustness in the convergence domain of the presented approach.

In this paper, a distributed observer-based approach is proposed to control the longitudinal motion of car-like vehicle platoon moving in an urban environment. To the best of our knowledge, this is the first work presenting an observer-based platoon controller that combines the advantages of high traffic capacity and a minimum number of communication links. To achieve a high traffic flow, a constant-spacing policy is used. However, for that policy, to make platoon string stable, the leader information must be broadcast to all the vehicles. Therefore, we propose a control law in which the predecessor position information is acquired by a sensor-based link while a communication-based link is used to obtain the leader information. Then, an observer is designed and integrated into the control law such that the velocity information of the predecessor can be estimated without the need to communicate with the preceding vehicle. For navigation in urban environments, we present a third order platoon model represented in the curvilinear coordinates. Conditions for asymptotic stability and string stability are given considering the vehicle actuator dynamics and the induced network/sensor time delay. Finally, we provide both simulation and real-time results to validate our approach feasibility and to corroborate our theoretical findings.

Managing multiple robots into a formation can be beneficial, especially in logistics sectors where multiple robots can work together to transport larger loads. This paper presents a consensus control law for formation with navigation and obstacle avoidance of multiple wheeled mobile robots. The formation control is based on adapting a consensus algorithm from flocking, and we propose an obstacle avoidance methodology that ensures the formation while navigating around obstacles. Simulations of the control law using four wheeled mobile robots as well as experiments using actual industrial robots are shown in order to validate the theory.

This paper explores the feasibility of a Multi-Sensor-Based Predictive Control (MSBPC) approach in order to have constraint-based backward non-parallel (perpendicular and diagonal) parking maneuvers capable of dealing with moving pedestrians and, if necessary, performing multiple maneuvers. Our technique relies solely in sensor data expressed relative to the vehicle and therefore no localization is inherently required. Since the proposed approach does not plan any path and instead the controller maneuvers the vehicle directly, the classical path planning related issues are avoided. Real experimentation validates the effectiveness of our approach.

Visual Servoing has been widely investigated in the last decades as it provides a powerful strategy for robot control. Thanks to the direct feed-back from a set of sensors, it allows to reduce the impact of some modeling errors and to perform tasks even in uncertain environments. The commonly exploited approach in this field is to use a model that expresses the rate of change of a set of features as a function of sensor twist. These schemes are commonly used to obtain a velocity command, which needs to be tracked by a low-level controller. Another approach that can be exploited consists in going one step further and to consider an acceleration model for the features. This strategy allows also to obtain a natural and direct link with the dynamic model of the controlled system. This study aims at comparing the use of velocity and acceleration-based models in feed-back linearization for Visual Servoing. We consider the case of a redundant manipulator and discuss what this implies for both control techniques. By means of simulations, we show that controllers based on features acceleration give better results than those based on velocity in presence of noisy feedback signals.

We present an experimental study for the generation of large 3D maps using our CoMapping framework. This framework considers a collaborative approach to efficiently manage, share, and merge maps between vehicles. The main objective of this work is to perform a cooperative mapping for urban and rural environments denied of continuous-GPS service. The study is split in to 2 stages: Pre-Local and Local. In the first stage, each vehicle builds a Pre-Local map of its surroundings in real-time using laser-based measurements, then relocates the map in a global coordinate system using just the low cost GPS data from the first instant of the map construction. In the second stage, vehicles share their pre-local maps, align and merge them in a decentralized way in order to generate more consistent and larger maps, named Local maps. To evaluate performance of all the cooperative system in terms of map alignments, tests are conducted using 3 cars equipped with LiDARs and GPS receiver devices in urban outdoor scenarios of the Ecole Centrale Nantes campus and rural environments.

In this research, a general control framework for platooning in urban environment is proposed. A consensus-based control law is described taking into account the nature of traveling in urban environment, that is the human driven leader travels with variable velocity. In addition, the proposed control law does not depend on the predecessor velocity, which in turn allows us to utilize a low cost limited bandwidth communication module by using a sensor-based link for predecessor distance and a communication-based link for leader's information. A constant-spacing policy is used to get a high capacity flow of vehicles. The control system is analyzed and conditions for both internal and string stability are set. The efficiency of the proposed framework and control law is verified via numerical analysis.

This paper explores the feasibility of a Multi-Sensor-Based Control (MSBC) approach for addressing forward nonparallel (perpendicular and diagonal) unparking problems of car-like vehicles as an alternative to classical approaches (e.g. path planning based, etc.). The results of individual cases are presented to illustrate the behavior and performance of the proposed approach as well as results from exhaustive simulations to evaluate the convergence and stability. The results presented in this work increase the versatility and validity of our MSBC approach towards a fully autonomous parking system.

Sampling-based planning algorithms have been extensively exploited to solve a wide variety of problems. In recent years, many efforts have been dedicated to extend these tools to solve problems involving constraints, such as geometric loop-closure, which lead the valid Configuration Space (CS) to collapse to a lower-dimensional manifold. One proposed solution considers an approximation of the constrained Configuration Space that is obtained by relaxing constraints up to a desired tolerance. The resulting set has then non-zero measure, allowing to exploit classical planning algorithms to search for a path connecting two given states. When constraints involve kinematic loops in the system, relaxation generally bears to undesired contact forces, which need to be compensated during execution by a proper control action. We propose a new tool that exploits relaxation to plan in presence of constraints. Local motions inside the approximated manifold are found as the result of an iterative scheme that uses Quadratic Optimization to proceed towards a new sample without falling outside the relaxed region. By properly guiding the exploration, paths are found with smaller relaxation factors and the need of a dedicated controller to compensate errors is reduced. We complete the analysis by showing the feasibility of the approach with experiments on a real platform.

Accurately localize a mobile vehicle with an easy and quickly deployable system can be very useful for many applications. Herein we present an EKF-SLAM algorithm which allows using radio frequency (RF) beacons without any prior knowledge of their location. As RF beacons provide only range information, recovering their positions is not an easy task. For this range-only SLAM case, a new procedure to instantiate the beacons in the filter is proposed. The method uses two range measurements from different robot's positions to initialize two hypotheses for the beacon's location, which are then integrated in the filter as a Gaussian mixture. This approach is evaluated and compared to other initialization techniques in simulation and with a real dataset. The results show that our approach performs as well as the other existing methods for both trajectory and map errors with a low computational cost.

A framework is presented in this paper for the control of a multisensor robot under several constraints. In this approach, the features coming from several sensors are treated as a single feature vector. The core of our approach is a weighting matrix that balances the contribution of each feature, allowing the taking of constraints into account. The constraints are considered as additional features that are smoothly injected in the control law. Multisensor modeling is introduced for the design of the control law, drawing similarities with linear quadratic control. The main properties are exposed and we propose several strategies to cope with the main drawbacks. The framework is validated in a complex experiment, illustrating various aspects of the approach. The goal is the positioning of a six-DOF robot arm with 3-D visual servoing. The considered constraints are both eye-in-hand and eye-to-hand visibility, together with joint limit avoidance. The system is thus highly overdetermined, yet the task can be performed while ensuring several combinations of constraints.

In this paper we address the problem of visibility in position-based visual servoing. It is well known that the observed object may leave the field of view in such schemes, as there is usually no control in the image. Recent control schemes try to cope with this issue by defining an image-based constraint such that the object stays in the image. We propose to increase the convergence domain of such schemes by defining a new constraint that allows the observed object to leave partially the field of view. The general formulation is exposed and the computation of this constraint is detailed. Experiments show that controlling the visibility loss allows performing position-based visual servoing tasks that were impossible to perform while keeping the whole object in the image.

This paper proposes a general framework to control underwater vehicle-manipulator systems from external sensors. The design of the control law follows several constraints and criteria to optimize the overall behavior. The task is defined in the sensor space, that is suited for visual servoing from camera or sonar images. The control input distribution between the arm and the body is considered and separated between the approach and the intervention phases. The proposed framework is validated in a realistic simulation environment, imposing constant and varying disturbances at the velocity level.

This paper presents a multi-task control method to maintain the visibility of an anatomic element of interest while the doctor tele-operates a 2D ultrasound probe held by a medical robot. The prior task consists in automatically maintaining several visual constraints that guarantee an inter-section between the ultrasound image plane and the anatomic object of interest and the second task allows the medical expert to manually apply probe motion through tele-operation. Unlike classical visual servoing technique which continually regulate the current visual features to desired values, our control approach gradually activates the regulation of one or several ultrasound visual features that go close to fixed limits in such a way to keep them in a safe domain. The main advantage of this approach is to give to the clinician the control of all the degrees of freedom of the probe to examine the patient while automatically preserving the visibility of the element of interest if required. Both simulations and experiments performed on an abdominal phantom demonstrate the efficiency of the visibility assistance task.

A low-level sensor fusion scheme is presented for the positioning of a multi-sensor robot. This non-hierarchical framework can be used for robot arms or other velocity-controlled robots, and is part of the task function approach. A stability analysis is presented for the general case, then several control laws illustrate the versatility of the framework. This approach is applied to the multi-camera eye-in-hand/eye-to-hand configuration in visual servoing. Experimental results point out the feasibility and the effectiveness of the proposed control laws. Mono-camera and multi-camera schemes are compared, showing that the proposed sensor fusion scheme improves the behavior of a robot arm.

Joint limits avoidance is a crucial issue in sensor-based control. In this paper we propose an avoidance strategy based on a low-level data fusion. The joint positions of a robot arm are considered as features that are continuously added to the control scheme when they approach the joint limits, and removed when the position is safe. We expose an optimal tuning of the avoidance scheme, ensuring the main task is disturbed as little as possible. We propose additional strategies to solve the particular cases of unsafe desired position and local minima. The control scheme is applied to the avoidance of joint limits while performing visual servoing. Both simulation and experimental results illustrate the validity of our approach.

In this paper we address the issue of hybrid 2D/3D visual servoing. Contrary to popular approaches, we consider the position-based visual servo as the core of our scheme. 2D information is only added when necessary, that is when the observed object is about to leave from the field of view, even partially. The injection of 2D information is tuned by only one parameter, that simply defines the area where a point is considered to be near to the image border. Simulations allow the comparison of the proposed scheme with both position-based and hybrid schemes, showing nice properties. Finally, experiments are performed on a real object that is tracked by a pose estimation algorithm. Results validate the approach by showing the object stays entirely in the image field of view.

In this paper we propose a new on-line sensor self-calibration framework. The approach is to consider the sensor/robot interaction that links the sensor signal variations to the robot velocity. By on-line calibration, we mean only the actual measurements are used to perform calibration under the condition that the interaction matrix is analytically known. This allows us to propose a very simple and versatile formulation of sensor parameter calibration. Various sensors can be considered, and calibration from different sensory data may be considered within the same process. Intrinsic and extrinsic parameters estimation are formulated as a non-linear minimization problem the Jacobian of which can be expressed analytically from the sensor model. Simulations and experiments are presented for a camera observing four points, showing good results in the case of separated intrinsic and extrinsic calibration, and illustrating the possible limitations in the case of simultaneous estimation.