This paper presents a simple algebraic method to estimate the pose of a camera relative to a planar target from n ≥ 4 reference points with known coordinates in the target frame and their corresponding bearing measurements in the camera frame. The proposed approach follows a hierarchical structure; first, the unit vector normal to the target plane is determined, followed by the camera’s position vector, its distance to the target plane, and finally, the full orientation. To improve the method’s robustness to measurement noise, an averaging methodology is introduced to refine the estimation of the target’s normal direction. The accuracy and robustness of the approach are validated through extensive experiments.

This paper revisits the problem of estimating the attitude, linear velocity and depth of an IMU-Camera with respect to a planar target. The considered solution relies on the measurement of the optical flow (extracted from the continuous homography) complemented with gyrometer and accelerometer measurements. The proposed deterministic observer is accompanied with an observability analysis that points out camera's motion excitation conditions whose satisfaction grants stability of the observer and convergence of the estimation errors to zero. The performance of the observer is illustrated by performing experiments on a testbed IMU-Camera system.

A nonlinear inertial-aided image-based visual servo control approach for the stabilization of (almost) fully actuated autonomous underwater vehicles (AUVs) is proposed. It makes use of the homography matrix between two images of a planar scene as feedback information while the system dynamics are exploited in a cascade manner in a control design: An outer-loop control defines a reference setpoint based on the homography matrix and an inner-loop control ensures the stabilization of the setpoint by assigning the thrust and torque controls. Unlike conventional solutions that only consider the system kinematics, the proposed control scheme is novel in considering the full system dynamics (incorporating all degrees of freedom, nonlinearities, and couplings, as well as interactions with the surrounding fluid) and in not requiring information of the relative depth and normal vector of the observed scene. Augmented with integral corrections, the proposed controller is robust with respect to model uncertainties and disturbances. The almost global asymptotic stability of the closed-loop system is demonstrated, which is the largest domain of attraction one can achieve by means of continuous feedback control. Simulation results illustrating these properties on a realistic AUV model subjected to a sea current are presented and finally experimental results on a real AUV are reported.

This paper addresses the landing problem of a vertical take-off and landing vehicle, exemplified by a quadrotor, on a moving platform using image-based visual servo control. Observable features on a flat and textured target plane are exploited to derive a suitable control law. The target plane may be moving with bounded linear acceleration in any direction. For control purposes, the image of the centroid for a collection of landmarks is used as position measurement, whereas the translational optical flow is used as velocity measurement. The proposed control law guarantees convergence to the desired landing spot on the target plane, without estimating any parameter related to the unknown height, which is also guaranteed to remain strictly positive. Moreover, convergence is guaranteed even in the presence of bounded and possibly time-varying disturbances, resulting, for example, from the motion of the target plane, measurement errors, or wind-induced force disturbances. To improve performance, an estimator for unknown constant force disturbances is also included in the control law. Simulation and experimental results are provided to illustrate and assess the performance of the proposed controller.

This paper presents an intuitive teleoperation scheme to safely operate a wide range of VTOL UAVs by an untrained user in a cluttered environment. This scheme includes a novel force-feedback algorithm that enables the user to feel the texture of the environment. In addition, a novel mapping function is introduced to teleoperate the UAV in an unlimited workspace in position control mode with a joystick which has a limited workspace. An obstacle avoidance strategy is designed to autonomously modify the position set point of the UAV independently of the pilot's commands. The stability analysis of the whole teleoperation loop is provided. Experiments demonstrate the successful teleoperation of a UAV using an haptic joystick and a hexacopter UAV equipped with a 2D laser-range scanner.

The paper concerns visuo-inertial filtering and estimation based on homography and angular velocity measurements, i.e. data obtained from a mono-camera/IMU sensor. We extend recently developed nonlinear filters on the special linear group of homographies to the estimation of scene parameters and velocity of the sensor. A validation of the proposed solution and a comparative evaluation based on real data is presented.

This paper presents the design and evaluation of a nonlinear control scheme for multirotor helicopters that takes first-order drag effects into account explicitly. A dynamic model including the blade flapping and induced drag forces is presented. Based on this model, a hierarchical nonlinear controller is designed to actively compensates for the nonlinear effects these drag forces. Reported simulation and experimental results indicate the significant performance improvement of the proposed drag-augmented control scheme with respect to a conventional nonlinear controller. For completeness, an offline procedure allowing for efficiently identifying the drag parameters is proposed.

This paper presents a nonlinear controller for a vertical take-off and landing (VTOL) unmanned aerial vehicle (UAV) that exploits a measurement optical flow to enable hover and landing control on a moving platform, such as, for example, the deck of a sea-going vessel. The VTOL vehicle is assumed to be equipped with a minimum sensor suite [i.e., a camera and an inertial measurement unit (IMU)], manoeuvring over a textured flat target plane. Two different tasks are considered in this paper. The first concerns the stabilization of the vehicle relative to the moving platform that maintains a constant offset from a moving reference. The second concerns regulation of automatic vertical landing onto a moving platform. Rigorous analysis of system stability is provided, and simulations are presented. Experimental results are provided for a quadrotor UAV to demonstrate the performance of the proposed control strategy.

The paper addresses the control of Vertical Take Off and Landing (VTOL) Underactuated Autonomous Vehicles (UAVs) in hover flight, based on measurements provided by an on-board video camera and rate gyros. The objective is to stabilize the vehicle to the pose associated with a visual image of a planar target. By using the homography matrix computed from the camera measurements of the target, stabilizing feedback laws are derived. Explicit stability conditions on the control parameters are provided. It is shown that very good robustness and performance can be achieved without any apriori information on the visual target (like geometry, or orientation), by a proper tuning of the control parameters. Simulation results confirm the effectiveness of the approach.

This paper presents a general approach for terrain following (including obstacle avoidance) of a vertical take-off and landing vehicle (VTOL) using multiple observation points. The VTOL vehicle is assumed to be a rigid body, equipped with a minimum sensor suite (camera, IMU and barometric altimeter), manoeuvering over a textured rough terrain made of a concatenation of planar surfaces. Assuming that the forward velocity is separately regulated to a desired non-zero value, the proposed control approach ensures terrain following and guarantees the vehicle does not collide with obstacles during the task. The proposed control acquires an optical flow from multiple spatially separate observation points, typically obtained via multiple cameras or non collinear directions in a unique camera. The proposed control algorithm has been tested extensively in simulation and then implemented on a quadrotor UAV to demonstrate the performance of the closed loop system.

This paper presents a nonlinear controller for hovering flight and landing control on a moving platform for a Vertical Take-off and Landing (VTOL) Unmanned Aerial Vehicle (UAV) by exploiting the measurement of the average optical flow. The VTOL vehicle is assumed to be equipped with a minimum sensor suite (a camera and an IMU), manoeuvring over a textured flat target plane. Two different tasks are considered in this paper: the first one concerns the stability of hovering flight and the second one concerns regulation of automatic vertical landing on a moving platform using the divergent optical flow as feedback information. Simulation and experimental results performed on a quad-rotor UAV demonstrate the performance of the proposed control strategy.

This paper presents a nonlinear controller for terrain following of a vertical take-off and landing vehicle (VTOL). The VTOL vehicle is assumed to be a rigid body, equipped with a minimum sensor suite (camera and IMU) along with a measure of the forward speed from another sensor such as global positioning system, maneuvering over a textured terrain made of planar surfaces. Assuming that the forward velocity is separately set to a desired value, the proposed control approach ensures terrain following and guaranties the vehicle does not collide with the ground during the task. The proposed control acquires an optical flow from three spatially separate observation points, typically obtained via three cameras or three non collinear directions in a unique camera. The proposed control algorithm has been tested extensively in simulation and then implemented on a quadrotor UAV to demonstrate the performance of the closed loop system.

A control approach for a class of underactuated vehicles with the objective of stabilizing reference trajectories either in velocity or position is proposed. The basic modeling assumption is that the vehicle is propulsed via a thrust force along a single body-fixed direction and that it has full torque actuation for attitude control (i.e. a typical actuation structure for aircrafts, vertical take-off and landing (VTOL) vehicles, submarines, etc.). Additional assumptions on the external forces applied to the vehicle are also introduced for the sake of control design and stability analyses. They are best satisfied for vehicles whose shape induces lift forces with limited amplitude and which are subjected to an external force field (e.g. gravity), as in the case of VTOL vehicles. The interactions of the vehicle with the surrounding fluid are often difficult to model precisely and they may significantly perturb its motion. Some novel nonlinear feedback control laws are proposed to compensate for modeling errors and perform robustly against such perturbations. Simulation results illustrating these properties on a realistic model of a VTOL vehicle are reported.

In the last decade, many vision-based robot controllers have been designed using Cartesian information encoded in the homography transformation that links two images of a planar object. For any approach, the performance of the closed-loop system depends on the quality of the homography estimates obtained. In this paper, we exploit the special linear Lie-group structure of the set of all homographies to develop a dynamic observer to estimate homographies online. The resulting estimates are effective and can be used to improve closed-loop response of several visual servoing algorithms.

In this paper, we investigate a range of image-based visual servo control algorithms for regulation of the position of a quadrotor aerial vehicle. The most promising control algorithms have been successfully implemented on an autonomous aerial vehicle and demonstrate excellent performance.

This paper considers the question of using a nonlinear complementary filter for attitude estimation of fixed-wing unmanned aerial vehicle (UAV) given only measurements from a low-cost inertial measurement unit. A nonlinear complementary filter is proposed that combines accelerometer output for low frequency attitude estimation with integrated gyrometer output for high frequency estimation. The raw accelerometer output includes a component corresponding to airframe acceleration, occurring primarily when the aircraft turns, as well as the gravitational acceleration that is required for the filter. The airframe acceleration is estimated using a simple centripetal force model (based on additional airspeed measurements), augmented by a first order dynamic model for angle-of-attack, and used to obtain estimates of the gravitational direction independent of the airplane manoeuvres. Experimental results are provided on a real-world data set and the performance of the filter is evaluated against the output from a full GPS/INS that was available for the data set.

This paper presents a nonlinear controller for hovering flight and touchdown control for a vertical take-off and landing (VTOL) unmanned aerial vehicle (UAV) using inertial optical flow. The VTOL vehicle is assumed to be a rigid body, equipped with a minimum sensor suite (camera and IMU), manoeuvring over a textured flat target plane. Two different tasks are considered in this paper: the first concerns the stability of hovering flight and the second one concerns regulation of automatic landing using the divergent optical flow as feedback information. Experimental results on a quad-rotor UAV demonstrate the performance of the proposed control strategy.

An image-based visual servo control is presented for an unmanned aerial vehicle (UAV) capable of stationary or quasi-stationary flight with the camera mounted onboard the vehicle. The target considered consists of a finite set of stationary and disjoint points lying in a plane. Control of the position and orientation dynamics is decoupled using a visual error based on spherical centroid data, along with estimations of the linear velocity and the gravitational inertial direction extracted from image features and an embedded inertial measurement unit. The visual error used compensates for poor conditioning of the image Jacobian matrix by introducing a nonhomogeneous gain term adapted to the visual sensitivity of the error measurements. A nonlinear controller, that ensures exponential convergence of the system considered, is derived for the full dynamics of the system using control Lyapunov function design techniques. Experimental results on a quadrotor UAV, developed by the French Atomic Energy Commission, demonstrate the robustness and performance of the proposed control strategy.

This paper presents a method to design guidance and control laws for small vertical take off and landing unmanned aerial vehicles when no measurement of linear velocity or angular velocity is available. The control strategy is based on the introduction of virtual states in the state equation of the system and allows the design of stabilizing feedback controllers without using any observer. Simulation results are provided for six degrees of freedom model of a small rotorcraft-based unmanned aerial vehicle.

This paper presents a coupled observer that uses accelerometer, gyrometer and vision sensors to provide estimates of pose and linear velocity for an aerial robotic vehicle. The observer is based on a non-linear complimentary filter framework and incorporates adaptive estimates of measurement bias in gyrometers and accelerometers commonly encountered in low-cost inertial measurement systems. Asymptotic stability of the observer estimates is proved as well as bounded energy of the observer error signals. Experimental data is provided for the proposed filter run on data obtained from an experiment involving a remotely controlled helicopter.

An image-based visual servo control is presented for an unmanned aerial vehicle (UAV) capable of stationary or quasi-stationary flight. The proposed control design addresses visual servo of 'eye-in-hand' type systems. The control of the position and orientation dynamics are decoupled using a visual error based on a spherical centroid data, along with estimation of the gravitational inertial direction. The error used compensates for the poor conditioning of the Jacobian matrix seen in earlier work in this area by introducing a non-homogeneous gain term adapted to the visual sensitivity of the error measurements. A nonlinear controller is derived for the full dynamics of the system. Experimental results on an experimental UAV known as an X4-flyer made by the French Atomic Energy Commission (CEA) demonstrate the robustness and performances of the proposed control strategy.

This paper considers the question of obtaining high quality attitude estimates from typical low cost inertial measurement units for applications in control of unmanned aerial vehicles. A nonlinear complimentary filter exploiting the structure of special orthogonal group S0(3) is proposed. The filter is expressed explicitly in terms of direct and untreated measurements. For a typical low cost inertial measurement where two inertial directions are measured (gravitational and magnetic fields) along with angular velocity, it is shown that the filter is well conditioned. If only a single direction is available (typically the gravitational field) along with angular velocity, it is shown that the full gyro bias vector is correctly estimated and that the estimated orientation converges to a set consistent with the measurements. Experimental results, for flight data from the HoverEyecopy UAV, demonstrate the efficiency of the proposed filter

This paper describes a control strategy to stabilize the position of a vertical takeoff and landing (VTOL) unmanned aerial vehicle (UAV) in wind gusts. The proposed approach takes advantage of the cascade structure of the system to design a hierarchical controller. The idea is to separate the controller in a high level controller devoted to position control and a low level controller devoted to stabilization and attitude control. Both controllers are designed by means of backstepping techniques that allow the stabilization of the vehicle's position while on-line estimation of the unknown aerodynamic forces. The global stability of the connected system is proven, and simulations as well as experimental results are presented

Image moments provide an important class of image features used for image-based visual servo control. Spherical image moments have the additional desirable property that they are invariant under rotation of the camera frame. For these features one can study the local and global stability and performance of the position control independently of the rotation control. In this paper we study a range of control algorithms including the classical approximately linearising control, a recently proposed robust control based on Lyapunov function design methodology, and modifications of these designs to improve global and asymptotic performance and robustness of the schemes. The comparison of performance demonstrates that the choice of image feature and control design for image-based visual servo are each equally important and highly coupled. We finally propose a control law using a modified image feature and a Lyapunov control design that ensures global asymptotic stability and good performance equally in image space as in task space

In this paper, we present a control design for the teleoperation of a miniature unmanned aerial vehicle known as an X4-flyer. A simple dynamic nonlinear model for the vehicle, valid for quasi-stationary flight conditions, is derived as a basis for the control design. An attitude control based on information issued from an inertial measurement unit is designed. In order to control the vehicle altitude, an adaptive controller avoiding the ground effects and based on measurements issued from an ultrasonic low cost sensor is designed. In order to compute the altitude velocity, an estimator based on the proposed modelling is used. At the end of the paper, experimental results are presented

This paper deals with the autonomous navigation of a ducted fan vertical take off and landing (VTOL) unmanned aerial vehicle (UAV) in constrained environment. The proposed strategy considers the translational dynamics of the vehicle to design a control of the thrust vector that ensures safe navigation in presence of encountered obstacles. Adaptive backstepping techniques are used to design a nonlinear feedback control that allows to navigate amidst obstacles despite wind perturbations. Simulations results and a first experiment are presented to illustrate the concept

In this paper, a nonlinear complimentary filter (x-estimator) is presented to estimate the attitude of a UAV (unmanned aerial vehicle). The measurements are taken from a low-cost SMU (inertial measurement unit) which consists of 3-axis accelerometers and 3-axis gyroscopes. The gyro bias are estimated online. A second nonlinear complimentary filter (z-estimator) is also designed, it combines 3-axis gyroscope readings with 3-axis magnetometer measurements. From the proposed estimators, the full rotation matrix R will be retrieved. Both estimators use the fact that the orientation matrix, evolving on SO(3), is not locally parameterized and thus could be used to describe any kind of 3D motion. Convergence of the two observers is theoretically proved and simulations as well as experiments are conducted on a real platform in hovering flight conditions.

An image-based "eye-in-hand" visual servo-control design is proposed for underactuated rigid-body dynamics. The dynamic model considered is motivated by recent work on vertical takeoff and landing aerial robotic vehicles. The task considered is that of tracking parallel linear visual features. The proposed design exploits the geometry of the task considered and passivity-like properties of rigid-body dynamics to derive a control Lyapunov function using backstepping techniques.

A pure image-based strategy for visual servo control of a class of dynamic systems is proposed. The proposed design concerns the dynamics of unmanned aerial vehicles capable of quasi-stationary flight (hover and near hover flight). The visual servo control task considered is to locate the camera relative to a stationary target. The paper extends earlier work, by weakening the assumption on the construction of the visual error used. In prior work some inertial information was used in the error construction to guarantee the passivity properties of the control design. In this paper the visual error is defined purely in terms of the 2D image features derived from, the camera signal.

An image-based visual servo control design is proposed for a four rotor vertical take-off and landing (VTOL) craft, known as an X4-flyer, accomplishing a trajectory tracking task. The approach taken is an image-based visual servo design that is applicable to under-actuated dynamic systems. The work is an extension of the authors earlier work (2000) to the trajectory tracking problem. Semiglobal stability of the closed loop system is proved for bounded trajectories.

An image-based visual servo control is presented for a robotic manipulator. The proposed control design addresses visual servo of 'eye-in-hand' type systems. Using a novel representation of the visual error based on a spherical representation of target centroid information along with a measure of the rotation between the camera and the target, the control of the position and orientation is decoupled. A non-linear gain introduced into the orientation feedback kinematics prevents the target image from leaving the visual field. Semi-global convergence of the closed loop system is proved.

In this paper a simplified model and a nonlinear control algorithm for a coaxial helicopter are presented. The model of the helicopter dynamics is obtained in hover conditions. By neglecting the small body forces due to the mechanism used to obtain the torque control, a dynamic controller, based on backstepping techniques, is developed. Some simulation results are presented to illustrate the performance of such a controller.

A new image-based control strategy for visual servoing of a class of under-actuated rigid body systems is presented. The proposed control design applies to "eye-in-hand" systems where the camera is fixed to a rigid body with actuated dynamics. The control design is motivated by a theoretical analysis of the dynamic equations of motion of a rigid body and exploits passivity-like properties of these dynamics to derive a Lyapunov control algorithm using robust backstepping techniques. The proposed control is novel in considering the full dynamic system incorporating all degrees of freedom (albeit for a restricted class of dynamics) and in not requiring measurement of the relative depths of the observed image points. A motivating application is the stabilization of a scale model autonomous helicopter over a marked landing pad.

A control design to stabilise a reduced scale autonomous helicopter equipped with a camera is presented. The proposed algorithm is motivated by recent work in image-based visual servoing control for under-actuated dynamic systems. This work is extended by considering models that contain weakly nonminimum phase zero dynamics. This is an important class of systems since an offset between the camera and the centre of mass for a typical 'dynamic' autonomous vehicle will result in zero dynamics occurring in the image dynamics. In this paper we propose a simplified model of dynamics of the helicopter and show that by placing some constraints in the choice of the position of the camera we can minimise the effect of the zero dynamics.

An image-based 'eye-in-hand' type visual servoing control design is presented for under-actuated rigid body dynamics. The dynamic model considered is often used to model the dynamics of unmanned aerial vehicles (UAV) such as helicopters and aeroplanes. The task considered is that of tracking parallel linear visual features. The proposed design exploits the geometry of the task considered and passivity-like properties of rigid body dynamics to derive a Lyapunov control algorithm using backstepping design techniques.

Classically, closed-loop path-following controller are used to perform the displacement of mobile robots along pre-planned paths. However, when the initial configuration of the robot is far from the reference path, the stabilization process may involve an important transition phase before the robot moves along the path. Though it is possible to guarantee that the robot will not collide with obstacles as it moves closely along the path, it is hard to insure the motion to be safe during the transition phase. The paper presents a path following controller for mobile robots, allowing it to avoid obstacles during the transition phase. The control design is based on sliding mode techniques. We define a time-varying sliding manifold and we use this degree of freedom to modify the shape of the robot's trajectory, during the path-following process. Our technique is applied to the model of a unicycle, making the simplifying hypothesis that the obstacles encountered are strictly convex.

This paper addresses the problem of planning a path following, robust with respect to small localization errors, allowing a mobile robot to reach safely its goal. A path planning method dealing with uncertainties is proposed where both uncertainty in localization and in control of a nonholonomic mobile robot are managed. The safeness of this method is due to the combination of the planning phase and the navigation phase by using the same localization and control processes. The localization process realized the multisensor fusion of data provided by a ring of telemetric sensors in combination with a dead-reckoning system. The robustness is due to a stabilizing controller on which the system remains stable when estimated values are used for feedback instead of exact ones. By determining a compact attractive domain bounding the regulation error, we dynamically compute a security margin during the path planning. This security margin ensures the safeness of the planned path.

The techniques of filtering and merging data coming from several sensors allow to localize a mobile robot in its environment with a precision which can be evaluated. However, as the localization error cannot be neglected, the design of robust closed-loop controller for wheeled robots constitutes a difficult problem. We present here a path following feedback controller robust with respect to localization error. The model is a dynamic extension of the usual kinematic model of a car, in the sense that the path curvature error is considered as a new state variable. The control inputs are respectively the linear velocity and the derivative of the curvature. We determine a variable structure control with sliding mode to stabilize the vehicle's motion around the reference path in the nominal case. Then, we prove that the system remains stable when the state feedback is computed from the estimated values instead of the exact ones. We show that the regulation error is contained in a compact attractive domain when the system has reached its steady state. From this domain, one can easily compute a security margin to guarantee obstacle avoidance during the path following process. Experimental results are presented at the end of the paper.