This paper presents a novel vision-based approach for tracking deformable contour targets using Unmanned Aerial Vehicles (UAVs) through combining image moments descriptor and a Policy Iteration scheme ensuring stability and generalization of knowledge to new tasks. This computationally efficient and optimal control scheme is suitable for diverse dynamic environments such as the surveillance and tracking of targets with evolving features. Due to the ability of the proposed scheme to comprehend an optimization output, the generated control sequence, from an offline successively approximated policy, makes the process less challenging. The proposed methodology is validated through extensive simulations and real-word exper-iments of environmental target surveillance using an octorotor UAV.

This paper introduces a model-based approach for diagnosing actuator faults in an underwater remote operated vehicle (ROV). The method employs structural analysis techniques to perform diagnosability analysis and to generate residuals, aiming to detect actuator faults at an early stage and issue timely warnings. To achieve this objective, we construct the mathematical model of the underwater robot, which facilitates the development of the system's structural model. This approach yields parity equations that serve as residual generators. A key advantage of the proposed method is its ability to provide practical solutions for residual generation in nonlinear systems. The CUSUM algorithm is used to detect changes in residual signals. The underwater remote operated vehicle BlueROV2 serves as the robotic platform for experimentation.

In this work, we present a nonlinear Model Predictive Control (NMPC) scheme for tracking a ground target using a multirotor with a cable-suspended load. The NMPC framework relies on the dynamic model of the UAV with the suspended load and, hence, an estimate of the load state is obtained by fusing the measurements of a downward-facing camera and a load cell with an Unscented Kalman Filter (UKF). Additionally, since the NMPC relies on the future behavior of the system, the trajectory of the ground target throughout the predicted time horizon of the NMPC, is required. Towards this direction, Bézier curves are employed in order to predict the future trajectory of the target, which moves in an arbitrary way. The ultimate goal of the proposed framework is to release the suspended load to the ground target and, consequently, a condition is checked at each time instant that triggers the opening of a gripper, located at the lower edge of the cable. The performance of the proposed control scheme is experimentally validated using an octorotor.

In this paper, we present an event-based control framework for the efficient tracking of contour-based areas, such as road pavements, using a multirotor aerial vehicle equipped with a bio-inspired Dynamic Vision Sensor (DVS). Concerning the detection part, the DVS camera captures events, which are asynchronously fed into a Neuromorphic Hough Transform algorithm running on a SpiNN-3 board and implemented as a Spiking Neural Network (SNN). Next, the asynchronous output of the detection module is fed into an analytically formulated event-based Partitioned Visual Servoing (PVS) algorithm, running on conventional processing hardware, which allows the multirotor to autonomously track and navigate along the detected contour. The proposed architecture achieves efficient tracking of contour-based areas, while constantly maintaining the latter inside the DVS camera's field of view. A set of real-time experiments in various settings employing an octorotor equipped with a downward-looking DVS and a SpiNN-3 board demonstrate the effectiveness of the suggested framework.

In this paper, an Event-triggered Image-based Visual Servoing Nonlinear Model Predictive Controller (ET-IBVS-NMPC) for multirotor aerial vehicles is presented. The proposed scheme is developed for the autonomous surveillance of contour-based areas with different characteristics (e.g. forest paths, coastlines, road pavements). For this purpose, an appropriately trained Deep Neural Network (DNN) is employed for the accurate detection of the contours. In an effort to reduce the remarkably large computational cost required by an IBVS-NMPC algorithm, a triggering condition is designed to define when the Optimal Control Problem (OCP) should be resolved and new control inputs will be calculated. Between two successive triggering instants, the control input trajectory is applied to the robot in an open-loop fashion, which means that no control input computations are required. As a result, the system's computing effort and energy consumption are lowered, while its autonomy and flight duration are increased. The visibility and input constraints, as well as the external disturbances, are all taken into account throughout the control design. The efficacy of the proposed strategy is demonstrated through a series of real-time experiments using a quadrotor and an octorotor both equipped with a monocular downward looking camera.

This paper addresses the problem of water sampling by using a multirotor UAV with a cable-suspended mechanism. In order to ensure the safe execution of the sampling procedure and the stabilization of the vehicle, the disturbances, induced by the water flow and transferred through the cable, have to be identified. Specifically, an estimate of the disturbances is extracted by integrating a depth sensor, a load cell, an ultrasonic sensor and a downward-looking camera into the UAV's sensor suite and fusing the respective measurements. Gaussian Processes are afterwards employed so as to learn the uncertain disturbances in real time and in a non-parametric manner. The predicted disturbances are incorporated into a geometric control scheme which is capable of stabilizing the UAV above the desired sampling position while compensating for the aforementioned disturbances. The performance of the proposed control strategy is demonstrated through both simulation and experimental results.

This paper presents a novel control strategy for the compensation of the slippage effect during non-rigidly grasped object manipulation. A detailed dynamic model of the interconnected system composed of the robotic manipulator, the object and the internal forces and torques induced by the slippage effect is provided. Next, we design a model-based variable impedance control scheme, in order to achieve simultaneously zero convergence for the trajectory tracking error and the slippage velocity of the object. The desired damping and stiffness matrices are formulated online, by taking into account the measurement of the slippage velocity on the contact. A formal Lyapunov-based analysis guarantees the stability and convergence properties of the resulting control scheme. A set of extensive simulation studies clarifies the proposed method and verifies its efficacy.

This paper addresses the problem of cooperative object transportation for multiple Underwater Vehicle Manipulator Systems (UVMSs) in a constrained workspace involving static obstacles. We propose a Nonlinear Model Predictive Control (NMPC) approach for a team of UVMSs in order to transport an object while avoiding significant constraints and limitations such as: kinematic and representation singularities, obstacles within the workspace, joint limits and control input saturations. More precisely, by exploiting the coupled dynamics between the robots and the object, and using certain load sharing coefficients, we design a distributed NMPC for each UVMS in order to cooperatively transport the object within the workspace's feasible region. Moreover, the control scheme adopts load sharing among the UVMSs according to their specific payload capabilities. Additionally, the feedback relies on each UVMS's locally measurements and no explicit data is exchanged online among the robots, thus reducing the required communication bandwidth. Finally, real-time simulation results conducted in UwSim dynamic simulator running in ROS environment verify the efficiency of the theoretical finding.

In this work, we present a decentralized motion planning and control architecture for the cooperative loading task using heterogeneous robotic agents operating in a cluttered workspace with static obstacles. Initially, we tackle the problem of calculating a set of feasible loading configurations via a Probabilistic Road Maps technique. Next, an optimal loading configuration is selected considering the connectivity of the space and the Euclidean distance between the robotic agents. A motion control scheme for each agent is designed and implemented in order to autonomously guide each robot to the desired loading configuration with guaranteed obstacle avoidance and convergence properties. The performance and the applicability of the proposed strategy is experimentally verified in a variety of loading scenarios using a redundant static manipulator and a mobile platform.

In this paper, we propose a visual servoing scheme that imposes predefined performance specifications on the image feature coordinate errors and satisfies the visibility constraints that inherently arise owing to the camera's limited field of view, despite the inevitable calibration and depth measurement errors. Its efficiency is demonstrated via comparative experimental and simulation studies.

This paper presents a novel Nonlinear Model Predictive Control (NMPC) scheme for underwater robotic vehicles operating in a constrained workspace including static obstacles. The purpose of the controller is to guide the vehicle towards specific way points. Various limitations such as: obstacles, workspace boundary, thruster saturation and predefined desired upper bound of the vehicle velocity are captured as state and input constraints and are guaranteed during the control design. The proposed scheme incorporates the full dynamics of the vehicle in which the ocean currents are also involved. Hence, the control inputs calculated by the proposed scheme are formulated in a way that the vehicle will exploit the ocean currents, when these are in favor of the way-point tracking mission which results in reduced energy consumption by the thrusters. The performance of the proposed control strategy is experimentally verified using a 4 Degrees of Freedom (DoF) underwater robotic vehicle inside a constrained test tank with obstacles.

This paper presents the design of a vision-based object grasping and motion control architecture for a mobile manipulator system. The optimal grasping areas of the object are estimated using the partial point cloud acquired from an onboard RGB-D sensor system. The reach-to-grasp motion of the mobile manipulator is handled via a Nonlinear Model Predictive Control scheme. The controller is formulated accordingly in order to allow the system to operate in a constrained workspace with static obstacles. The goal of the proposed scheme is to guide the robot's end-effector towards the optimal grasping regions with guaranteed input and state constraints such as occlusion and obstacle avoidance, workspace boundaries and field of view constraints. The performance of the proposed strategy is experimentally verified using an 8 Degrees of Freedom KUKA Youbot in different reach-to-grasp scenarios.

This paper addresses the fault tolerant control problem for an omni-directional mobile platform with four mecanum wheels moving on a well-known flat and constrained workspace with static obstacles. As a fault, we consider the case where a wheel cannot be actuated and hence it rotates freely around its drive shaft owing to the friction with the flat surface. Depending on the multitude of the faults, a robust motion control scheme is developed that achieves any desired configuration within the operational workspace, avoids collisions with the obstacles and does not violate the workspace boundaries despite the presence of dynamic model uncertainties. The challenge with respect to the current state of the art in fault tolerant control for such mobile platforms, where only one faulty wheel has been considered (i.e., the platform still retains its full actuation capabilities), lies in completely compensating up to two faulty wheels (i.e., the model becomes underactuated in this way) despite the dynamic model uncertainty and the presence of static obstacles in the workspace. Navigation Functions are innovatively incorporated with adaptive control techniques to deal with the parametric uncertainty in the robot dynamics, extending thus greatly the current state of the art in robust motion planning and collision avoidance by studying second order dynamics with parametric uncertainty. Finally, an extensive experimental study clarifies the proposed method and verifies its efficiency in various faults.

Stereo-matching is an indispensable process of dense 3D information extraction for a wide range of applications. Relevant methods rely on cost functions and optimization algorithms for estimating accurate disparities. This work analyses a novel cost for stereo matching under radiometric differences in the stereo-pair, which is based on a modification of the widely used census transformation. It is proposed to define the census on image x and y gradients. The modified census (MC) on gradients is evaluated as an independent matching cost in the presence of severe radiometric differences. For this, the original and the modified census transformation (CT) are implemented in three different aggregation schemes, namely fixed rectangular windows, adaptive cross-based support regions and semi-global matching. It is shown that the MC can provide better results in the cases of local radiometric differences, such as different illumination conditions. Thus, this approach can extend the inherent capability of the original CT to address global monotonic radiometric differences.

This paper proposes an autonomous landing scheme for a small-scale flybarless helicopter equipped with low-cost navigation sensors. The main contribution of this paper is the design of a model-free motion controller that guarantees autonomous landing with prescribed transient and steady state response, despite the presence of external disturbances acting on the vehicle. The proposed control scheme is of low complexity and does not require any knowledge of the helicopter dynamic parameters. Hence, it can be easily implemented in embedded control platforms integrated on small-scale helicopters. In order to provide the controller with accurate estimation of the vehicle's state vector during the landing procedure, an asynchronous sensor fusion and state estimation algorithm, based on an Unscented Kalman Filter (UKF), has been also implemented. The performance and the efficiency of the overall scheme are experimentally verified using a small-scale flybarless helicopter in a real autonomous landing process.

This paper addresses the problem of cooperative object transportation by two nonholonomic wheeled robots, with the coordination relying exclusively on implicit communication. We implement a leader-follower scheme, considering compliant contact between the object and the follower. Only the leader has knowledge of the object's goal configuration. The follower employs force/torque measurements to keep the contact stable and align itself with the object. The control scheme of the follower is based on the prescribed performance methodology guaranteeing thus the satisfaction of certain predefined force/torque constraints. In this way, the overall system acts as a perturbed version of the nominal car-like model. As a result, the leader implements a discontinuous control scheme, that drives robustly the system arbitrarily close to the goal configuration. No explicit data is exchanged among the robots, thus reducing bandwidth and increasing robustness and stealthiness. Finally, the proposed method is experimentally validated using two Pioneer mobile robots interconnected with a rod.

It is well known that a real-time visual servoing task which employs a Visual Tracking Algorithm (VTA) imposes high computational cost to robotic system, which consequently results in higher energy consumption and lower autonomy. Motivated by this fact, this paper presents a novel Image Based Visual Servoing-Model Predictive Control (IBVS-MPC) scheme which is combined with a mechanism that decides when the VTA needs to be triggered and new control inputs must be calculated. Between two consecutive triggering instants, the control input trajectory is applied to the robot in an openloop fashion, i.e, no visual measurements and calculation of the control inputs are required during that period. This results in the reduction of the computational effort, energy consumption and increases the autonomy of the system. These factors are of utmost importance in the case of small autonomous robotic systems which perform vision based tasks, such as surveillance and inspection of indoors and outdoors environments. The visibility and inputs constraints, optimality rate of the MPC, as well as the external disturbances, are being considered during the control design. The efficiency of the proposed scheme is demonstrated through a set of real-time experiments using an eye-in-hand mobile robotic system.

This paper presents a vision-based Leader-Follower cooperative scheme which consists of two underwater vehicles in the absence of explicit communications and direct information interchange. A novel method for implicitly calculating the relative pose between the two vehicles is introduced. The absolute and relative localization algorithm is solely based on the observation of visual features projected on the vehicles common workspace and it is performed in a strictly decentralized manner. The cooperation task consists of a Leader vehicle which is tele-operated in an open-loop fashion while inspecting a flat surface and a Follower vehicle which follows the Leader while keeping a fixed 2D distance offset. The Follower is able to track the Leader's motion at all desired configurations via a motion tracking controller designed accordingly. The proposed control scheme for the Follower is relayed on vision-based implicit communications data and has analytically guaranteed stability and convergence properties. The accuracy and reliability of the implicit vision-based localization system, the performance of the designed control scheme, as well as the efficiency of the overall system in the proposed cooperative task, are experimentally verified using two small Remotely Operated Vehicles (ROVs) in a test tank.

In this paper, we study the decentralized control problem of a two-agent system under local goal specifications given as temporal logic formulas. The agents collaboratively carry an object in a leader-follower scheme and lack means to exchange messages on-line, i.e., to communicate explicitly. Specifically, we propose a decentralized control protocol and a leader re-election strategy that secure the accomplishment of both agents' local goal specifications. The challenge herein lies in exploiting exclusively implicit inter-robot communication that is a natural outcome of the physical interaction of the robots with the object. An illustrative experiment is included clarifying and verifying the approach.

This paper presents a novel Vision-based Nonlinear Model Predictive Control (NMPC) scheme for an under-actuated underwater robotic vehicle. In this scheme, the control loop does not close periodically, but instead a self-triggering framework decides when to provide the next control update. Between two consecutive triggering instants, the control sequence computed by the NMPC is applied to the system in an open-loop fashion, i.e, no state measurements are required during that period. This results to a significant smaller number of requested measurements from the vision system, as well as less frequent computations of the control law, reducing in that way the processing time and the energy consumption. The image constraints (i.e preserving the target inside the camera's field of view), the external disturbances induced by currents and waves, as well as the vehicle's kinematic constraints due to under-actuation, are being considered during the control design. The closed-loop system has analytically guaranteed stability and convergence properties, while the performance of the proposed control scheme is experimentally verified using a small under-actuated underwater vehicle in a test tank.

This paper describes the design and implementation of a robust position tracking control scheme for an Autonomous Underwater Vehicle (AUV). The proposed controller does not require knowledge of the vehicle's dynamic parameters and guarantees prescribed transient and steady state performance despite the presence of external disturbances acting on the vehicle. The resulting scheme is of low complexity and computational cost and thus can be easily integrated to an embedded control platform of an AUV. The proposed control scheme has analytically guaranteed stability and convergence properties, while its applicability and performance are experimentally verified using the Girona500 AUV into two different missions: a) navigation and stabilization to a specific configuration, b) meandrus-like trajectory tracking. In both cases the vehicle was under the influence of time-varying external disturbances caused by a high-pressure water jet installed on the Girona500 manipulator.

Tracking an underwater chain using an autonomous vehicle can be a first step towards more efficient solutions for cleaning and inspecting mooring chains. We propose to use a forward looking sonar as a primary perception sensor to enable the vehicle operation in limited visibility conditions and overcome the turbidity arisen during marine growth removal. Despite its advantages, working with acoustic imagery raises additional challenges to the involved image processing and control methodologies. In this paper we present a robust framework to perform chain following, combining perception, planning and control disciplines. We first introduce a detection system that exploits the sonar's high frame rate and applies local pattern matching to handle the complexity of detecting link chains in acoustic images. Then, a planning system deals with the dispersed detections and determines the link waypoints that the vehicle should reach. Finally, the vehicle is guided through these waypoints using a high level controller that has been tailored to simultaneously traverse the chain and keep track of upcoming links. Experiments on real data demonstrate the capability of autonomously follow a chain with sufficient accuracy to perform subsequent cleaning or inspection tasks.

This paper describes the design and implementation of a visual servo control scheme for an Autonomous Underwater Vehicle (AUV). The purpose of the control scheme is to navigate and stabilize the vehicle towards a visual target. The controller does not utilize the vehicle's dynamic model parameters and guarantees prescribed transient and steady state performance despite the presence of external disturbances representing ocean currents and waves. The proposed control scheme is of low complexity and can be easily integrated to an embedded control platform of an Autonomous Underwater Vehicle (AUV) with limited power and computational resources. Moreover, through the appropriate selection of certain performance functions, the proposed scheme guarantees that the target lies inside the onboard camera's field of view for all time. The resulting control scheme has analytically guaranteed stability and convergence properties, while its applicability and performance are experimentally verified using the Girona 500 AUV.

This paper describes the design and implementation of a model-based sonar servoing control scheme for Autonomous Underwater Vehicles (AUVs). The proposed controller is designed for autonomous surveillance of underwater structures and it is robust against external disturbances and parametric uncertainties in the AUV dynamic model. The sensor suite includes a Multi-beam Imaging Sonar which provides measurements to a RANSAC-based algorithm for structure detection and pose estimation of the vehicle with respect to the structure. The sonar-based pose estimation is properly fused with the rest of the state measurements provided by a navigation module and the resulted state vector is incorporated as feedback to the controller. The proposed control scheme has analytically guaranteed stability and convergence properties, while its applicability and performance are experimentally verified using the Nessie VI AUV in the presence of external disturbances (medium height waves).

We describe the design and implementation of an on-line identification scheme for Autonomous Underwater Vehicles (AUVs). The proposed method estimates the dynamic parameters of the vehicle based on a global derivative-free optimization algorithm. It is not sensitive to initial conditions, unlike other on-line identification schemes, and does not depend on the differentiability of the model with respect to the parameters. The identification scheme consists of three distinct modules: a) System Excitation, b) Metric Calculator and c) Optimization Algorithm. The System Excitation module sends excitation inputs to the vehicle. The Optimization Algorithm module calculates a candidate parameter vector, which is fed to the Metric Calculator module. The Metric Calculator module evaluates the candidate parameter vector, using a metric based on the residual of the actual and the predicted commands. The predicted commands are calculated utilizing the candidate parameter vector and the vehicle state vector, which is available via a complete navigation module. Then, the metric is directly fed back to the Optimization Algorithm module, and it is used to correct the estimated parameter vector. The procedure continues iteratively until the convergence properties are met. The proposed method is generic, demonstrates quick convergence and does not require a linear formulation of the model with respect to the parameter vector. The applicability and performance of the proposed algorithm is experimentally verified using the AUV Girona 500.

This paper presents a visual servoing control scheme that is applied to an underwater robotic vehicle. The objective of the proposed control methodology is to provide a human operator the capability to move the vehicle without loosing the target from the vision system's field of view. On-line estimation of the vehicle states is achieved by fusing data from a Laser Vision System (LVS) and an Inertial Measurement Unit (IMU) using an asynchronous Unscented Kalman Filter (UKF). A controller designed at the kinematic level, is backstepped into the dynamics of the system, maintaining its analytical stability guarantees. It is shown that the under-actuated degree of freedom is input-to-state stable and an energy based shaping of the user input with stability guarantees is implemented. The resulting control scheme has analytically guaranteed stability and convergence properties, while its applicability and performance are experimentally verified using a small Remotely Operated Vehicle (ROV) in a test tank.

This paper presents a novel modification of the Dual Unscented Kalman Filter (DUKF) for the on-line concurrent state and parameter estimation. The developed algorithm is successfully applied to an under-actuated underwater vehicle. Like in the case of conventional DUKF the proposed algorithm demonstrates quick convergence of the parameter vector. In addition, experimental results indicate an increased performance when the proposed methodology is utilized. The applicability and performance of the proposed algorithm is experimentally verified by combining the proposed DUKF with a non-linear controller on a modified Videoray ROV in a test tank. The on-line estimation of the vehicle states and dynamic parameters is achieved by fusing data from a Laser Vision System (LVS) and an Inertial Measurement Unit (IMU).

This paper describes a position-based visual servo control scheme designed for an underwater vehicle. The methodology proposes a path planning technique, which guarantees that a flat target is kept in the camera optical field, while the vehicle avoids collision with the surface the target lays on. The vehicle pose (position and orientation) with respect to the target is obtained using a laser vision system (LVS). The LVS projects two laser dots in the image plane while it tracks the target using computer vision algorithms. The position of each laser dot in the image plane is directly related to the distance between the vehicle and the surface the target is located. The path planning strategy is based on the artificial potential field method (APF). The attractive part of the APF is responsible for minimizing the error between the current vehicle position and the desired. The repulsive part of the APF restricts the target inside the camera optical field while keeps away the laser dots from image regions related to small distances between the vehicle and the surface the target is located. The steering control of the vehicle is achieved by feeding the computed points of the path planning into a Cartesian kinematic controller, which was slightly modified for the needs of the methodology. The overall efficiency of the system, was proved through an extensive experimental procedure, using a small remotely operated vehicle (ROV) in a test tank.