Marine debris poses an alarming threat to ocean environments. Conventional methods of sea and ocean cleaning rely heavily on manual collection, a process that has repeatedly demonstrated its inefficiency and extensive demand for resources. This paper presents the SeaClear system, a novel multi-robot platform designed to autonomously detect and collect marine debris, thereby offering a more efficient solution to this environmental challenge. An overview of the system is presented, followed by a detailed description of each robot’s capabilities. Leveraging artificial intelligence, the system employs the deep-learning-based computer vision algorithm You Only Look Once (YOLO) for the detection of underwater litter, addressing the challenges of poor visibility and hydrodynamic disturbances of underwater environments. Additionally, the paper explores the implemented navigation and control methodologies, which are an essential part of the workflow of the system. The performance of the designed system is validated via field tests conducted in a real-world underwater environment. Finally, directions for future work are proposed. • The infrastructure of a multi-robot system for autonomous underwater debris detection, mapping, and collection is presented. • The suitability of YOLO-based deep learning for real-time debris detection in shallow waters is validated. • The sensing and control scheme that enables the operation of the multi-robotic platform is presented. • The practical performance of the system is confirmed with field experiments to validate the feasibility of AI-driven underwater operations.

This article addresses the tracking control problem of 3-D trajectories for underactuated underwater robotic vehicles operating in a constrained workspace including obstacles. More specifically, a robust nonlinear model predictive control (NMPC) scheme is presented for the case of underactuated autonomous underwater vehicles (AUVs) (i.e., unicycle-like vehicles actuated only in the surge, heave, and yaw). The purpose of the controller is to steer the unicycle-like AUV to the desired trajectory with guaranteed input and state constraints (e.g., obstacles, predefined vehicle velocity bounds, and thruster saturations) inside a partially known and dynamic environment where the knowledge of the operating workspace is constantly updated via the vehicle’s onboard sensors. In particular, considering the sensing range of the vehicle, obstacle avoidance with any of the detected obstacles is guaranteed by the online generation of a collision-free trajectory tracking path, despite the model dynamic uncertainties and the presence of external disturbances representing ocean currents and waves. Finally, realistic simulation studies verify the performance and efficiency of the proposed framework.Note to Practitioners—This article was motivated by the problem of robust trajectory tracking for an autonomous underwater vehicle (AUV) operating in an uncertain environment where the knowledge of the operating workspace (e.g., obstacle positions) is constantly updated online via the vehicle’s onboard sensors (e.g., multibeam imaging sonars and laser-based vision systems). In addition, there may be other system limitations (e.g., thruster saturation limits) and other operational constraints, induced by the need of various common underwater tasks (e.g., a predefined vehicle speed limit for inspecting the seabed, and mosaicking), where it should also be considered into the control strategy. However, based on the existing trajectory tracking control approaches for underwater robotics, there is a lack of an autonomous control scheme that provides a complete and credible control strategy that takes the aforementioned issues into consideration. Based on this, we present a reliable control strategy that takes into account the aforementioned issues, along with dynamic uncertainties of the model and the presence of ocean currents. In future research, we will extend the proposed methodology for multiple AUV performing collaborative inspection tasks in an uncertain environment.

This paper addresses the problem of cooperative object transportation for multiple Underwater Vehicle Manipulator Systems (UVMSs) in a constrained workspace involving static obstacles. We propose a Nonlinear Model Predictive Control (NMPC) approach for a team of UVMSs in order to transport an object while avoiding significant constraints and limitations such as: kinematic and representation singularities, obstacles within the workspace, joint limits and control input saturations. More precisely, by exploiting the coupled dynamics between the robots and the object, and using certain load sharing coefficients, we design a distributed NMPC for each UVMS in order to cooperatively transport the object within the workspace's feasible region. Moreover, the control scheme adopts load sharing among the UVMSs according to their specific payload capabilities. Additionally, the feedback relies on each UVMS's locally measurements and no explicit data is exchanged online among the robots, thus reducing the required communication bandwidth. Finally, real-time simulation results conducted in UwSim dynamic simulator running in ROS environment verify the efficiency of the theoretical finding.

In this paper, we propose a visual servoing scheme that imposes predefined performance specifications on the image feature coordinate errors and satisfies the visibility constraints that inherently arise owing to the camera's limited field of view, despite the inevitable calibration and depth measurement errors. Its efficiency is demonstrated via comparative experimental and simulation studies.

This paper presents a novel Nonlinear Model Predictive Control (NMPC) scheme for underwater robotic vehicles operating in a constrained workspace including static obstacles. The purpose of the controller is to guide the vehicle towards specific way points. Various limitations such as: obstacles, workspace boundary, thruster saturation and predefined desired upper bound of the vehicle velocity are captured as state and input constraints and are guaranteed during the control design. The proposed scheme incorporates the full dynamics of the vehicle in which the ocean currents are also involved. Hence, the control inputs calculated by the proposed scheme are formulated in a way that the vehicle will exploit the ocean currents, when these are in favor of the way-point tracking mission which results in reduced energy consumption by the thrusters. The performance of the proposed control strategy is experimentally verified using a 4 Degrees of Freedom (DoF) underwater robotic vehicle inside a constrained test tank with obstacles.

This paper presents the design of a vision-based object grasping and motion control architecture for a mobile manipulator system. The optimal grasping areas of the object are estimated using the partial point cloud acquired from an onboard RGB-D sensor system. The reach-to-grasp motion of the mobile manipulator is handled via a Nonlinear Model Predictive Control scheme. The controller is formulated accordingly in order to allow the system to operate in a constrained workspace with static obstacles. The goal of the proposed scheme is to guide the robot's end-effector towards the optimal grasping regions with guaranteed input and state constraints such as occlusion and obstacle avoidance, workspace boundaries and field of view constraints. The performance of the proposed strategy is experimentally verified using an 8 Degrees of Freedom KUKA Youbot in different reach-to-grasp scenarios.

It is well known that a real-time visual servoing task which employs a Visual Tracking Algorithm (VTA) imposes high computational cost to robotic system, which consequently results in higher energy consumption and lower autonomy. Motivated by this fact, this paper presents a novel Image Based Visual Servoing-Model Predictive Control (IBVS-MPC) scheme which is combined with a mechanism that decides when the VTA needs to be triggered and new control inputs must be calculated. Between two consecutive triggering instants, the control input trajectory is applied to the robot in an openloop fashion, i.e, no visual measurements and calculation of the control inputs are required during that period. This results in the reduction of the computational effort, energy consumption and increases the autonomy of the system. These factors are of utmost importance in the case of small autonomous robotic systems which perform vision based tasks, such as surveillance and inspection of indoors and outdoors environments. The visibility and inputs constraints, optimality rate of the MPC, as well as the external disturbances, are being considered during the control design. The efficiency of the proposed scheme is demonstrated through a set of real-time experiments using an eye-in-hand mobile robotic system.

This paper presents a novel Vision-based Nonlinear Model Predictive Control (NMPC) scheme for an under-actuated underwater robotic vehicle. In this scheme, the control loop does not close periodically, but instead a self-triggering framework decides when to provide the next control update. Between two consecutive triggering instants, the control sequence computed by the NMPC is applied to the system in an open-loop fashion, i.e, no state measurements are required during that period. This results to a significant smaller number of requested measurements from the vision system, as well as less frequent computations of the control law, reducing in that way the processing time and the energy consumption. The image constraints (i.e preserving the target inside the camera's field of view), the external disturbances induced by currents and waves, as well as the vehicle's kinematic constraints due to under-actuation, are being considered during the control design. The closed-loop system has analytically guaranteed stability and convergence properties, while the performance of the proposed control scheme is experimentally verified using a small under-actuated underwater vehicle in a test tank.

In this paper, robustness analysis of constrained Image Based Visual Servoing based on Nonlinear Model Predictive Control (NMPC) is presented. It is known, that real applications such an aerial or a fast underwater robotic systems, suffer from the presence of external disturbances. These kinds of disturbances are inevitable in the physical systems, so it is of great interest to employ robust controllers. Therefore, a rigorous robustness analysis should be conducted. In this paper, the Image Based Visual Servoing system under the MPC framework is proven to be Input-to-State Stable (ISS) and a permissible upper bound of the disturbances is provided. Finally, the validity of the theoretic results is illustrated through a simulated example.

In this paper, we propose a novel image based visual servoing scheme that imposes prescribed transient and steady state response on the image feature coordinate errors and satisfies the visibility constraints that inherently arise owing to the limited field of view (FOV) of cameras. Visualizing the aforementioned performance specifications as error bounds, the key idea is to provide an error transformation that converts the original constrained problem into an equivalent unconstrained one, the stabilization of which proves sufficient to achieve prescribed performance guarantees and satisfy the inherent visibility constraints. The performance of the developed scheme is a priori and explicitly imposed by certain designer-specified performance functions, and is fully decoupled by the control gains selection, thus simplifying the control design. Moreover, its computational complexity proves significantly low. It is actually a static scheme involving very few and simple calculations to output the control signal, which enables easily its implementation on fast embedded control platforms. Finally, real-time experiments using an eye-in-hand robotic system verify the theoretical findings.

Tracking an underwater chain using an autonomous vehicle can be a first step towards more efficient solutions for cleaning and inspecting mooring chains. We propose to use a forward looking sonar as a primary perception sensor to enable the vehicle operation in limited visibility conditions and overcome the turbidity arisen during marine growth removal. Despite its advantages, working with acoustic imagery raises additional challenges to the involved image processing and control methodologies. In this paper we present a robust framework to perform chain following, combining perception, planning and control disciplines. We first introduce a detection system that exploits the sonar's high frame rate and applies local pattern matching to handle the complexity of detecting link chains in acoustic images. Then, a planning system deals with the dispersed detections and determines the link waypoints that the vehicle should reach. Finally, the vehicle is guided through these waypoints using a high level controller that has been tailored to simultaneously traverse the chain and keep track of upcoming links. Experiments on real data demonstrate the capability of autonomously follow a chain with sufficient accuracy to perform subsequent cleaning or inspection tasks.