The visual perception system of unmanned surface vessels (USVs) is often subjected to various adversarial attacks (e.g., lens stains, sun glare, ship painting, etc.), impacting the safety of autonomous navigation in maritime environments. To enhance the reliability and robustness of situational awareness in complex environments, we proposed a defensive model to effectively counteract multiple attacks targeting the perception system. Specifically, we first constructed a maritime instance segmentation dataset including various adversarial attack samples, with accurate annotations for the sky, water, land, ships and obstacles. To address the degradation in perception accuracy caused by adversarial attacks, we introduced a Monte Carlo-based random fusion module (MC Fusion) to enhance the adaptability of USVs in various dynamic environments. Additionally, as USVs are always equipped with onboard PC with limited computing resources, we incorporated the lightweight universal inverted bottleneck (UIB) module into the backbone to ensure effective feature extraction while reducing model parameters. Finally, we conducted comparative experiments under various adversarial attack scenarios. Our results demonstrate that, even in the presence of multiple adversarial attacks, our method improves ship detection accuracy by 13.9% and increases the mean accuracy of segmentation masks by over 10% compared to state-of-the-art models, enhancing the safety of USVs in navigation. The source code and datasets are available at https://github.com/huangyanh/ADNet.

Unmanned surface vehicles (USVs) require reliable fault diagnosis to ensure effective human-robot collaboration, yet operators often lack transparent, real-time insights into system failures, undermining trust. This paper presents a novel fault diagnosis system for USVs that enhances human-robot trust and collaboration by integrating model-based and data-driven approaches. The proposed method combines an extended Kalman filter (EKF) for physics-based state estimation with online logistic regression (OLR) for adaptive, real-time fault classification, detecting thruster faults in average 0.5 seconds with 95% accuracy across 40 simulation runs per fault condition. The system demonstrates robust performance even under challenging environmental conditions, maintaining reliable detection and low false positive rates in strong ocean winds up to 15 m/s. A multimodal graphical user interface (GUI) communicates fault probabilities, disturbance trends, and vehicle status, making diagnostic reasoning transparent to operators. Results show the system identifies port and starboard thruster faults with high reliability and fast response. This system illustrates a practical step toward trustworthy, collaborative USV operations, aligning with the growing need for human-centric robotic autonomy.

Intelligent decision-making systems that can solve task allocation problems are critical for multi-robot systems to conduct industrial applications in a collaborative and automated way, such as warehouse inspection using mobile robots, hydrographic surveying using unmanned surface vehicles, etc. This paper, therefore, aims to address the task allocation problem for multi-agent autonomous mobile systems to autonomously and intelligently allocate multiple tasks to a fleet of robots. Such a problem is normally regarded as an independent decision-making process decoupled from the following task planning for the member robots. To avoid the sub-optimal allocation caused by the decoupling, an end-to-end task allocation framework is proposed to tackle this combinatorial optimisation problem while taking the succeeding task planning into account during the optimisation process. The problem is formulated as a special variant of the multi-depot multiple travelling salesmen problem (mTSP). The proposed end-to-end task allocation framework employs deep reinforcement learning methods to replace the handcrafted heuristics used in previous works. The proposed framework features a modular design of the reinforcement learning agent which can be customised for various applications. Moreover, a real-robot implementation setup based on the Robot Operating System 2 is presented to fulfil the simulation-to-reality gap. A warehouse inspection mission is executed to validate the training outcome of the proposed framework. The framework has been cross-validated via both simulated and real-robot tests with various parameter settings, where adaptability and performance are well demonstrated.Note to Practitioners—This paper is motivated by the problem of dispatching a fleet of autonomous mobile robots to tackle a mission that can be resolved into multiple waypoint-following tasks. An end-to-end modular framework is proposed, making task allocation decisions based on the given waypoint information. By using the reinforcement learning technique, the deep neural network could learn sophisticated policies for allocating tasks. The policies are trained in a specific pattern which ensures their joint optimisation for a solver that outputs the near optimal task execution sequences in an efficient way. This leads to a multiple travelling salesmen problem (mTSP) solution. Pre-trained policies are tested in several industrial scenarios reflecting the applications of search and rescue, maritime surveying, and warehouse automation, among others. A hardware implementation configuration based on the Robot Operating System 2 is also presented to support the practical deployment the framework.

This paper introduces a novel sonar odometry system that estimates the relative spatial transformation between two sonar image frames. Considering the unique challenges, such as low resolution and high noise, of sonar imagery for odometry and Simultaneous Localization and Mapping (SLAM), the proposed Direct Imaging Sonar Odometry (DISO) system is designed to estimate the relative transformation between two sonar frames by minimizing the aggregated sonar intensity errors of points with high intensity gradients. Moreover, DISO is implemented to incorporate a multi-sensor window optimization technique, a data association strategy and an acoustic intensity outlier rejection algorithm for reliability and accuracy. The effectiveness of DISO is evaluated using both simulated and real-world sonar datasets, showing that it outperforms the existing geometric-only method on localization accuracy and achieves state-of-the-art sonar odometry performance. We release the source codes of the DISO implementation to the community. The source code is available at https://github.com/SenseRoboticsLab/DISO.

In this paper, we introduce a method to tackle the problem of robot local path planning among pushable objects –an open problem in robotics. In particular, we simultaneously train multiple agents in a physics-based simulation environment, utilizing an Advantage Actor-Critic algorithm coupled with a deep neural network. The developed online policy enables these agents to push obstacles in ways that are not limited to axial alignments, adapt to unforeseen changes in obstacle dynamics instantaneously, and effectively tackle local path planning in confined areas. We tested the method in various simulated environments to prove the adaptation effectiveness to various unseen scenarios in unfamiliar settings. Moreover, we have successfully applied this policy on an actual quadruped robot, confirming its capability to handle the unpredictability and noise associated with real-world sensors and the inherent uncertainties in unexplored object-pushing tasks.

In the past decade, there is an increasing interest in the deployment of unmanned surface vehicles (USVs) for undertaking ocean missions in dynamic, complex maritime environments. The success of these missions largely relies on motion planning algorithms that can generate optimal navigational trajectories to guide a USV. Apart from minimising the distance of a path, when deployed a USVs’ motion planning algorithms also need to consider other constraints such as energy consumption, the affected of ocean currents as well as the fast collision avoidance capability. In this paper, we propose a new algorithm named anisotropic GPMP2 to revolutionise motion planning for USVs based upon the fundamentals of GP (Gaussian process) motion planning (GPMP, or its updated version GPMP2). Firstly, we integrated the anisotropy into GPMP2 to make the generated trajectories follow ocean currents where necessary to reduce energy consumption on resisting ocean currents. Secondly, to further improve the computational speed and trajectory quality, a dynamic fast GP interpolation is integrated in the algorithm. Finally, the new algorithm has been validated on a WAM-V 20 USV in a ROS environment to show the practicability of anisotropic GPMP2. Note to Practitioners—The work reported in this article will be significant for USVs to conduct missions in complex, dynamic maritime environments where various obstacles and time-varying ocean currents exit. We develop this novel motion planning algorithm based on Gaussian process and optimise the trajectory using probabilistic inferences. The new algorithm can generate collision free trajectories that also minimise the influences caused by adverse ocean currents in a highly efficient way. In addition, the planning has been undertaken in a continuous-time domain making the generated trajectory have a guaranteed smoothness and readily feasible for autopilots to track. We use a coastal area with time-varying vortexes to present a challenging practical maritime environment. The presented algorithm integrates the available information about a fluid field regarding energy consumption and hazard level, along with the density of obstacles to plan a navigational route efficiently. To increase the practical performance of the proposed method, diverse models for generating ocean currents need to be developed in the future to tackle unpredictable situations.

This paper introduces a novel deep learning approach to semantic segmentation of the shoreline environments with a high frames-per-second (fps) performance, making the approach readily applicable to autonomous navigation for Unmanned Surface Vehicles (USV). The proposed ShorelineNet is an efficient deep neural network of high performance relying only on visual input. ShorelineNet uses monocular visual input to produce accurate shoreline separation and obstacle detection compared to the state-of-the-art, and achieves this with real-time performance. Experimental validation on a challenging multi-modal maritime obstacle detection dataset, the MODD2 dataset, achieves a much faster inference (25fps on an NVIDIA Tesla K80 and 6fps on a CPU) with respect to the recent state-of-the-art methods, while keeping the performance equally high (73.1% F-score). This makes ShorelineNet a robust and effective model to be used for reliable USV navigation that require real-time and high-performance semantic segmentation of maritime environments.

With the increasing interest in autonomous shipping technology, to ensure ship’s safety it is vital to equip the ship with a navigational algorithm to continuously track the moving vessels in the area of interest and subsequently avoid any collisions. To address this problem, in this paper, an intelligent and robust tracking algorithm using the Kalman filter (KF) and Interacting Multiple Model (IMM) scheme has therefore been designed. The algorithm is able to provide accurate motion information of other ships in constrained environments using least navigational information. The capability of the proposed algorithm has been tested and verified in a number of computer-based simulations.